Skip to main content

Module properties

Module properties 

Source
Expand description

The forge properties command: several agents look for bugs in the targets independently and in parallel, each in one session in its own copy of the project, and prove each bug with a failing test. Forge runs the tests of every agent again in a fresh copy on every seed, reports a bug only when enough agents agree on its root cause, and keeps one copy of each passing test file. An optional verifier then tries to disprove each reported bug.

Re-exportsΒ§

use crate::cmd::test::RerunFailure; πŸ”’
use crate::result::TestStatus; πŸ”’
use crate::sandbox::RUN_TIMEOUT; πŸ”’
use crate::sandbox::Sandbox; πŸ”’
use crate::sandbox::TestFile; πŸ”’
use crate::sandbox::ensure_inside; πŸ”’
use crate::sandbox::run_session; πŸ”’
use crate::sandbox::stderr; πŸ”’
use crate::sandbox::suite_results; πŸ”’
use crate::workspace; πŸ”’
use alloy_primitives::U256; πŸ”’
use alloy_primitives::keccak256; πŸ”’
use clap::Parser; πŸ”’
use eyre::Context; πŸ”’
use eyre::Result; πŸ”’
use eyre::ensure; πŸ”’
use eyre::eyre; πŸ”’
use foundry_cli::opts::BuildOpts; πŸ”’
use foundry_cli::utils::LoadConfig; πŸ”’
use foundry_common::fs::canonicalize_path; πŸ”’
use foundry_common::sh_println; πŸ”’
use foundry_common::sh_warn; πŸ”’
use foundry_compilers::Graph; πŸ”’
use foundry_compilers::compilers::Language; πŸ”’
use foundry_compilers::compilers::multi::MultiCompilerLanguage; πŸ”’
use foundry_compilers::compilers::multi::MultiCompilerParser; πŸ”’
use foundry_compilers::utils::source_files_iter; πŸ”’
use foundry_config::Config; πŸ”’
use foundry_config::impl_figment_convert; πŸ”’
use foundry_evm::fuzz::CounterExample; πŸ”’
use regex::Regex; πŸ”’
use serde::Deserialize; πŸ”’
use serde::Deserialize; πŸ”’
use serde::Serialize; πŸ”’
use serde::Serialize; πŸ”’
use solar::parse::Cursor; πŸ”’
use std::cmp::Reverse; πŸ”’
use std::collections::BTreeMap; πŸ”’
use std::collections::BTreeSet; πŸ”’
use std::collections::HashSet; πŸ”’
use std::ffi::OsString; πŸ”’
use std::fs; πŸ”’
use std::io; πŸ”’
use std::io::Write; πŸ”’
use std::ops::Add; πŸ”’
use std::path::Path; πŸ”’
use std::path::PathBuf; πŸ”’
use std::process::Command; πŸ”’
use std::process::Output; πŸ”’
use std::process::Stdio; πŸ”’
use std::sync::LazyLock; πŸ”’
use std::sync::atomic::AtomicUsize; πŸ”’
use std::sync::atomic::Ordering; πŸ”’
use std::thread; πŸ”’
use std::time::SystemTime; πŸ”’

ModulesΒ§

matcher πŸ”’
The root-cause matcher: decides which failing tests of different agents prove the same bug.

StructsΒ§

AgentFailure πŸ”’
A test of one agent that fails on every seed.
AgentInputs πŸ”’
The inputs that every agent shares.
BugGroup πŸ”’
The failing tests of all agents that have one root cause.
Campaign πŸ”’
The settings of one forge properties run.
Evidence πŸ”’
A quote from a file that supports a verdict.
Failure πŸ”’
A test that fails on every seed.
Judgment πŸ”’
The verdict file of a verifier.
PropertiesArgs
Run agents that prove bugs with failing tests and report the bugs that they agree on.
Rejected πŸ”’
Files of an agent that Forge did not run.
RunDirs πŸ”’
The output directories that a run reserved. The ones it left empty are removed on drop.
Session πŸ”’
What one agent session left.
TestRun πŸ”’
The tests of some files, run on every seed.
Usage πŸ”’
Model usage that an agent reports.
Verification πŸ”’
What one verifier session left for a bug group.

EnumsΒ§

AgentExit πŸ”’
How an agent command ended.
Verdict πŸ”’
The verdict of a verifier.

ConstantsΒ§

DEFAULT_AGENTS πŸ”’
On 12 held-out cases, four small-model agents with a quorum of two found bugs in as many cases as five (8.6 against 9.0 of 12, a tie in a bootstrap over cases) at four fifths of the cost. With a quorum of two, they kept no report that reviewers labelled wrong, because wrong claims of independent agents rarely repeat.
DEFAULT_FOCUSES πŸ”’
The focus of each agent without --focus, by agent number, wrapping: the four strategy lanes of ultrafuzz’s smoke benchmark (MIT). On an audited vault, five runs of four agents with these focuses found 3.6 distinct real bugs per run against 2.2 without, at a lower cost and with the same hit rate on the audit’s findings. A single agent gets no focus.
DEFAULT_QUORUM πŸ”’
VERIFY_PROMPT πŸ”’
The task of a verifier session. The placeholders are the targets, the group file, the verdict file, and the tests of the group.

StaticsΒ§

BLOCKED πŸ”’
The errors of what Forge blocks in its workspaces: vm.ffi without --allow-ffi, network cheatcodes without --allow-network, env cheatcodes, which see only a few variables, and inline config that --restrict-inline-config refuses.
COMMENT πŸ”’
Solidity line and block comments.
CONTRACT πŸ”’
A contract declaration and its name.

FunctionsΒ§

collect_tests πŸ”’
Returns the Solidity files under the output directory out of a workspace, by path relative to the workspace, in path order. Links are not followed, and out itself must not be one. A file whose path has a glob character is rejected: Forge selects the files to run by a glob.
collected_imports πŸ”’
Returns the collected files that each collected file imports, directly or not.
failure_reason πŸ”’
Appends the counterexample calls to a failure reason.
group_failures πŸ”’
Groups the failing tests of all agents by root cause with the root-cause matcher. Groups with a claim come first, then groups with more agents.
group_files πŸ”’
Returns the files of a group: the files of its failing tests, with the files of the same agent that they import, directly or not.
kept_files πŸ”’
Returns the files whose tests all pass on every seed, with the files that they import. A passing file that imports a file with a failing test is not kept.
normalize_test_content πŸ”’
Compares Solidity tokens without comments or whitespace, preserving literal bytes and token boundaries so different expressions cannot collapse into the same comparison key.
project_fingerprint πŸ”’
Returns the size and modification time of each file that workspaces share with the project, by path relative to the root: the files in the dependencies directories, whose entries workspaces link, and behind links that lead outside the project. Workspaces copy every other file, so a change to one cannot come from an agent. Hidden entries and the build and cache directories are skipped. Each directory is read once.
read_usage πŸ”’
Reads the usage file of an agent or verifier. A missing file is no usage, and an invalid one is a warning.
seed_dir πŸ”’
Returns the directory for one seed’s builds and persisted state in a workspace.
session_prompt πŸ”’
Returns the task of an agent session. On 26 held-out cases, plain independent small-model audits with a task of this kind found 0.05 to 0.10 more planted bugs at every spend level than agents that returned candidate JSON in short rounds.
test_claim πŸ”’
Returns the comment right above a test function, without comment markers, as one line. The function is the first one with this name after the contract declaration and before the next one. Declarations are searched outside comments, so a claim such as the contract pays twice is not a declaration.
test_display_name πŸ”’
Returns Contract::test for a test identity.
test_path πŸ”’
Returns the file of a test identity.
test_results πŸ”’
Returns every test result in Forge’s JSON output, with one result per invariant predicate of a merged invariant campaign.
verify_prompt πŸ”’
Returns the task of a verifier session.

Type AliasesΒ§

Fingerprint πŸ”’
The size and modification time of each project file, by path relative to the root.