Expand description
The forge properties command: several agents look for bugs in the targets independently and
in parallel, each in one session in its own copy of the project, and prove each bug with a
failing test. Forge runs the tests of every agent again in a fresh copy on every seed, reports a
bug only when enough agents agree on its root cause, and keeps one copy of each passing test
file. An optional verifier then tries to disprove each reported bug.
Re-exportsΒ§
use crate::cmd::test::RerunFailure;πuse crate::result::TestStatus;πuse crate::sandbox::RUN_TIMEOUT;πuse crate::sandbox::Sandbox;πuse crate::sandbox::TestFile;πuse crate::sandbox::ensure_inside;πuse crate::sandbox::run_session;πuse crate::sandbox::stderr;πuse crate::sandbox::suite_results;πuse crate::workspace;πuse alloy_primitives::U256;πuse alloy_primitives::keccak256;πuse clap::Parser;πuse eyre::Context;πuse eyre::Result;πuse eyre::ensure;πuse eyre::eyre;πuse foundry_cli::opts::BuildOpts;πuse foundry_cli::utils::LoadConfig;πuse foundry_common::fs::canonicalize_path;πuse foundry_common::sh_println;πuse foundry_common::sh_warn;πuse foundry_compilers::Graph;πuse foundry_compilers::compilers::Language;πuse foundry_compilers::compilers::multi::MultiCompilerLanguage;πuse foundry_compilers::compilers::multi::MultiCompilerParser;πuse foundry_compilers::utils::source_files_iter;πuse foundry_config::Config;πuse foundry_config::impl_figment_convert;πuse foundry_evm::fuzz::CounterExample;πuse regex::Regex;πuse serde::Deserialize;πuse serde::Deserialize;πuse serde::Serialize;πuse serde::Serialize;πuse solar::parse::Cursor;πuse std::cmp::Reverse;πuse std::collections::BTreeMap;πuse std::collections::BTreeSet;πuse std::collections::HashSet;πuse std::ffi::OsString;πuse std::fs;πuse std::io;πuse std::io::Write;πuse std::ops::Add;πuse std::path::Path;πuse std::path::PathBuf;πuse std::process::Command;πuse std::process::Output;πuse std::process::Stdio;πuse std::sync::LazyLock;πuse std::sync::atomic::AtomicUsize;πuse std::sync::atomic::Ordering;πuse std::thread;πuse std::time::SystemTime;π
ModulesΒ§
- matcher π
- The root-cause matcher: decides which failing tests of different agents prove the same bug.
StructsΒ§
- Agent
Failure π - A test of one agent that fails on every seed.
- Agent
Inputs π - The inputs that every agent shares.
- BugGroup π
- The failing tests of all agents that have one root cause.
- Campaign π
- The settings of one
forge propertiesrun. - Evidence π
- A quote from a file that supports a verdict.
- Failure π
- A test that fails on every seed.
- Judgment π
- The verdict file of a verifier.
- Properties
Args - Run agents that prove bugs with failing tests and report the bugs that they agree on.
- Rejected π
- Files of an agent that Forge did not run.
- RunDirs π
- The output directories that a run reserved. The ones it left empty are removed on drop.
- Session π
- What one agent session left.
- TestRun π
- The tests of some files, run on every seed.
- Usage π
- Model usage that an agent reports.
- Verification π
- What one verifier session left for a bug group.
EnumsΒ§
ConstantsΒ§
- DEFAULT_
AGENTS π - On 12 held-out cases, four small-model agents with a quorum of two found bugs in as many cases as five (8.6 against 9.0 of 12, a tie in a bootstrap over cases) at four fifths of the cost. With a quorum of two, they kept no report that reviewers labelled wrong, because wrong claims of independent agents rarely repeat.
- DEFAULT_
FOCUSES π - The focus of each agent without
--focus, by agent number, wrapping: the four strategy lanes of ultrafuzzβs smoke benchmark (MIT). On an audited vault, five runs of four agents with these focuses found 3.6 distinct real bugs per run against 2.2 without, at a lower cost and with the same hit rate on the auditβs findings. A single agent gets no focus. - DEFAULT_
QUORUM π - VERIFY_
PROMPT π - The task of a verifier session. The placeholders are the targets, the group file, the verdict file, and the tests of the group.
StaticsΒ§
- BLOCKED π
- The errors of what Forge blocks in its workspaces:
vm.ffiwithout--allow-ffi, network cheatcodes without--allow-network, env cheatcodes, which see only a few variables, and inline config that--restrict-inline-configrefuses. - COMMENT π
- Solidity line and block comments.
- CONTRACT π
- A contract declaration and its name.
FunctionsΒ§
- collect_
tests π - Returns the Solidity files under the output directory
outof a workspace, by path relative to the workspace, in path order. Links are not followed, andoutitself must not be one. A file whose path has a glob character is rejected: Forge selects the files to run by a glob. - collected_
imports π - Returns the collected files that each collected file imports, directly or not.
- failure_
reason π - Appends the counterexample calls to a failure reason.
- group_
failures π - Groups the failing tests of all agents by root cause with the root-cause matcher. Groups with a claim come first, then groups with more agents.
- group_
files π - Returns the files of a group: the files of its failing tests, with the files of the same agent that they import, directly or not.
- kept_
files π - Returns the files whose tests all pass on every seed, with the files that they import. A passing file that imports a file with a failing test is not kept.
- normalize_
test_ πcontent - Compares Solidity tokens without comments or whitespace, preserving literal bytes and token boundaries so different expressions cannot collapse into the same comparison key.
- project_
fingerprint π - Returns the size and modification time of each file that workspaces share with the project, by
path relative to the root: the files in the
dependenciesdirectories, whose entries workspaces link, and behind links that lead outside the project. Workspaces copy every other file, so a change to one cannot come from an agent. Hidden entries and the build and cache directories are skipped. Each directory is read once. - read_
usage π - Reads the usage file of an agent or verifier. A missing file is no usage, and an invalid one is a warning.
- seed_
dir π - Returns the directory for one seedβs builds and persisted state in a workspace.
- session_
prompt π - Returns the task of an agent session. On 26 held-out cases, plain independent small-model audits with a task of this kind found 0.05 to 0.10 more planted bugs at every spend level than agents that returned candidate JSON in short rounds.
- test_
claim π - Returns the comment right above a test function, without comment markers, as one line. The
function is the first one with this name after the contract declaration and before the next
one. Declarations are searched outside comments, so a claim such as
the contract pays twiceis not a declaration. - test_
display_ πname - Returns
Contract::testfor a test identity. - test_
path π - Returns the file of a test identity.
- test_
results π - Returns every test result in Forgeβs JSON output, with one result per invariant predicate of a merged invariant campaign.
- verify_
prompt π - Returns the task of a verifier session.
Type AliasesΒ§
- Fingerprint π
- The size and modification time of each project file, by path relative to the root.