Fuzz Testing

In many cases, a test needs to verify function behavior for multiple possible values. While it is possible to come up with these cases on your own, it is often impractical, especially when you want to test against a large number of possible arguments.

ℹ️ Info Currently, snforge fuzzer only supports using randomly generated values. This way of fuzzing doesn't support any kind of value generation based on code analysis, test coverage or results of other fuzzer runs. In the future, more advanced fuzzing execution modes will be added.

Random Fuzzing

To convert a standard test into a random fuzz test, you need to add parameters to the test function and include the #[fuzzer] attribute. These arguments can then be used in the test body. The test will be run many times against different randomly generated values.

fn sum(a: felt252, b: felt252) -> felt252 {
    return a + b;
}

#[cfg(test)]
mod tests {
    use super::sum;

    #[test]
    #[fuzzer]
    fn test_sum(x: felt252, y: felt252) {
        assert_eq!(sum(x, y), x + y);
    }
}

Then run snforge test like usual.

$ snforge test
Output: 
Collected 2 test(s) from fuzz_testing package
Running 2 test(s) from src/
[PASS] fuzz_testing::with_parameters::tests::test_sum (runs: 22, gas: {max: ~124, min: ~121, mean: ~123.00, std deviation: ~0.90})
[PASS] fuzz_testing::basic_example::tests::test_sum (runs: 256, gas: {max: ~124, min: ~121, mean: ~123.00, std deviation: ~0.81})
Tests: 2 passed, 0 failed, 0 skipped, 0 ignored, 0 filtered out
Fuzzer seed: [..]

Types Supported by the Fuzzer

Fuzzer currently supports generating values for these types out of the box:

  • felt252
  • u8, u16, u32, u64, u128, u256
  • i8, i16, i32, i64, i128
  • ByteArray

To use other types, it is required to implement the Fuzzable trait for them. Providing non-fuzzable types will result in a compilation error.

Fuzzer Configuration

It is possible to configure the number of runs of the random fuzzer as well as its seed for a specific test case:

fn sum(a: felt252, b: felt252) -> felt252 {
    return a + b;
}

#[cfg(test)]
mod tests {
    use super::sum;

    #[test]
    #[fuzzer(runs: 22, seed: 38)]
    fn test_sum(x: felt252, y: felt252) {
        assert_eq!(sum(x, y), x + y);
    }
}

It can also be configured globally, via command line arguments:

$ snforge test --fuzzer-runs 1234 --fuzzer-seed 1111

Or in Scarb.toml file:

# ...
[tool.snforge]
fuzzer_runs = 1234
fuzzer_seed = 1111
# ...