1232120 - Fuzzing for entire-Firefox runs / multithreaded coverage fuzzing support

Status: NEW

(Core :: Fuzzing, defect)

Description

[Gian-Carlo Pascutto [:gcp]]

Out of a discussion on ways on how to fuzz the IPC stack (and everything sitting on top of it), we've gathered that the main limitation in using smart, coverage based fuzzers (like afl or llvm's libfuzzer) is that they interact badly with multithreaded binaries, where the non-determinism confuses the fuzzer who can't tell run to run variation from the fuzz modifications activating new codepaths.

Possible approaches:

a) Maybe this problem isn't so bad that parts of Firefox can still be successfully fuzzed by it.

b) Make a fuzzer that knows about what part of the code is actually being fuzzed by the tests, and ignore coverage changes that aren't in that part of the code. This will likely require writing our own or modifying existing fuzzers, but at least LLVM ASAN SanitzerCoverage output looks pretty easy to reuse so it may not be all that difficult.

Comment 1

[Andrew McCreight [:mccr8]]

This sounds vaguely related to techniques used to eliminate nondeterminism for replay debugging (like rr and in bug 1207696). Though it sounds like you need a stronger property, where multiple independent runs are executed similarly if their inputs are similar.

Comment 2

[Gian-Carlo Pascutto [:gcp]]

Yes, the problem attacked is the same, but the needed properties are different. That bug seems to be about ensuring the same input generates the same run. Here we want to run different inputs, but ensure that the difference in executed code is limited to differences caused by the input itself (as opposed to timing).

Note that code that is "covered" by the technique in bug 1207696 would not need to be excluded from analysis by the fuzzer that's being described here, so there's some opportunity to combine the approaches.

Comment 3

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

SpiderMonkey has a --enable-more-deterministic compile-time configuration mode that the fuzzers use to fuzz. The SM folks might be able to give more information as to whether this will help your case.

Comment 4

[Christian Holler (:decoder)]

(In reply to Gian-Carlo Pascutto [:gcp] from comment #0)

> b) Make a fuzzer that knows about what part of the code is actually being
> fuzzed by the tests, and ignore coverage changes that aren't in that part of
> the code.


I already implemented this for AFLFuzz, the code is in my fork of it on Github. Basically my code changes allow you to define a whitelist of source files that will be instrumented during compilation and only these will report back coverage. Since the rest is not instrumented, it both runs faster and doesn't add noise to the fuzzer. I didn't test it on Firefox though, only briefly on the JS shell.