Like arithfuzz (tracked in bug 465274), this fuzzer compares JIT results to interpreter results. But this one compares the output of entire scripts, not just expressions inside loops. It expects stdout and stderr to be exactly the same, except for the jitstats at the end of stdout.
This fuzzer is a lot slower than arithfuzz and jsfunfuzz, mostly because it keeps restarting ./js. It tests more JS constructs than arithfuzz, but fewer than jsfunfuzz (which does no correctness testing except for decompiler and uneval).
In theory, this fuzzer could compare the behavior of two completely different JS engines if it were lenient about differences in error messages. But I'm currently only testing "js" against "js -j".
Differential testing is now integrated with jsfunfuzz.
A random set of flags is chosen when running jsfunfuzz:
The output is compared against running the same shell with no special flags:
We also check that poking the garbage collector never affects output:
Differential testing excludes Date, Math.random, and a few other things:
These parts of jsfunfuzz are especially designed for differential testing: