Last Comment Bug 706433 - Add JS shell compile flag for more determinism (differential testing/fuzzing)
: Add JS shell compile flag for more determinism (differential testing/fuzzing)
Product: Core
Classification: Components
Component: JavaScript Engine (show other bugs)
: Trunk
: All All
-- enhancement (vote)
: mozilla11
Assigned To: Christian Holler (:decoder)
: Jason Orendorff [:jorendorff]
Depends on:
Blocks: 751700
  Show dependency treegraph
Reported: 2011-11-30 04:30 PST by Christian Holler (:decoder)
Modified: 2012-06-24 18:32 PDT (History)
6 users (show)
See Also:
Crash Signature:
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---

Patch (2.38 KB, patch)
2011-11-30 04:30 PST, Christian Holler (:decoder)
jorendorff: review+
Details | Diff | Splinter Review
Updated patch (2.36 KB, patch)
2011-12-01 15:38 PST, Christian Holler (:decoder)
choller: review+
Details | Diff | Splinter Review

Description User image Christian Holler (:decoder) 2011-11-30 04:30:26 PST
Created attachment 577920 [details] [diff] [review]

In differential testing we usually compare the output of two shells running with different options (e.g. TI vs. Interp) and there are quite a few functions that behave differently under those options or even completely non-deterministic. While we can remove or wrap most of those functions at runtime, some obstacles remain.

The attached patch does two things: It makes gc() return an empty string (Jesse mentioned on IRC that wrapping gc() could cause problems) and it makes js_ReportOverRecursed print out a notice on stderr when it is called. The reason for that notice is to recognize when a test hits the recursion limit so we can ignore it, as the recursion limits across different options is not consistent.

Both modifications are not enabled unless the --enable-more-deterministic flag is passed at compile time.
Comment 1 User image Jason Orendorff [:jorendorff] 2011-12-01 10:49:39 PST
Comment on attachment 577920 [details] [diff] [review]

Review of attachment 577920 [details] [diff] [review]:

r=me with the nit below.

::: js/src/shell/js.cpp
@@ +1212,5 @@
>      char buf[256];
> +    JS_snprintf(buf, sizeof(buf), "");
> +#else

Just buf[0] = '\0'; here please.
Comment 2 User image Christian Holler (:decoder) 2011-12-01 15:38:34 PST
Created attachment 578429 [details] [diff] [review]
Updated patch

Updated patch to fix the nit. Jason, can you check this in?
Comment 3 User image Christian Holler (:decoder) 2011-12-01 15:48:46 PST
Comment on attachment 578429 [details] [diff] [review]
Updated patch

r+ taken from first patch.
Comment 4 User image Gary Kwong [:gkw] [:nth10sd] 2011-12-01 15:50:45 PST
Landed on m-i:
Comment 5 User image Marco Bonardo [::mak] 2011-12-02 03:24:33 PST

Note You need to log in before you can comment on or make changes to this bug.