1206987 - SpiderMonkey on Windows unable to create crash dumps when they crash

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

Attachment: patch v1

In Q1 of this year, I implemented fuzzing harness support for using crash dumps on Windows [1]. This worked fine till May/June, when I noticed that we were no longer getting the dumps. I dismissed this as something transitional until I started investigating more recently.

I realised that on one of the fuzzing computers on my desk, it was still generating crash dumps for non-SpiderMonkey binaries even till July, so I began suspecting something that landed in SM.

I went into hgweb and found that we were suppressing error dialogs from the SM side [2] because jit-tests were timing out when crashing, causing intermittent failures (bug 1170665 and its patch landing [3]).

Then, I did a test where I #ifndef'ed that code, and sure enough, the crash dumps came back when I ran through it through a known crash (e.g. bug 1205870). Thus, the regressor is bug 1170665.

I have attached a patch which disables that code when we are compiling more-deterministic builds; however, Terrence suggested on IRC that we should #ifndef this to use --fuzzing-safe terminology instead, which might be a better solution.

Setting [fuzzblocker] because this prevents us from getting crash dumps on Windows.

[1] https://msdn.microsoft.com/en-us/library/windows/desktop/bb787181%28v=vs.85%29.aspx
[2] http://hg.mozilla.org/mozilla-central/annotate/197af2fb7e29/js/src/shell/js.cpp#l6172
[3] http://hg.mozilla.org/mozilla-central/rev/617e217524a2

Comment 1

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

We will also have to backport this to the other branches probably as NPOTB.

Comment 2

[Terrence Cole [:terrence]]

Attachment: fix_crash_dialog_suppression-v0.diff

Literally 5 lines below my SetErrorMode call, there is a second one that I have no idea *at all* how I missed when adding the new one. The existing SetErrorMode is guarded by the environment variable XRE_NO_WINDOWS_CRASH_DIALOG. This variable appears to be set by most of the test suites in our tree except for jstests, jit-tests, and jsapi-tests. The right solution here is to merge our duplicate SetErrorMode calls with the environment check and add that environment variable to our test suites.

Comment 4

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

Comment on attachment 8664486 [details] [diff] [review]
fix_crash_dialog_suppression-v0.diff

This works great, thanks!

Comment 5

[Steve Fink [:sfink] [:s:]]

Comment on attachment 8664486 [details] [diff] [review]
fix_crash_dialog_suppression-v0.diff

Review of attachment 8664486 [details] [diff] [review]:
-----------------------------------------------------------------

Sure, seems ok to me.

Comment 6

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/5e9104665b88

Comment 7

[Pulsebot]

Backout:
https://hg.mozilla.org/integration/mozilla-inbound/rev/5b431fc0ca79

Comment 8

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/748829f0a151

Comment 9

[Terrence Cole [:terrence]]

Slurped it to my windows machine and fixed the build there.

Comment 10

[Pulsebot]

Backout:
https://hg.mozilla.org/integration/mozilla-inbound/rev/e6dddf8043e8

Comment 11

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Looks like you also managed to bust some spidermonkey jobs: https://treeherder.mozilla.org/logviewer.html#?job_id=14602539&repo=mozilla-inbound

Comment 12

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

Seems like we're missing an "import os" line in https://hg.mozilla.org/integration/mozilla-inbound/annotate/748829f0a151/js/src/tests/lib/tasks_win.py

Comment 13

[Terrence Cole [:terrence]]

That, and the environment entries need to 'str' and not 'unicode' typed. Appears to work now. Will check in in after I test some more locally.

Comment 14

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/6d452e559186

Comment 15

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Windows XP jit tests have been timing out like https://treeherder.mozilla.org/logviewer.html#?job_id=14720960&repo=mozilla-inbound since this landed.

Backed out in https://hg.mozilla.org/integration/mozilla-inbound/rev/27e9d9dfc369

Comment 16

[Terrence Cole [:terrence]]

Attachment: only_disable_gpf_dialog_during_testing-v0.diff

Now we can be fairly sure that we're getting the right environment in the test suite, so let's use it to do a minimal fix. Try seems to be green on at least one run:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=9b3f009541dd

Comment 17

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

Comment on attachment 8672815 [details] [diff] [review]
only_disable_gpf_dialog_during_testing-v0.diff

Tested that this patch also causes crash dumps to appear as intended.

Comment 18

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/a419ffb1b62f

Comment 19

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/a419ffb1b62f