Intermittent failure in test_crashreporter_crash.js | test failed (with xpcshell return code: 0) and false == true

RESOLVED WORKSFORME

Status

()

defect
RESOLVED WORKSFORME
9 years ago
7 years ago

People

(Reporter: philor, Unassigned)

Tracking

({intermittent-failure})

Trunk
x86
Linux
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments, 2 obsolete attachments)

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1276991242.1276992271.20801.gz
Rev3 Fedora 12 mozilla-central opt test xpcshell on 2010/06/19 16:47:22
s: talos-r3-fed-048

TEST-UNEXPECTED-FAIL | /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/test_crashreporter_crash.js | test failed (with xpcshell return code: 0), see following log:
  >>>>>>>
  TEST-INFO | (xpcshell/head.js) | test 1 pending
arg[0] = -g
arg[1] = /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/firefox
arg[2] = -f
arg[3] = /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/crasher_subprocess_head.js
arg[4] = -f
arg[5] = /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/crasher_subprocess_tail.js
TEST-PASS | /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/head_crashreporter.js | [do_crash : 65] 256 != 0
TEST-PASS | /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/head_crashreporter.js | [do_crash : 84] true == true
TEST-PASS | /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/test_crashreporter_crash.js | [anonymous : 10] true == true
TEST-PASS | /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/test_crashreporter_crash.js | [anonymous : 11] true == true
TEST-PASS | /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/test_crashreporter_crash.js | [anonymous : 12] true == true
TEST-UNEXPECTED-FAIL | /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/test_crashreporter_crash.js | false == true - See following stack:
JS frame :: /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/head.js :: do_throw :: line 257
JS frame :: /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/head.js :: do_check_eq :: line 287
JS frame :: /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/head.js :: do_check_true :: line 299
JS frame :: /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/test_crashreporter_crash.js :: anonymous :: line 13
JS frame :: /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/head_crashreporter.js :: do_crash :: line 88
JS frame :: /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/test_crashreporter_crash.js :: run_test :: line 9
JS frame :: /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/head.js :: _execute_test :: line 151
JS frame :: -e :: <TOP_LEVEL> :: line 1
TEST-INFO | (xpcshell/head.js) | exiting test

  <<<<<<<
I'm getting this a lot with bug 568619 branch, and I can reproduce it very consistently on x86-linux if I don't debug it ;-)

AnnotateCrashReport(StartupTime, 1277824175)
  crashReporterAPIData=StartupTime=1277824175

ExceptionHandler::GenerateDump
API string: StartupTime=1277824175


MinidumpCallback, crashReporterAPIData<8397332>:
open

So sometime during generation of the minidump we trash crashReporterAPIData. Attaching a debugger, or adding a busy-sleep to the process initialization seems to fix it.
Holy Moses. We're lazily trying to resolve a symbol (open in this case) and the dynamic loader is trashing memory. I'm not sure *why* it would trash memory at this point, but I think the simple answer here is to force the symbols we need to be early-loaded.
Assignee: nobody → benjamin
Status: NEW → ASSIGNED
Attachment #454878 - Flags: review?(ted.mielczarek)
Attachment #454878 - Flags: review?(ted.mielczarek) → review+
Comment on attachment 454878 [details] [diff] [review]
Resolve open/write early, rev. 1

Yuck, but ok.
bsmedberg: did you land your other patch and not this, thus making it perma-orange?
No, I landed http://hg.mozilla.org/mozilla-central/rev/99ac7d16e904 with the other one and it didn't help. I have now disabled the crashreporter tests on Linux-x86 until I can experiment with using the libc-replacement stuff.
The plugin crash-submission tests were still failing after the band-aid; pushed http://hg.mozilla.org/mozilla-central/rev/567178446a35 to disable those too.
Assignee: benjamin → ted.mielczarek
This backs out the previous patch and the test disabling, and uses the Breakpad wrappers for Linux syscalls, as well as the trivial Breakpad libc helpers.

The tests pass on my x86-64 build, and I'm doing a quick OS X build to ensure that I didn't break that, and I'll also push-to-try to check that this fixes the test bustage.
Attachment #454878 - Attachment is obsolete: true
Comment on attachment 455674 [details] [diff] [review]
Use libc/syscall wrappers on Linux

Works on OS X, pushing to try to see if it fixes the test bustage.
Attachment #455674 - Flags: review?(benjamin)
Comment on attachment 455674 [details] [diff] [review]
Use libc/syscall wrappers on Linux

Tests still fail on try.
Attachment #455674 - Flags: review?(benjamin)
Since this bug is now getting spammed with the x86-64 test failures, I think we should land this patch. This fixes the test failure on my x86-64 machine, but last I tried it it didn't fix x86.
Attachment #455674 - Attachment is obsolete: true
Attachment #462125 - Flags: review?(benjamin)
Attachment #462125 - Flags: review?(benjamin) → review+
May have found related bug: 691632
Assignee: ted.mielczarek → nobody
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → WORKSFORME
Excitingly, we haven't been running these tests at all recently: bug 733501.
Whiteboard: [orange]
You need to log in before you can comment on or make changes to this bug.