Closed Bug 599475 Opened 9 years ago Closed 9 years ago

test_crashreporter_crash.js, test_crashreporter_crash_profile_lock.js, test_punycodeURIs.js fail on 10.5 from a i386+x86_64 universal build

Categories

(Toolkit :: Crash Reporting, defect, critical)

x86
macOS
defect
Not set
critical

Tracking

()

RESOLVED FIXED
mozilla2.0b8
Tracking Status
blocking2.0 --- beta8+

People

(Reporter: nthomas, Assigned: benjamin)

Details

(Keywords: relnote, Whiteboard: [tb33needed])

Attachments

(2 files)

Originally found in bug 598507 comment #19, where there's a log at attachment
477805. We're switching from ppc+i386 universal to i386+x86_64 universal in bug 571367, and are planning to temporarily disable this test for that. Basically we need to make sure we launch the i386 arch of any util the test uses, when running on 10.5

TEST-INFO | /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/xpcshell/tests/toolkit/crashreporter/test/unit/test_crashreporter_crash.js | running test ...
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/xpcshell/tests/toolkit/crashreporter/test/unit/test_crashreporter_crash.js | test failed (with xpcshell return code: 0), see following log:
  >>>>>>>
  TEST-INFO | (xpcshell/head.js) | test 1 pendingarg[0] = -g
arg[1] = /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/Minefield.app/Contents/MacOSarg[2] = -f
arg[3] = /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/xpcshell/tests/toolkit/crashreporter/test/unit/crasher_subprocess_head.js
arg[4] = -f
arg[5] = /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/xpcshell/tests/toolkit/crashreporter/test/unit/crasher_subprocess_tail.js
dyld: unknown required load command 0x80000022
TEST-PASS | /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/xpcshell/tests/toolkit/crashreporter/test/unit/head_crashreporter.js | [do_crash : 65] 256
 != 0TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/xpcshell/tests/toolkit/crashreporter/test/unit/head_crashreporter.js | No minid
ump found! - See following stack:JS frame :: /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/xpcshell/head.js :: do_throw :: line 317
JS frame :: /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/xpcshell/tests/toolkit/crashreporter/test/unit/head_crashreporter.js :: do_crash :: line 8
0
JS frame :: /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/xpcshell/tests/toolkit/crashreporter/test/unit/test_crashreporter_crash.js :: run_test :: 
line 9
JS frame :: /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/xpcshell/head.js :: _execute_test :: line 203
JS frame :: -e :: <TOP_LEVEL> :: line 1
TEST-INFO | (xpcshell/head.js) | exiting test
  <<<<<<<TEST-INFO | /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/xpcshell/tests/toolkit/crashreporter/test/unit/test_crashreporter_crash_profile_lock.js | 
running test ...
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/xpcshell/tests/toolkit/crashreporter/test/unit/test_crashreporter_crash_profile
_lock.js | test failed (with xpcshell return code: 0), see following log:
  >>>>>>>  TEST-INFO | (xpcshell/head.js) | test 1 pending
arg[0] = -garg[1] = /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/Minefield.app/Contents/MacOS
arg[2] = -f
arg[3] = /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/xpcshell/tests/toolkit/crashreporter/test/unit/crasher_subprocess_head.jsarg[4] = -e
arg[5] = ((function () {var env = Components.classes['@mozilla.org/process/environment;1'].getService(Components.interfaces.nsIEnvironment);var profd = env.get("XPCSHELL_TEST_PROFILE_DIR");var dir = Components.classes['@mozilla.org/file/local;1'].createInstance(Components.interfaces.nsILocalFile);dir.initWithPath(profd);var mycrasher = Components.classes['@mozilla.org/testcrasher;1'].createInstance(Components.interfaces.nsITestCrasher);var lock = mycrasher.lockDir(dir);}))();
arg[6] = -f
arg[7] = /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/xpcshell/tests/toolkit/crashreporter/test/unit/crasher_subprocess_tail.js
dyld: unknown required load command 0x80000022
TEST-PASS | /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/xpcshell/tests/toolkit/crashreporter/test/unit/head_crashreporter.js | [do_crash : 65] 256 != 0
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/xpcshell/tests/toolkit/crashreporter/test/unit/head_crashreporter.js | No minidump found! - See following stack:
JS frame :: /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/xpcshell/head.js :: do_throw :: line 317
JS frame :: /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/xpcshell/tests/toolkit/crashreporter/test/unit/head_crashreporter.js :: do_crash :: line 80
JS frame :: /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/xpcshell/tests/toolkit/crashreporter/test/unit/test_crashreporter_crash_profile_lock.js :: run_test :: line 23
JS frame :: /Users/cltbld/talos-slave/mozilla-central_leopard_test-xpcshell/build/xpcshell/head.js :: _execute_test :: line 203
JS frame :: -e :: <TOP_LEVEL> :: line 1
TEST-INFO | (xpcshell/head.js) | exiting test

  <<<<<<<
Yeah, these all want to run xpcshell recursively.

We should probably just add a helper function to the xpcshell harness to run a nested xpcshell instance, and it can do the "arch" magic.
I think it's the same problem with test_punycodeURIs.js. It's not in the log, but on the slave you get a Problem Report dialog when WriteArgument.exe crashes, with the dyld 0x80000022 error. It tries to launch it as 64bit.
Summary: test_crashreporter_crash.js, head_crashreporter.js, test_crashreporter_crash_profile_lock.js fail on 10.5 from a i386+x86_64 universal build → test_crashreporter_crash.js, test_crashreporter_crash_profile_lock.js, test_punycodeURIs.js fail on 10.5 from a i386+x86_64 universal build
Assignee: nobody → joshmoz
blocking2.0: ? → betaN+
"test_crashreporter_crash.js" still causes a timeout on Mac OS X 10.5 opt.

http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTry/1287182105.1287184477.23251.gz&fulltext=1
This patch just re-enables the tests, I'm not actually sure that they will pass yet.
The problem with "test_crashreporter_crash.js" is that it just hangs on 10.5, the full log is:

TEST-INFO | (xpcshell/head.js) | test 1 pending
dyld: lazy symbol binding failed: Symbol not found: _MD5_Update
  Referenced from: /Users/josh/Desktop/firefox-4.0b8pre.en-US.mac64.tests/Minefield.app/Contents/MacOS/XUL
  Expected in: /Users/josh/Desktop/firefox-4.0b8pre.en-US.mac64.tests/Minefield.app/Contents/MacOS/libssl3.dylib

dyld: Symbol not found: _MD5_Update
  Referenced from: /Users/josh/Desktop/firefox-4.0b8pre.en-US.mac64.tests/Minefield.app/Contents/MacOS/XUL
  Expected in: /Users/josh/Desktop/firefox-4.0b8pre.en-US.mac64.tests/Minefield.app/Contents/MacOS/libssl3.dylib
The problem with "test_punycodeURIs.js" is that the "WriteArguments" binary is launched in x86_64 mode despite the fact that it appears to be launched via nsIProcess.
This test failure is consistent with the fact that crash reporting is broken for 10.5 users in beta7.  Disabling tests is pretty dangerous, we keep finding!
Severity: normal → critical
blocking2.0: betaN+ → beta8+
Ted, this appears to work from tryserver. How do you want to handle this with upstream breakpad?
Assignee: joshmoz → benjamin
Status: NEW → ASSIGNED
Attachment #489585 - Flags: review?(ted.mielczarek)
If you can produce a patch against Breakpad SVN, you can submit it for upstream review here:
http://breakpad.appspot.com/
and I'll make sure it gets reviewed and landed.
(you may also have to modify the XCode projects if things aren't linking to the internal md5 code already.)
http://breakpad.appspot.com/229001 filed. Ted, I'm going to hand this to you for integration.
Assignee: benjamin → ted.mielczarek
Status: ASSIGNED → NEW
Comment on attachment 489585 [details] [diff] [review]
Use internal MD5 implementation, rev. 1

We'll sort this out upstream, but we need to get crash reporting working on 10.5 again.
Attachment #489585 - Flags: review?(ted.mielczarek) → review+
Whiteboard: [tb33needs]
Keywords: checkin-needed
http://hg.mozilla.org/mozilla-central/rev/0012ca751ce1
Assignee: ted.mielczarek → benjamin
Status: NEW → RESOLVED
Closed: 9 years ago
Keywords: checkin-needed
Resolution: --- → FIXED
Target Milestone: --- → mozilla2.0b8
Whiteboard: [tb33needs] → [tb33needed]
Comment on attachment 487575 [details] [diff] [review]
basic re-enable v1.0

Pushed this to try http://hg.mozilla.org/try/rev/2c48bfab32a2 to verify all is well with reenabling the tests.
Results: green on these four test suites
Rev3 MacOSX Leopard 10.5.8 tryserver debug test xpcshell
Rev3 MacOSX Leopard 10.5.8 tryserver opt test xpcshell
Rev3 MacOSX Snow Leopard 10.6.2 tryserver debug test xpcshell
Rev3 MacOSX Snow Leopard 10.6.2 tryserver opt test xpcshell

Output:
TEST-INFO | /Users/cltbld/talos-slave/tryserver_snowleopard_test-xpcshell/build/xpcshell/tests/toolkit/crashreporter/test/unit/test_crashreporter_crash.js | running test ...
TEST-PASS | /Users/cltbld/talos-slave/tryserver_snowleopard_test-xpcshell/build/xpcshell/tests/toolkit/crashreporter/test/unit/test_crashreporter_crash.js | test passed
TEST-INFO | /Users/cltbld/talos-slave/tryserver_snowleopard_test-xpcshell/build/xpcshell/tests/toolkit/crashreporter/test/unit/test_crashreporter_crash_profile_lock.js | running test ...
TEST-PASS | /Users/cltbld/talos-slave/tryserver_snowleopard_test-xpcshell/build/xpcshell/tests/toolkit/crashreporter/test/unit/test_crashreporter_crash_profile_lock.js | test passed
...
TEST-INFO | /Users/cltbld/talos-slave/tryserver_snowleopard_test-xpcshell/build/xpcshell/tests/uriloader/exthandler/tests/unit/test_punycodeURIs.js | running test ...
TEST-PASS | /Users/cltbld/talos-slave/tryserver_snowleopard_test-xpcshell/build/xpcshell/tests/uriloader/exthandler/tests/unit/test_punycodeURIs.js | test passed

checkin-needed on attachment 487575 [details] [diff] [review].
Whiteboard: [tb33needed] → [tb33needed] checkin-needed
Comment on attachment 487575 [details] [diff] [review]
basic re-enable v1.0

r=me for re-enabling, just in case someone decides to act on your checkin-needed.
Attachment #487575 - Flags: review+
Whiteboard: [tb33needed] checkin-needed → [tb33needed]
test_punycodeURIs.js is now failing, disabled it again
http://hg.mozilla.org/mozilla-central/rev/da25c74c9166

We can fix that in bug 561350, details of failure there too.
FYI this just got fixed upstream as a separate patch:
http://breakpad.appspot.com/296001
http://code.google.com/p/google-breakpad/source/detail?r=819

There's some deeper analysis on the breakpad issue there.
You need to log in before you can comment on or make changes to this bug.