700503 - all unit test suites should be green on Mac OS X 10.7 (lion)

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[John Ford [:jhford] CET/CEST Berlin Time]

There are failing tests on Mac OS X 10.7 (lion).  We should figure out why they are failing and make them report green.

The test suites that fail on OS X 10.7 opt builds are:
-mochitest 1,2,5 and other
-reftest
-xpcshell

The test suites that fail on OS X 10.7 debug builds are:
-mochitest 1,2,3,5 and other
-crashtests
-reftests
-xpcshell

All other suites and talos pass.

For more information, including brief log snippets about the failures, please see my blog post here: 
http://blog.johnford.info/mac-os-x-10-7-lion-testing-machines-now-online/

This bug is a general tracking bug for getting these tests green as well as the bug that will track unhiding the tests once they are green.

Comment 1

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

I landed https://hg.mozilla.org/mozilla-central/rev/84117219ded0 to annotate the reftest image comparison failures.  There were 4 unexpected passes (now marked as failing on 10.[56] instead of on all Mac) and one unexpected fail (now filed as bug 705044).

Comment 2

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

opt reftest is now green (but I don't know how to unhide it).  Though it's only green because bug 705184 is covering up bug 705047.

Comment 4

[[e-mail] (:bhearsum)]

Because there's so many builders hidden right now there's a huge potential to miss real failures in these test jobs. Can we mark the specific problematic tests as EXPECTED-FAIL or disable them until they're fixed, so that we can have coverage for the remainder of those suites?

Comment 5

[Phil Ringnalda (:philor)]

Intentionally removing some dependencies that are fixed or worked around so that they no longer block being green.

Comment 6

[Phil Ringnalda (:philor)]

Status:

debug mochitest-1, mochitest-3, and crashtest have a plugin leak, bug 705570

xpcshell fails in unit_ipc/test_content_annotation.js, and alas xpcshell manifests don't do runtime version, so it'll have to abort in the test instead of in the manifest where it should, bug 757722

Comment 7

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

See also bug 786084 for getting tests green on 10.8.

Comment 8

[Phil Ringnalda (:philor)]

All green and unhidden on the trunk.