680164 - Several subtests in test_reftests_with_caret.html fail on Windows

Status: RESOLVED DUPLICATE of bug 961919

(Core :: Layout, defect)

Description

[Jonathan Griffin (:jgriffin)]

Due to a bug in this test, failures in this test were not being reported in TBPL, perhaps since as long ago as 2010-11-23.  This problem was fixed in bug 678420, which allowed these other problems to become visible.

There are three subtests which nearly always fail on Win 7 opt:

Reftest http://mochi.test:8888/tests/layout/base/tests/bug613807-1.html FAILED
Reftest http://mochi.test:8888/tests/layout/base/tests/bug646382-2.html FAILED
Reftest http://mochi.test:8888/tests/layout/base/tests/bug664087-2.html FAILED

There are additional failures which are intermittent:

Reftest http://mochi.test:8888/tests/layout/base/tests/bug646382-1.html FAILED
Reftest http://mochi.test:8888/tests/layout/base/tests/bug597519-1.html FAILED

and possibly others.

The patch for bug 678420 was backed out until this is addressed in order to avoid a perma orange.  I'd like to comment out the always-fail subtests, file bugs for them and for the intermittent failures, so I can land the patch for bug 678420 again.  Until we do that, all failures in this test case are invisible.

Comment 1

[Jonathan Griffin (:jgriffin)]

Attachment: comment out perma failing tests until they're fixed

Comment 2

[(no longer active)]

Comment on attachment 554143 [details] [diff] [review]
comment out perma failing tests until they're fixed

r=me if you file one bug for each of these failures, CC me on them, and mark each test with the corresponding bug.  Sorry for the total mess.  :(

Comment 4

[Jonathan Griffin (:jgriffin)]

Subtests bug613807-1.html, bug646382-2.html, and bug664087-2.html disabled in http://hg.mozilla.org/mozilla-central/rev/6009974c1e1c, and bugs filed.

Bugs have been filed for the intermittent failures as well, but those haven't been disabled.

I'll land bug 678420 again on Monday, when I have time to watch the tree.

Comment 5

[Phil Ringnalda (:philor)]

Attachment: Expose OS on SpecialPowers

This should look familiar, since I copy-pasted your xpcomabi :)

Comment 6

[Phil Ringnalda (:philor)]

Attachment: Disable the troublemakers, on Windows only

A little tough to stomach disabling this many on every platform just for one problem OS, so I made the test a little less pretty to be able to only skip them on Windows.

Comment 7

[Phil Ringnalda (:philor)]

http://tbpl.allizom.org/?tree=Try&usebuildbot=1&rev=861ee717f60b

Comment 8

[Phil Ringnalda (:philor)]

http://hg.mozilla.org/integration/mozilla-inbound/rev/656f7fbcdb12
http://hg.mozilla.org/integration/mozilla-inbound/rev/9225d92382b8

Not entirely sure whether this is a tracking bug that happened to get some disabling patches in it, and should stay open when those merge, or a "Disable the tests that are failing and file bugs on them" bug and it should be closed.

Comment 9

[Ed Morley [:emorley]]

http://hg.mozilla.org/mozilla-central/rev/656f7fbcdb12
http://hg.mozilla.org/mozilla-central/rev/9225d92382b8

As per comment 8, not sure if this should remain open or not, please close if more appropriate.