427878 - random mochitest failures in dom-level2-html tests

Status: RESOLVED FIXED

(Core :: DOM: Core & HTML, defect)

Description

[Justin Dolske [:Dolske]]

I've noticed this has happened a number of times on qm-centos5-01, at least over the past few days:

*** 19579 INFO Running /tests/dom/tests/mochitest/dom-level2-html/test_HTMLBodyElement06.html...
*** 19580 INFO PASS | Asize
*** 19581 INFO PASS | Asize
*** 19582 INFO PASS | vLinkLink
*** 19584 INFO Running /tests/dom/tests/mochitest/dom-level2-html/test_HTMLBodyElement07.html...
*** 19585 INFO PASS | isSupportedHTML
*** 19587 INFO Running /tests/dom/tests/mochitest/dom-level2-html/test_HTMLBodyElement08.html...
TinderboxPrint: mochitest<br/>FAIL

Complicating factors here are that the error reporting here is kind of sucky, (no stack/crash data), and although tinderbox goes orange the "brief log" gives no sign of the problem, and you just have to know where to look in the full log.

Nomming for blocking because this is a repeated test problem, and that this looks to be a public w3c test suite.

Not sure who should look at this, or if it's reproducible locally.

Comment 1

[Justin Dolske [:Dolske]]

Looks like this also can happen on Windows (qm-win2k3-01):

*** 19825 INFO Running /tests/dom/tests/mochitest/dom-level2-html/test_HTMLElement07.html...
*** 19826 INFO PASS | Asize
*** 19827 INFO PASS | Asize
*** 19828 INFO PASS | idLink
*** 19830 INFO Running /tests/dom/tests/mochitest/dom-level2-html/test_HTMLElement08.html...
NEXT ERROR *** 19831 ERROR FAIL | Test timed out. | 
*** 19833 INFO Running /tests/dom/tests/mochitest/dom-level2-html/test_HTMLElement09.html...
NEXT ERROR *** 19834 ERROR FAIL | Test timed out. | 
*** 19836 INFO Running /tests/dom/tests/mochitest/dom-level2-html/test_HTMLElement10.html...
*** 19837 ERROR FAIL | Test timed out. | 
*** 19839 INFO Running /tests/dom/tests/mochitest/dom-level2-html/test_HTMLElement100.html...
*** 19840 ERROR FAIL | Test timed out. | 
*** 19841 ERROR FAIL | Too many test timeouts, giving up. | 

Comment 2

[Damon Sicore (:damons)]

Dolske, can you get a regression range?  With this little of information to go on, it's difficult to say that we'd block the release for this.

Please re-nom if this issue escalates.

Comment 3

[Justin Dolske [:Dolske]]

From IRC, Bernd said that this sounded similar to the problems he saw on OS X, where the tests are simply disabled. So it sounds like all platforms have problems with these recently-landed tests... We may want to just disable them entirely [:-(] until a cause can be found/fixed.

Comment 4

[Bernd]

Attachment: until a cause can be found/fixed. ==> Sankt Nimmerlein

Comment 5

[Andrew Schultz]

when I hit this, it seems to be caused by bug 427267.  see http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1208400240.1208404524.25983.gz&fulltext=1

Comment 6

[Justin Dolske [:Dolske]]

Comment on attachment 315044 [details] [diff] [review]
until a cause can be found/fixed.  ==> Sankt Nimmerlein

[Sayrer gave r+ for disabling these tests on IRC. Checking this is because the continual test failures are annoying.]

Checking in dom/tests/mochitest/Makefile.in;
  new revision: 1.14; previous revision: 1.13

Comment 7

[Justin Dolske [:Dolske]]

This hasn't been fixed, the tests were simply disabled.

Someone should investigate *why* they're crashing. It's a W3C standard test suite, so crashing on it is kind of bad.

Comment 8

[Bernd]

They crash only on tinderbox, the only way to investigate is to enable them again but then one would need some more diagnostic then we have now or one would just repeat the same enable disable cycle. 

I do not have expertise with  events and threads that are implied by comment 5.

Comment 9

[Justin Dolske [:Dolske]]

So, I tried reenabling these tests last night, to see what happens. I verified that they pass on a local OS X build.

After running over night, all platforms failed consecutively (6 Linux cycles, 9 OS X cycles, 9 Windows cycles) on just a single test (test_HTMLDocument12.html). I'm going to disable just that test, and leave the rest enabled. I filed bug 471139 for this specific failure.

Hopefully whatever was causing the random failures has been fixed since these tests were initially disabled. It will probably take a few days of test cycles to confirm that, so I'll leave this bug open for now.

Comment 10

[Bernd]

I didn't know that there is a Santa Claus in mozilla land, but I have seen him.

Thanks Justin for turning the tests again on.

Comment 11

[Justin Dolske [:Dolske]]

Haven't seen any signs of these tests causing problems, so let's call this fixed.

Also reenabled on 1.9.1: http://hg.mozilla.org/releases/mozilla-1.9.1/rev/188bfca07db0