442169 - intermittent failures on qm-centos5-moz2-01

Status: RESOLVED DUPLICATE of bug 438871

(Release Engineering :: General, defect, P2)

Description

[(not currently active) Ted Mielczarek]

qm-centos5-moz2-01 has been failing random tests for a good chunk of the day. It's been orange for the last two cycles, timing out on two different tests.

Latest cycle:
*** 776 ERROR FAIL | Test timed out. |  | chrome://mochikit/content/chrome/toolkit/mozapps/downloads/tests/chrome/test_retention_is_0_closes.xul

Previous cycle:
*** 13193 ERROR FAIL | Test timed out. |  | /tests/docshell/test/navigation/test_opener.html
*** 13196 ERROR FAIL | Unable to restore focus, expect failures and timeouts. |  | /tests/docshell/test/navigation/test_popup-navigates-children.html
*** 13203 ERROR FAIL | Unable to restore focus, expect failures and timeouts. |  | /tests/docshell/test/navigation/test_reserved.html
*** 13214 ERROR FAIL | Unable to restore focus, expect failures and timeouts. |  | /tests/docshell/test/navigation/test_sibling-matching-parent.html
*** 13221 ERROR FAIL | Unable to restore focus, expect failures and timeouts. |  | /tests/docshell/test/navigation/test_sibling-off-domain.html
*** 13228 ERROR FAIL | Unable to restore focus, expect failures and timeouts. |  | /tests/docshell/test/test_bug344861.html
*** 13233 ERROR FAIL | Unable to restore focus, expect failures and timeouts. |  | /tests/docshell/test/test_bug369814.html

Prior to that it failed a few times with the same error, but in different tests:
*** 495 ERROR FAIL | Unable to restore focus, expect failures and timeouts. |  | chrome://mochikit/content/chrome/toolkit/content/tests/chrome/test_bug331215.xul

I don't know what's happening here, but maybe it could use a kick or a restart or something.

Comment 1

[matthew zeier [:mrz]]

Not clear which steps here (http://wiki.mozilla.org/Unittest:Linux:Moz2:ITSupport) I'm supposed to follow, punting over.

Comment 2

[Phil Ringnalda (:philor)]

I wondered if my feeling about how awful it is was right, so I ran back through the last 70 builds from it, and within the limits of my losing count, it was green 38 times, and orange when the rest of the tree wasn't 32 times. So basically the only reason the tree hasn't been closed almost 50% of the time since the middle of the day last Thursday is because most people have learned to just completely ignore failure on the only Linux unit test box.

It wasn't clear to me in the "bunch of unit test VMs are starved for RAM" bug, does this one actually have enough?

Comment 3

[(not currently active) Ted Mielczarek]

John: can we get someone to take a look at this box? It's been intermittently orange for days now.

Phil: there's bug 428865, I don't know if that has any answers.

Comment 4

[(not currently active) Ted Mielczarek]

This is causing major problems for Firefox development.

Comment 5

[Mike Schroepfer]

*** 776 ERROR FAIL | Test timed out. |  |
chrome://mochikit/content/chrome/toolkit/mozapps/downloads/tests/chrome/test_retention_is_0_closes.xul

Downloads files - are we sure this box has the appropriate write permissions/etc setup.   Did this test fail sporadically or repeatedly? 

Comment 6

[(not currently active) Ted Mielczarek]

It's failing various tests sporadically, doesn't look like the same test twice in a row, and it cycles green occasionally.

Comment 7

[Mike Schroepfer]

I'd say we need to remove the box from the tree until it is stable - having a semi-random orange is worse than nothing...

Comment 8

[Shawn Wilsher :sdwilsh]

(In reply to comment #5)
> *** 776 ERROR FAIL | Test timed out. |  |
> chrome://mochikit/content/chrome/toolkit/mozapps/downloads/tests/chrome/test_retention_is_0_closes.xul
I recently checked some code in (bug 441716) to make this test more deterministic.  The problem I have been seeing, however, is that when this test fails, there are a series of gdk assertions before it.

Comment 9

[Shawn Wilsher :sdwilsh]

Keep seeing leaks from this machine:
leaked 124036 bytes during test execution 

same amount every time

Comment 10

[(not currently active) Ted Mielczarek]

This bug turns out to not be as useful as individual bugs on specific tests blocking the tracking bug.