diagnose issues with talos-r3-leopard-031 (re-imaged - releng put back)

RESOLVED FIXED

Status

defect
P3
normal
RESOLVED FIXED
8 years ago
6 years ago

People

(Reporter: mbrubeck, Assigned: bhearsum)

Tracking

({intermittent-failure})

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [test which aborts the suite][badslave?][buildduty])

This new failure showed up today on two different branches, but the same slave (talos-r3-leopard-031).  Filing under releng because I suspect this is a problem with the slave.

http://tbpl.allizom.org/php/getParsedLog.php?id=6194204
Rev3 MacOSX Leopard 10.5.8 mozilla-beta debug test reftest on 2011-08-30 09:04:56 PDT for push edf2f673b882

REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/reftest-sanity/needs-focus.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-PASS | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/503531-1.html | image comparison (==)
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/542116-1.html | image comparison (!=)
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/542116-2.html | image comparison (!=)
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/542116-3.html | image comparison (!=)
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/568441.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/598726-1.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/613433-1.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/613433-1.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/613433-1.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/613433-2.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/613433-2.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/613433-2.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/613433-3.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/613433-3.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/613433-3.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/css-placeholder/input/placeholder-focus.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/css-placeholder/input/placeholder-blur.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/css-placeholder/textarea/placeholder-focus.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/css-placeholder/textarea/placeholder-blur.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/css-ui-invalid/default-style/input-focus.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/css-ui-invalid/default-style/button-focus.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/css-ui-invalid/default-style/textarea-focus.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/css-ui-invalid/default-style/select-focus.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/editor/caret_on_textarea_lastline.html | image comparison (!=)
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/editor/input-text-onfocus-reframe.html | load failed: timed out waiting for reftest-wait to be removed
buildbot.slave.commands.TimeoutError: command timed out: 7200 seconds elapsed, attempting to kill

http://tbpl.allizom.org/php/getParsedLog.php?id=6186506
Rev3 MacOSX Leopard 10.5.8 mozilla-central debug test reftest on 2011-08-29 23:11:56 PDT for push b61af4d7dc7c

REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/reftest-sanity/needs-focus.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-PASS | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/503531-1.html | image comparison (==)
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/542116-1.html | image comparison (!=)
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/542116-2.html | image comparison (!=)
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/542116-3.html | image comparison (!=)
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/568441.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/598726-1.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/613433-1.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/613433-1.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/613433-1.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/613433-2.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/613433-2.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/613433-2.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/613433-3.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/613433-3.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/613433-3.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/652301-1a.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/652301-1b.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/css-placeholder/input/placeholder-focus.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/css-placeholder/input/placeholder-blur.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/css-placeholder/textarea/placeholder-focus.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/css-placeholder/textarea/placeholder-blur.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/css-ui-invalid/default-style/input-focus.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/css-ui-invalid/default-style/button-focus.html | load failed: timed out waiting for reftest-wait to be removed
REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/css-ui-invalid/default-style/textarea-focus.html | load failed: timed out waiting for reftest-wait to be removed
buildbot.slave.commands.TimeoutError: command timed out: 7200 seconds elapsed, attempting to kill
There's no dialog box up if that's what you're suspecting. At least, not while it's idle. Would a slow machine (say from a sick disk) be consistent with that failure ?
(In reply to Nick Thomas [:nthomas] from comment #1)
> There's no dialog box up if that's what you're suspecting. At least, not
> while it's idle. Would a slow machine (say from a sick disk) be consistent
> with that failure ?

I have no idea.  Adding cjones who might know the answer.
http://tinderbox.mozilla.org/showlog.cgi?log=UX/1314732852.1314740204.15130.gz as well. I think the only way it would be consistent with a slow disk would be if the slow disk was putting up a dialog, or minimizing the Fx window.

Looking at its recent/, it's been hitting way more than its share of xpcshell runs winding up in bug 667593 today - guessing wildly, maybe a post-crash dialog that's sticking after some reboots, but eventually going away so it's not there now for you to see?
I've disabled talos-r3-leopard-031 until we get a handle on this.
Whiteboard: [orange][test which aborts the suite] → [orange][test which aborts the suite][badslave?]
Priority: -- → P3
Whiteboard: [orange][test which aborts the suite][badslave?] → [orange][test which aborts the suite][badslave?][buildduty]
Moving over to relops to pull this machine and run some diagnostics.
Assignee: nobody → server-ops-releng
Component: Release Engineering → Server Operations: RelEng
QA Contact: release → zandr
Assignee: server-ops-releng → mlarrain
Summary: TEST-UNEXPECTED-FAIL | reftest-sanity/needs-focus.html | load failed: timed out waiting for reftest-wait to be removed, followed by many other failures, on talos-r3-leopard-031 → diagnose issues with talos-r3-leopard-031
colo-trip: --- → scl1
Reimaged the machine and it seems to be responding better. Before it was REALLY sluggish and would lock up. Can we re-enable it and see how it handles.
Status: NEW → ASSIGNED
Assignee: mlarrain → nobody
Component: Server Operations: RelEng → Release Engineering
QA Contact: zandr → release
Status: ASSIGNED → NEW
Priority: P3 → --
Summary: diagnose issues with talos-r3-leopard-031 → diagnose issues with talos-r3-leopard-031 (re-imaged - releng put back)
Priority: -- → P3
Assignee: nobody → bhearsum
Back in production.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Whiteboard: [orange][test which aborts the suite][badslave?][buildduty] → [test which aborts the suite][badslave?][buildduty]
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.