419761 - not enough information about random red on windows unittest box

Status: RESOLVED WORKSFORME

(Release Engineering :: General, defect, P3)

Description

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

We just had a random *red* cycle on the windows unittest box that caused us to close the tree for about half an hour.  Something failed at the end of the crashtest run, such that the crashtests finished but no output was "TinderboxPrint"-ed, and since the mochitests were after that, we didn't have any mochitest results at all.

The log is at http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1204071561.1204074316.14651.gz&fulltext=1
and it ended with:

===================================================================
REFTEST PASS (LOAD ONLY): file:///C:/slave/trunk_2k3/mozilla/view/crashtests/382756-1.xul
REFTEST PASS (LOAD ONLY): file:///C:/slave/trunk_2k3/mozilla/widget/src/cocoa/crashtests/397209-1.html
REFTEST PASS (LOAD ONLY): file:///C:/slave/trunk_2k3/mozilla/widget/src/cocoa/crashtests/403296-1.xhtml

<style type="text/css">
    p.error {
      color: red;
===================================================================
  ...style sheet omitted...
===================================================================
      float: left;
    }
</style>
<a href="#tbend">
<p class="error">buildbot.slave.commands.TimeoutError: SIGKILL failed to kill process</p>
</a><div class="stackTrace"></div><a name="tbend"> </a>
<p class="error">buildbot.slave.commands.TimeoutError: SIGKILL failed to kill process</p>
Traceback (most recent call last):
Failure: buildbot.slave.commands.TimeoutError: SIGKILL failed to kill process

No More Errors
===================================================================

There *may* have been a real browser problem here (i.e., a crash), but there's not enough information to tel.  In particular:
 * did the process terminate before it was expected to and there's no process left to kill?  (That doesn't make sense, since the browser should quit on its own when it's done the crashtest run.)
 * did the process hang (i.e., fail to finish by some timeout), and then buildbot is trying (and failing) to kill it?

More diagnostics here would be useful:  why is buildbot trying to kill the process, and why is it failing?  (Does the process not exist anymore?)  Or is this really just the output that shows up when the process crashes?

[26 17:48:10] <joduinn> dbaron: for now, file it against build&release, and mark it as P1 with some comment about closing the tree.

Comment 1

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

reopened tree, so no longer a p1. 

Keeping open to track recurring issues...

Comment 2

[Rob Campbell [:rc] (:robcee)]

it looks like the test run froze and buildbot tried to kill it unsuccessfully (see bug 381004).

Closing this as there's nothing to do here.