440531 - qm-win2k3-pgo01 sporadic red with "LINK : fatal error LNK1104: cannot open file"

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect)

Description

[Daniel Holbert [:dholbert]]

Three out of the last four cycles on qm-win2k3-pgo01 (a Firefox 3 tinderbox) have been red with link errors.

[most recent] Red: LINK : fatal error LNK1104: cannot open file 'npbasic.lib'
[next build ] Red: LINK : fatal error LNK1104: cannot open file 'npwinless.lib'
[next build ] Orange: Sporadic TUnit failure
[next build ] Red: LINK : fatal error LNK1104: cannot open file 'npbasic.lib'
[next build ] Green


Logs for the link errors:
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1213898520.1213902562.31927.gz
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1213867281.1213870299.15472.gz
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1213826788.1213829279.2424.gz

There have been zero checkins since the last green, with the exception of one mailnews checkin by mnyromyr that went in just before the most recent red.  (which I don't think is related)  

So I'm guessing it's due to some bad state on the server.  Maybe this box needs a clobber or a reboot or something?

Comment 1

[Daniel Holbert [:dholbert]]

(In reply to comment #0)
> [most recent] Red: LINK : fatal error LNK1104: cannot open file 'npbasic.lib'
> [next build ] Red: LINK : fatal error LNK1104: cannot open file 'npwinless.lib'
> [next build ] Orange: Sporadic TUnit failure
> [next build ] Red: LINK : fatal error LNK1104: cannot open file 'npbasic.lib'
> [next build ] Green

Sorry -- if it wasn't clear, by "[next build]", I meant "next as you scroll down the tinderbox page".  (i.e. they're really [previous build]s time-wise)

Comment 2

[Mark Smith [:xb95]]

Justin says to escalate this to build, since I can't figure out how to log into this machine as ctlbld or buildbot.

If someone wants to walk me through this on IRC, let me know.

Comment 3

[Daniel Holbert [:dholbert]]

Just went red again (after two long green cycles).

This time it's:
 LINK : fatal error LNK1104: cannot open file 'npscriptable.lib'

Log:
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1213925576.1213928370.6953.gz

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

Tried a clobber.

Comment 5

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #2)
> Justin says to escalate this to build, since I can't figure out how to log into
> this machine as ctlbld or buildbot.
I was able to login as buildbot on first try.


> If someone wants to walk me through this on IRC, let me know.
I just walked through this on irc with Mark. Turns out this machine was not listed with the right account/pswd info on the IT page, so he never tried using "buildbot" user. Doc corrected.

Comment 6

[Robert Strong (they/them - no direct email)]

It seems to alternate between

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1213996881.1214008322.9751.gz&fulltext=1#err0
../../../../_tests/xpcshell-simple/test_dm/unit/test_bug_401430.js: FAIL

and

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1213983250.1213995190.5468.gz&fulltext=1
a bunch of test failures followed by
Failure: buildbot.slave.commands.TimeoutError: SIGKILL failed to kill process

Comment 7

[Rob Campbell [:rc] (:robcee)]

this seems to be a problem with the Windows link tools. Not much we can do about it, unfortunately, and we see it on all of our windows build machines on occasion.

Comment 8

[Phil Ringnalda (:philor)]

Hmm. I haven't been box-watching as much the last few months, but the recent evidence doesn't seem to bear that out: going back to the dawn of time for this tree (the morning of July 4), this box has hit this exact problem six times, so once every single day, while all of the other boxes have hit it zero times - a couple of other non-PGO unit test boxes have had the other long-known things, the TimeoutError and the readdir error, but not a single one, including the nightly box that builds PGO vastly more times per day than this one, has had the linker fail to open a file.

Comment 9

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

While the specific error message is different, I wonder if this is related to bug#442819.