905365 - (t-w864-ix-083) t-w864-ix-083 problem tracking

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task, P3)

Description

[Ryan VanderMeulen [:RyanVM]]

Every job is dying like the below:
https://tbpl.mozilla.org/php/getParsedLog.php?id=26551689&tree=B2g-Inbound

========= Started 'c:\mozilla-build\hg\hg clone ...' failed (results: 5, elapsed: 0 secs) (at 2013-08-14 12:32:42.023864) =========
'c:\\mozilla-build\\hg\\hg' 'clone' 'http://hg.mozilla.org/build/mozharness' 'scripts'
 in dir C:\slave\test-pgo\. (timeout 1320 secs)
 watching logfiles {}
 argv: ['c:\\mozilla-build\\hg\\hg', 'clone', 'http://hg.mozilla.org/build/mozharness', 'scripts']
 environment:
  ALLUSERSPROFILE=C:\ProgramData
  APPDATA=C:\Users\cltbld.T-W864-IX-083\AppData\Roaming
  COMMONPROGRAMFILES=C:\Program Files (x86)\Common Files
  COMMONPROGRAMFILES(X86)=C:\Program Files (x86)\Common Files
  COMMONPROGRAMW6432=C:\Program Files\Common Files
  COMPUTERNAME=T-W864-IX-083
  COMSPEC=C:\windows\system32\cmd.exe
  FP_NO_HOST_CHECK=NO
  HOMEDRIVE=C:
  HOMEPATH=\Users\cltbld.T-W864-IX-083
  LOCALAPPDATA=C:\Users\cltbld.T-W864-IX-083\AppData\Local
  LOGONSERVER=\\T-W864-IX-083
  NUMBER_OF_PROCESSORS=8
  OS=Windows_NT
  PATH=C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\mozilla-build\python27;C:\mozilla-build\python27\Scripts;C:\mozilla-build\msys\bin;C:\mozilla-build\vim\vim72;C:\mozilla-build\wget;C:\mozilla-build\info-zip;C:\CoreUtils\bin;C:\mozilla-build\buildbotve\scripts;C:\mozilla-build\hg
  PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC
  PROCESSOR_ARCHITECTURE=x86
  PROCESSOR_ARCHITEW6432=AMD64
  PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 30 Stepping 5, GenuineIntel
  PROCESSOR_LEVEL=6
  PROCESSOR_REVISION=1e05
  PROGRAMDATA=C:\ProgramData
  PROGRAMFILES=C:\Program Files (x86)
  PROGRAMFILES(X86)=C:\Program Files (x86)
  PROGRAMW6432=C:\Program Files
  PROMPT=$P$G
  PSMODULEPATH=C:\windows\system32\WindowsPowerShell\v1.0\Modules\
  PUBLIC=C:\Users\Public
  PWD=C:\slave\test-pgo
  SYSTEMDRIVE=C:
  SYSTEMROOT=C:\windows
  TEMP=C:\Users\CLTBLD~1.T-W\AppData\Local\Temp
  TMP=C:\Users\CLTBLD~1.T-W\AppData\Local\Temp
  USERDOMAIN=T-W864-IX-083
  USERDOMAIN_ROAMINGPROFILE=T-W864-IX-083
  USERNAME=cltbld
  USERPROFILE=C:\Users\cltbld.T-W864-IX-083
  WINDIR=C:\windows
 using PTY: False
program finished with exit code -1073741800
elapsedTime=0.109000
========= Finished 'c:\mozilla-build\hg\hg clone ...' failed (results: 5, elapsed: 0 secs) (at 2013-08-14 12:32:42.209821) =========

Comment 1

[Ryan VanderMeulen [:RyanVM]]

Disabled in slavealloc

Comment 2

[Chris Cooper [:coop] (he/him)]

When I connected via VNC, I saw the time listed briefly as 2:58 when it was actually 4:21 (pm). Of course, the very act of connecting triggered the time to update, so it quickly corrected itself.

I've rebooted+re-enabled the slave and am watching the current job:

http://buildbot-master69.srv.releng.use1.mozilla.com:8201/builders/WINNT%206.2%20mozilla-inbound%20debug%20test%20jsreftest/builds/772

Comment 3

[Chris Cooper [:coop] (he/him)]

4 passed jobs in a row.

Comment 4

[Phil Ringnalda (:philor)]

Made it ten days, hasn't done a job since August 24th.

Comment 5

[[e-mail] (:bhearsum)]

Been working away for awhile.

Comment 6

[Armen [:armenzg]]

It got into a weird state after bug 977615.

Comment 7

[Phil Ringnalda (:philor)]

Apparently got out of that weird state, since it's been reenabled and working away for awhile.

Comment 8

[Release Engineering SlaveAPI Service]

Attempting SSH reboot...Failed.
Attempting IPMI reboot...Failed.
Filed IT bug for reboot (bug 1172993)

Comment 9

[Phil Ringnalda (:philor)]

Disabled while it doesn't have its bad drive attached to it.

Comment 10

[Phil Ringnalda (:philor)]

Reenabled after its trip back to iX.