890312 - (t-w732-ix-002) t-w732-ix-002 problem tracking

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task, P3)

Description

[[e-mail] (:bhearsum)]

went down, trying pdu reboot

Comment 1

[[e-mail] (:bhearsum)]

back up

Comment 2

[Phil Ringnalda (:philor)]

According to https://secure.pub.build.mozilla.org/buildapi/recent/t-w732-ix-002?numbuilds=3000, this slave has actually never once been able to run xperf (sample log https://tbpl.mozilla.org/php/getParsedLog.php?id=27973749&tree=Fx-Team).

Disabled in slavealloc.

Comment 3

[Justin Wood (:Callek)]

Joel, is there anything about xperf that can narrow down to a thing or set-of-things that could cause this problem?

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

Most likely this doesn't have the xperf toolchain and the .mof installed, i.e. bug 879879.

It would be great if most of the failed xperf runs are related to bad slaves like this one.

Comment 5

[Justin Wood (:Callek)]

(In reply to Joel Maher (:jmaher) from comment #4)
> Most likely this doesn't have the xperf toolchain and the .mof installed,
> i.e. bug 879879.
> 
> It would be great if most of the failed xperf runs are related to bad slaves
> like this one.

Q, can you confirm Bug 879879 is properly assigned to w732 in GPO? And if this particular host hasn't begun diags by then, check if there was an issue with that specifically here?

This isn't the only host having xperf issues, but is the one with this answer :-)

Comment 6

[Q]

It is the right place but I will double check that all policies are bring applied.

I will ask in this bug because my previous conversations were inconclusive. Is there a programmatic way test to see if the step 

"mofcomp c:\Users\cltbld\mozprofilerprobe.mof"


Worked correctly ?

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

I know of no method to determine if that was successful or not :(

Comment 8

[[e-mail] (:bhearsum)]

Going to see if the reimage fixed this.

Comment 9

[Phil Ringnalda (:philor)]

Still can't run xperf, but this time instead of disabling it and losing the use of it until someone figures out how to fix it, I filed bug 923877 so people can just star it and ignore it.

Comment 10

[Joel Maher ( :jmaher ) (UTC -8)]

If this can't be fixed, we need to get rid of the machine.  It is unacceptable to expect a specific machine to fail on a given test.  The error in bug 923877 is different than I have seen before, I would like to check out this specific machine and run xperf on it locally.

Comment 11

[Joel Maher ( :jmaher ) (UTC -8)]

please remove this from production:
https://tbpl.mozilla.org/php/getParsedLog.php?id=28962259&tree=Try&full=1#error2
18:34:37     INFO -  start_xperf.py: error: ERROR: xperf_path 'c:/Program Files/Microsoft Windows Performance Toolkit/xperf.exe' does not exist

if we have done the reimaging and xperf isn't installed this machine should not be put back in production.

Comment 12

[Q]

I agree Joel.

However, it does appear the binary is there:

cltbld@T-W732-IX-002 /c/program files
$ ls -al "c:\Program Files/Microsoft Windows Performance Toolkit\xperf.exe"
-rwxr-xr-x 1 cltbld Administrators 335128 Mar 18  2010 c:\Program Files/Microsof
t Windows Performance Toolkit\xperf.exe

The read and execute permission look right up the entire tree.


Is there any other reason this error would appear?

Comment 13

[Q]

and xperf seems to run manually:

c:\Program Files\Microsoft Windows Performance Toolkit>xperf -on latency -stackw
alk profile

c:\Program Files\Microsoft Windows Performance Toolkit>xperf -d out.etl
Merged Etl: out.etl
The trace you have just captured "out.etl" may contain personally identifiable i
nformation, including but not necessarily limited to paths to files accessed, pa
ths to registry accessed and process names. Exact information depends on the eve
nts that were logged. Please be aware of this when sharing out this trace with o
ther people.


c:\Program Files\Microsoft Windows Performance Toolkit>cat out.etl

  ¯L..                             \ S y s t e m R oo
 t \ s y s t e m 3 2 \ D R I V E R S \ e 1 q 6 2 3 2 . s y s     .
A@   h§Lu3T%IO.'bs%`_.        ??? ø.     .oLu s b p o r t . s y s   X
      ¬Lu3T%IO.'bs%`_.       ?#. inJg 2 0 0 e w m . s y s   M a t r ox
   G 2 0 0 e W   M i n i p o r t   D r i v e r   6 . 1 2 . 0 1 . 1 0 1 0   6 . 1
 2 . 1 . 1 0 1 0   0   M a t r o x   G 2 0 0 e W   M i n i p o r t   D r i v e r
   M a t r o x   G r a p h i c s   I n c .   6 . 1 2 . 0 1 . 1 0 1 0   0 0 0 0 d
 a 3 9 a 3 e e 5 e 6 b 4 b 0 d 3 2 5 5 b f e f 9 5 6 0 1 8 9 0 a f d 8 0 7 0 9 0
 0 0 0 d a 3 9 a 3 e e 5 e 6 b 4 b 0 d 3 2 5 5 b f e f 9 5 6 0 1 8 9 0 a f d 8 0
 7 0 9     X
A  . h¯Lu3T%IO.'bs%`_.         . cIzOe 1 q 6 2 3 2 . s y s   I n t e l(
      ¬Lu3T%IO.'bs%`_.        PO??#.     inJg 2 0 0 e w m . s y s   `  1 1 . 1 7
A# . h . 0   b u i l t   b y :   W i n D D K   1 1 . 1 7 . 2 7 . 0   0   I n t e
      ¬Lu3T%IO.'bs%`_.        PO?        inJ.   c:\build\g200ser\g200e i o n   1
w\sources1.01\drivers\miniport\objfre\i386\G200ewm.pdb        . .Az ..hf e f 9 5
  l0 1 8 9 0 a .                             \ S y s t e m R oo4 b 0 d 3 2 5 5 b
 t \ s y s t e m 3 2 \ D R I V E R S \ g 2 0 0 e w m . s y s     :.
A@   h
      ¬Lu3T%IO.'bs%`_.        ÿ  .[Js e r e n u m . s y s   S e r i al
   P o r t   E n u m e r a t o r   6 . 1 . 7 6 0 0 . 1 6 3 8 5   ( w i n 7 _ r t
 m . 0 9 0 7 1 3 - 1 2 5 5 )   6 . 1 . 7 6 0 0 . 1 6 3 8 5   1 0 3 3   M i c r o
 s o f t r   W i n d o w s r   O p e r a t i n g   S y s t e m   M i c r o s o f
 t   C o r p o r a t i o n   6 . 1 . 7 6 0 0 . 1 6 3 8 5   0 0 0 0 d a 3 9 a 3 e
 e 5 e 6 b 4 b 0 d 3 2 5 5 b f e f 9 5 6 0 1 8 9 0 a f d 8 0 7 0 9   0 0 0 0 f 5
 1 9 f e e c 4 8 6 d e 8 7 e d 7 3 c b 9 2 d 3 c a c 8 0 2 4 0 0 0 0 0 0 0 0   X

A  . h
      ¬Lu3T%IO.'bs%`_.        ??? ÿ      .[Js e r e n u m . s y s   X
A$ . h
      ¬Lu3T%IO.'bs%`_.        ???    ).¦E.Y%B.¤
                                               IQ¤I..   SerEnum.pdb ..
Az ..h
  l
  ¬L      >U                              \ S y s t e m R oo
 t \ s y s t e m 3 2 \ D R I V E R S \ s e r e n u m . s y s     n.
A@   h
      «Lu3T%IO.'bs%`_.        D  YTLc o m p o s i t e b u s . s y s  M
 u l t i - T r a n s p o r t   C o m p o s i t e   B u s   E n u m e r a t o r 6
 . 1 . 7 6 0 1 . 1 7 5 1 4   ( w i n 7 s p 1 _ r t m . 1 0 1 1 1 9 - 1 8 5 0 ) 6
 . 1 . 7 6 0 1 . 1 7 5 1 4   1 0 3 3   M i c r o s o f t r   W i n d o w s r   O
 p e r a t i n g   S y s t e m   M i c r o s o f t   C o r p o r a t i o n   6 .
 1 . 7 6 0 1 . 1 7 5 1 4   0 0 0 0 d a 3 9 a 3 e e 5 e 6 b 4 b 0 d 3 2 5 5 b f e
 f 9 5 6 0 1 8 9 0 a f d 8 0 7 0 9   0 0 0 0 f 5 1 9 f e e c 4 8 6 d e 8 7 e d 7
 3 c b 9 2 d 3 c a c 8 0 2 4 0 0 0 0 0 0 0 0     b
A  . h
      «Lu3T%IO.'bs%`_.         ?? D      YTLc o m p o s i t e b u s .s
 y s         ]
A$ . h
      «Lu3T%IO.'bs%`_.         ??    Kqs)
                                         b    . .A" ..h
  l
  «L      p?.                             \ S y s t e m R oo
 t \ s y s t e m 3 2 \ D R I V E R S \ C o m p o s i t e B u s . s y s   o.
A@   h
      _Lu3T%IO.'bs%`_.        p  j ¦Gm v 2 . s y s   U l t r a V n c  m
 i n i p o r t   d r i v e r 2   6 . 0 . 6 0 0 0 . 1 6 3 8 6   b u i l t   b y :
   W i n D D K   6 . 0 . 6 0 0 0 . 1 6 3 8 6   0   U l t r a V n c   m i n i p o
 r t   d r i v e r 2   U V N C   B V B A   6 . 0 . 6 0 0 0 . 1 6 3 8 6   0 0 0 0
 d a 3 9 a 3 e e 5 e 6 b 4 b 0 d 3 2 5 5 b f e f 9 5 6 0 1 8 9 0 a f d 8 0 7 0 9
   0 0 0 0 d a 3 9 a 3 e e 5 e 6 b 4 b 0 d 3 2 5 5 b f e f 9 5 6 0 1 8 9 0 a f d
 8 0 7 0 9   P
A  . h
      _Lu3T%IO.'bs%`_.        d?? p      j ¦Gm v 2 . s y s   -
A$ . h
      _Lu3T%IO.'bs%`_.        d??    fU
¦_I~zc0A?¥ª.   c:\winddk\6000
\src\video\displays\mirror\mini\objfre_wlh_x86\i386\mv2.pdb   . .A- ..h
  l
  _L?? p      .w                              \ S y s t e m R oo
 t \ s y s t e m 3 2 \ D R I V E R S \ m v 2 . s y s     O.
A@   h
      ¨Lu3T%IO.'bs%`_.          O-OHv r t a u c b l . s y s   K e r ne
 l - m o d e   W D M   d r i v e r   4 . 9 . 0 . 2 1 5 5   4 . 9 . 0 . 2 1 5 5 1
 0 3 3   V i r t u a l   A u d i o   C a b l e   E u g e n e   V .   M u z y c h
 e n k o   4 . 9 . 0 . 1 6 5 1   0 0 0 0 d a 3 9 a 3 e e 5 e 6 b 4 b 0 d 3 2 5 5
 b f e f 9 5 6 0 1 8 9 0 a f d 8 0 7 0 9   0 0 0 0 d a 3 9 a 3 e e 5 e 6 b 4 b 0
 d 3 2 5 5 b f e f 9 5 6 0 1 8 9 0 a f d 8 0 7 0 9         Z
A  . h
      ¨Lu3T%IO.'bs%`_.        ``?       O-OHv r t a u c b l . s y s  <

A$ . h
      ¨Lu3T%IO.'bs%`_.        ``?    ,4IB.FM~Tñ,ZA.   c:\music\win\v
ac\4\4xx\driver\objfre_wxp_x86\i386\vrtaucbl.pdb      . .Aÿ ..h
  l
  ¨L`? D      O..                             \ S y s t e m R oo
 t \ s y s t e m 3 2 \ D R I V E R S \ v r t a u c b l . s y s   t.
A@   h
      ¨Lu3T%IO.'bs%`_.        d. dE[Jp o r t c l s . s y s   P o r t  C
 l a s s   ( C l a s s   D r i v e r   f o r   P o r t / M i n i p o r t   D e v
 i c e s )   6 . 1 . 7 6 0 0 . 1 6 3 8 5   ( w i n 7 _ r t m . 0 9 0 7 1 3 - 1 2
 5 5 )   6 . 1 . 7 6 0 0 . 1 6 3 8 5   1 0 3 3   M i c r o s o f t r   W i n d o
 w s r   O p e r a t i n g   S y s t e m   M i c r o s o f t   C o r p o r a t i
 o n   6 . 1 . 7 6 0 0 . 1 6 3 8 5   0 0 0 0 d a 3 9 a 3 e e 5 e 6 b 4 b 0 d 3 2
 5 5 b f e f 9 5 6 0 1 8 9 0 a f d 8 0 7 0 9   0 0 0 0 f 5 1 9 f e e c 4 8 6 d e
 8 7 e d 7 3 c b 9 2 d 3 c a c 8 0 2 4 0 0 0 0 0 0 0 0       X
A  . h
      ¨Lu3T%IO.'bs%`_.        0'? d.     dE[Jp o r t c l s . s y s   X
A$ . h
      ¨Lu3T%IO.'bs%`_.        0'?    !"qwHø_ Q'jæ.   portcls.pdb ..
Az ..h
  l
  ¨L.     <ý.                             \ S y s t e m R oo
 t \ s y s t e m 3 2 \ D R I V E R S \ p o r t c l s . s y s     F.
A@   h
      ALu3T%IO.'bs%`_.        ?. oO[Jd r m k . s y s   M i c r o s o ft
   T r u s t e d   A u d i o   D r i v e r s   6 . 1 . 7 6 0 0 . 1 6 3 8 5   ( w
 i n 7 _ r t m . 0 9 0 7 1 3 - 1 2 5 5 )   6 . 1 . 7 6 0 0 . 1 6 3 8 5   1 0 3 3
   M i c r o s o f t r   W i n d o w s r   O p e r a t i n g   S y s t e m   M i
 c r o s o f t   C o r p o r a t i o n   6 . 1 . 7 6 0 0 . 1 6 3 8 5   0 0 0 0 d
 a 3 9 a 3 e e 5 e 6 b 4 b 0 d 3 2 5 5 b f e f 9 5 6 0 1 8 9 0 a f d 8 0 7 0 9 0
 0 0 0 f 5 1 9 f e e c 4 8 6 d e 8 7 e d 7 3 c b 9 2 d 3 c a c 8 0 2 4 0 0 0 0 0
 0 0 0     R
A  . h
      ALu3T%IO.'bs%`_.         ? ?.     oO[Jd r m k . s y s         U
A$ . h
      ALu3T%IO.'bs%`_.         ?    T"O)<J"pø..Y¨.   drmk.pdb    . .
A~ ..h
  l
  ALO.                             \ S y s t e m R o o
 t \ s y s t e m 3 2 \ D R I V E R S \ d r m k . s y s   (.
A@   h
      ALu3T%IO.'bs%`_.        @. UTLk s . s y s   K e r n e l   C S A
 L i b r a r y   6 . 1 . 7 6 0 1 . 1 7 5 1 4   ( w i n 7 s p 1 _ r t m . 1 0 1 1
 1 9 - 1 8 5 0 )   6 . 1 . 7 6 0 1 . 1 7 5 1 4   0   M i c r o s o f t r   W i n
 d o w s r   O p e r a t i n g   S y s t e m   M i c r o s o f t   C o r p o r a
 t i o n   6 . 1 . 7 6 0 1 . 1 7 5 1 4   0 0 0 0 d a 3 9 a 3 e e 5 e 6 b 4 b 0 d
 3 2 5 5 b f e f 9 5 6 0 1 8 9 0 a f d 8 0 7 0 9   0 0 0 0 f 5 1 9 f e e c 4 8 6
 d e 8 7 e d 7 3 c b 9 2 d 3 c a c 8 0 2 4 0 0 0 0 0 0 0 0   N
A  . h
      ALu3T%IO.'bs%`_.        ø-? @.     UTLk s . s y s     S
A$ . h
      ALu3T%IO.'bs%`_.        ø-?    øYp._b3D,=á;9..   ks.pdb      ..
A" ..h
  l
  AL-? @.     X4.                             \ S y s t e m R oo
 t \ s y s t e m 3 2 \ D R I V E R S \ k s . s y s       V.
A@   h
      ALu3T%IO.'bs%`_.         . T[Ja g i l e v p n . s y s   R A S  A
 g i l e   V p n   M i n i p o r t   C a l l   M a n a g e r   6 . 1 . 7 6 0 0 .
 1 6 3 8 5   ( w i n 7 _ r t m . 0 9 0 7 1 3 - 1 2 5 5 )   6 . 1 . 7 6 0 0 . 1 6
 3 8 5   1 0 3 3   M i c r o s o f t r   W i n d o w s r   O p e r a t i n g   S
 y s t e m   M i c r o s o f t   C o r p o r a t i o n   6 . 1 . 7 6 0 0 . 1 6 3
 8 5   0 0 0 0 d a 3 9 a 3 e e 5 e 6 b 4 b 0 d 3 2 5 5 b f e f 9 5 6 0 1 8 9 0 a
 f d 8 0 7 0 9   0 0 0 0 f 5 1 9 f e e c 4 8 6 d e 8 7 e d 7 3 c b 9 2 d 3 c a c
 8 0 2 4 0 0 0 0 0 0 0 0     Z
A  . h
      ALu3T%IO.'bs%`_.        dT?  .     T[Ja g i l e v p n . s y s  Y

A$ . h
      ALu3T%IO.'bs%`_.        dT?    3®p#G~ú@B.O._.   AgileVpn.pdb .
 .Aÿ ..h
  l
  ALT?  .     SE.                             \ S y s t e m R oo
 t \ s y s t e m 3 2 \ D R I V E R S \ A g i l e V p n . s y s   Z.
A@   h
      ALu3T%IO.'bs%`_.        ?. 9[Jr a s l 2 t p . s y s   R A S   L2
 T P   m i n i - p o r t / c a l l - m a n a g e r   d r i v e r   6 . 1 . 7 6 0
 0 . 1 6 3 8 5   ( w i n 7 _ r t m . 0 9 0 7 1 3 - 1 2 5 5 )   6 . 1 . 7 6 0 0 .
 1 6 3 8 5   1 0 3 3   M i c r o s o f t r   W i n d o w s r   O p e r a t i n g
   S y s t e m   M i c r o s o f t   C o r p o r a t i o n   6 . 1 . 7 6 0 0 . 1
 6 3 8 5   0 0 0 0 d a 3 9 a 3 e e 5 e 6 b 4 b 0 d 3 2 5 5 b f e f 9 5 6 0 1 8 9
 0 a f d 8 0 7 0 9   0 0 0 0 f 5 1 9 f e e c 4 8 6 d e 8 7 e d 7 3 c b 9 2 d 3 c
 a c 8 0 2 4 0 0 0 0 0 0 0 0         X
A  . h
      ALu3T%IO.'bs%`_.        >>? ?.     9[Jr a s l 2 t p . s y s   X
A$ . h
      ALu3T%IO.'bs%`_.        >>?    `go1JIoC.u.ñ.   rasl2tp.pdb ..
Az ..h
  l
  ALI.                             \ S y s t e m R oo
 t \ s y s t e m 3 2 \ D R I V E R S \ r a s l 2 t p . s y s     T.
A@   h
      ALu3T%IO.'bs%`_.        ø  0[JN D I S T A P I . S Y S   N D I S
 3 . 0   c o n n e c t i o n   w r a p p e r   d r i v e r   6 . 1 . 7 6 0 0 . 1
 6 3 8 5   ( w i n 7 _ r t m . 0 9 0 7 1 3 - 1 2 5 5 )   6 . 1 . 7 6 0 0 . 1 6 3
 8 5   1 0 3 3   M i c r o s o f t r   W i n d o w s r   O p e r a t i n g   S y
 s t e m   M i c r o s o f t   C o r p o r a t i o n   6 . 1 . 7 6 0 0 . 1 6 3 8
 5   0 0 0 0 d a 3 9 a 3 e e 5 e 6 b 4 b 0 d 3 2 5 5 b f e f 9 5 6 0 1 8 9 0 a f
 d 8 0 7 0 9   0 0 0 0 f 5 1 9 f e e c 4 8 6 d e 8 7 e d 7 3 c b 9 2 d 3 c a c 8
 0 2 4 0 0 0 0 0 0 0 0       Z
A  . h
      ALu3T%IO.'bs%`_.        ?o? ø      0[JN D I S T A P I . S Y S  Y

A$ . h
      ALu3T%IO.'bs%`_.        ?o?    ¥z?IKcO
                                             .Aÿ ..h
  l
  AL? ø      ?0.                             \ S y s t e m R oo
 t \ s y s t e m 3 2 \ D R I V E R S \ n d i s t a p i . s y s   f.
A@   h
      Lu3T%IO.'bs%`_.         . ?LN D I S W A N . S Y S   M S   P PP
   F r a m i n g   D r i v e r   ( S t r o n g   E n c r y p t i o n )   6 . 1 .
 7 6 0 1 . 1 7 5 1 4   ( w i n 7 s p 1 _ r t m . 1 0 1 1 1 9 - 1 8 5 0 )   6 . 1
 . 7 6 0 1 . 1 7 5 1 4   1 0 3 3   M i c r o s o f t r   W i n d o w s r   O p e
 r a t i n g   S y s t e m   M i c r o s o f t   C o r p o r a t i o n   6 . 1 .
 7 6 0 1 . 1 7 5 1 4   0 0 0 0 d a 3 9 a 3 e e 5 e 6 b 4 b 0 d 3 2 5 5 b f e f 9
 5 6 0 1 8 9 0 a f d 8 0 7 0 9   0 0 0 0 f 5 1 9 f e e c 4 8 6 d e 8 7 e d 7 3 c
 b 9 2 d 3 c a c 8 0 2 4 0 0 0 0 0 0 0 0     X
A  . h
      Lu3T%IO.'bs%`_.        @??  .     ?LN D I S W A N . S Y S   X
A$ . h
      Lu3T%IO.'bs%`_.        @??    (LMI?0oB.`Y.V'.   ndiswan.pdb ..
Az ..h
  l
  L??  .     >.                             \ S y s t e m R oo
 t \ s y s t e m 3 2 \ D R I V E R S \ n d i s w a n . s y s     ^.
A@   h
      Lu3T%IO.'bs%`_.        ?. M[Jr a s p p p o e . s y s   R A S  P
 P P o E   m i n i - p o r t / c a l l - m a n a g e r   d r i v e r   6 . 1 . 7
 6 0 0 . 1 6 3 8 5   ( w i n 7 _ r t m . 0 9 0 7 1 3 - 1 2 5 5 )   6 . 1 . 7 6 0
 0 . 1 6 3 8 5   1 0 3 3   M i c r o s o f t r   W i n d o w s r   O p e r a t i
 n g   S y s t e m   M i c r o s o f t   C o r p o r a t i o n   6 . 1 . 7 6 0 0
 . 1 6 3 8 5   0 0 0 0 d a 3 9 a 3 e e 5 e 6 b 4 b 0 d 3 2 5 5 b f e f 9 5 6 0 1
 8 9 0 a f d 8 0 7 0 9   0 0 0 0 f 5 1 9 f e e c 4 8 6 d e 8 7 e d 7 3 c b 9 2 d
 3 c a c 8 0 2 4 0 0 0 0 0 0 0 0     Z
A  . h
      Lu3T%IO.'bs%`_.         ?? ?.     M[Jr a s p p p o e . s y s  Y

A$ . h
      Lu3T%IO.'bs%`_.         ??    .kÿP$ݪIE`£¤<Px.   raspppoe.pdb .
 .Aÿ ..h
  l
  L
</snip>

Comment 14

[Joel Maher ( :jmaher ) (UTC -8)]

I retriggered and it worked, I have been messing around with the toolchain which invokes xperf, but the retrigger would have the same problems.

Maybe I could get this as a loaner and run the toolchain on it to see what is going on.

Comment 15

[Justin Wood (:Callek)]

(In reply to Joel Maher (:jmaher) from comment #10)
> If this can't be fixed, we need to get rid of the machine.  It is
> unacceptable to expect a specific machine to fail on a given test.  The
> error in bug 923877 is different than I have seen before, I would like to
> check out this specific machine and run xperf on it locally.

I agree.

(In reply to Joel Maher (:jmaher) from comment #14)
> I retriggered and it worked, I have been messing around with the toolchain
> which invokes xperf, but the retrigger would have the same problems.
> 
> Maybe I could get this as a loaner and run the toolchain on it to see what
> is going on.

I'll be happy to hand it over as a loaner during my buildduty coverage on monday, or coop's over the rest of the week. Can you get an explicit "Loan..." bug on file please.

Comment 16

[Joel Maher ( :jmaher ) (UTC -8)]

I have run 10 runs of xperf with success, using this slave is not helping me at all.  

Poking around in the system logs on this machine, I see that we are trying to run windows update all the time and it fails.  I would bet every time we reboot.  I see this error (pardon my typing it out):
windows update could not be installed because of error 2149842967 "" (Command line: "wusa.exe c:\mfp\Windows6.1-KB968211-x86-RefreshPkg.msu /quiet /norestart")

Can somebody point me to the schedule and test process for applying windows updates to the slaves?

I also found other errors (wmi errors probably during login, and a once a day sxs guidgen.exe error around 12:30am)

Looking around more, I see that we schedule a disk defragmentation at 1am every wednesday which takes a few minutes, can I get confirmation that this machine is removed from scheduling jobs while this is taking place?

Also we have analyze system at 6am every 14 days scheduled?  Again, I would like confirmation that this is run while this slave is removed from production.

Comment 17

[Joel Maher ( :jmaher ) (UTC -8)]

after a reboot and further testing, this is not able to reproduce.  I suggest we reimage this machine and put it back in production.

I would really like to know the answers to my questions in comment 16.

Comment 18

[Ed Morley [:emorley]]

(In reply to Joel Maher (:jmaher) from comment #17)
> I would really like to know the answers to my questions in comment 16.

Broken these out to bug 926965

Comment 19

[[e-mail] (:bhearsum)]

Back in production. Good luck, t-w732-ix-002!

Comment 20

[Jordan Lund (:jlund)]

This machine is disabled. There is nothing wrong with it. We will be swapping its chassis slot with t-xp32-ix-085 to help with 085's investigation (determine if we have a bad slot).

when bug 975006 is done moving the machine, we can enable it again in production and remove 975006 from dep list.

Comment 21

[Jordan Lund (:jlund)]

this machine is back in production in its new slot. Will post findings in bug 975006

Comment 22

[Jordan Lund (:jlund)]

jobs were running well again.

I disabled the slave and the slots were switched in https://bugzilla.mozilla.org/show_bug.cgi?id=975006

it is being re imaged once again and can then be brought back into production by EOD

Comment 23

[Phil Ringnalda (:philor)]

End of some day, anyway ;) Reenabled and rebooted into production.

Comment 24

[Phil Ringnalda (:philor)]

Picked up some kind of a busted on the evening of July 1st (oddly, at the same time as t-xp32-ix-075) and hasn't done anything since.

Comment 25

[Phil Ringnalda (:philor)]

The busted for both of them being that they no longer liked that master - I rebooted it during the bug 1049227 rolling gracefuls to make it hop masters, and now it's happy again.

Comment 26

[Chris Cooper [:coop] (he/him)]

Re-imaged and returned to production.

Comment 27

[Ryan VanderMeulen [:RyanVM]]

Hitting bug 1090639.

Comment 28

[Kim Moir [:kmoir] ET]

couldn't set resolution correctly so reimaged

Comment 29

[Kim Moir [:kmoir] ET]

Fixed resolution and put back into production

Comment 30

[Phil Ringnalda (:philor)]

It fails when it has to run the quantum-pageload talos suite, "ERROR - Unknown zip extension for filename 'python3.6.zip'", https://treeherder.mozilla.org/logviewer.html#?job_id=103588411&repo=autoland

Disabled.

Comment 31

[Alin Selagea [:aselagea]]

Re-imaged, green jobs atm.