839407 - please run diagnostics on foopy24

Status: RESOLVED FIXED

(Infrastructure & Operations :: DCOps, task)

Description

[Justin Wood (:Callek)]

We've seen multiple signs of failures over the past day on this foopy.

Please run diagnostics, but do not reimage (even if you can't correct any diagnostic failures)

This is a production foopy, but we do have capacity to let this wait until first half of next week, if you don't have anyone who can do this on Friday.

Comment 1

[Amy Rich [:arr] [:arich]]

I did a filesystem check that showed no issues.  Will leave for dcops to boot from cdrom and do other hardware diagnostics.

Comment 2

[Vinh Hua [:vinh]]

Only issues TechTool Diagnostics found were some file corruptions e.g. jpg, png, bmp etc.

Comment 3

[Justin Wood (:Callek)]

Per https://bugzilla.mozilla.org/show_bug.cgi?id=839375#c3 this is still severely hosed.

I also see:

foopy24:~ cltbld$ ps auxwww | grep Tech
root             258   5.0  0.8   707260  63676   ??  SNs  Fri02PM  14:57.67 /Library/PreferencePanes/TechTool Protection.prefPane/Contents/PlugIns/TechToolProDaemon.app/Contents/MacOS/TechToolProDaemon
root             280   0.0  0.0   634764   1608   ??  Ss   Fri02PM  19:34.76 /Applications/TechTool Pro 6.app/Contents/MacOS/BackgroundTool
cltbld           262   0.0  0.3 19315960  21356   ??  Ss   Fri02PM   0:00.65 /Library/PreferencePanes/TechTool Protection.prefPane/Contents/PlugIns/TechToolProAgent.app/Contents/MacOS/TechToolProAgent
cltbld           216   0.0  1.1   849516  95548   ??  S    Fri02PM  28:29.86 /Applications/TechTool Pro 6.app/Contents/MacOS/TechTool Pro 6 -psn_0_98328
foopy24:~ cltbld$ ps auxwww | grep Safari
cltbld           142   0.0  0.4  2547680  33676   ??  S    Fri02PM   0:17.82 /Applications/Safari.app/Contents/MacOS/Safari -psn_0_28679
cltbld         36646   0.0  0.0  2434892    448 s001  S+    4:07PM   0:00.00 grep Safari

Which looks bad for this system. What is TechTool *doing*, also why is Safari running (for some reason my VNC client is not letting me get in past the basic login screen, not even showing me errors/etc once I hit submit on password)

Comment 4

[Vinh Hua [:vinh]]

I believe Techtool is running in the background to anticipate any future hardware fails. Safari was opened so that I could download Techtool from my GoogleDrive, however it just hung.  I will swing by MV office tomorrow to take a look at this host.

Comment 5

[Vinh Hua [:vinh]]

Ran the techtool and apple disk utility diagnostics again and it came up clean. I've also uninstalled techtool.  At this point, perhaps a reimage might help?

Comment 6

[Salvador Espinoza [:sal]]

Host is racked and back online

Comment 7

[Justin Wood (:Callek)]

(In reply to Vinh Hua [:vinh] from comment #5)
> Ran the techtool and apple disk utility diagnostics again and it came up
> clean. I've also uninstalled techtool.

Pretty sure this didn't work (from ps): 

root              79   0.3  0.0   590440    196   ??  UNs   4:16PM   0:00.05 /Library/PreferencePanes/TechTool Pr
otection.prefPane/Contents/PlugIns/TechToolProDaemon.app/Contents/MacOS/TechToolProDaemon

> At this point, perhaps a reimage might help?

It might help, but it'd also sink about 2 days of actual work, more in prep - for releng alone. The mac foopies are vestiges of an old time where we don't have documentation for how to set them up, we also are trying to get the permits to tear these buildings down, trying to repair a cracked ceiling in one floor is imho too much work, when there is a new type of floor that can come in fully ready (linux foopy) in minutes.

If this is a "DOA" foopy, and needs a reimage, I don't want to put anyone on the hook for dealing with this more and instead want to get someone on the hook for giving us a single mtv1 linux foopy to replace this. -- In advance of scl1's linux-foopy setup/move.

Comment 8

[Amy Rich [:arr] [:arich]]

Callek: why not redistribute the tegras to existing linux foopies?  Have you determined how many can be serviced by one foopy?

Comment 9

[Vinh Hua [:vinh]]

:Callek - Are we decommissioning this mac mini?  I'm not able to ping it at the moment either.

Comment 10

[Amy Rich [:arr] [:arich]]

This machine is getting reimaged as part of being retasked.