598038 - linux-ix-slave16 has been off for 5 days

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[[e-mail] (:bhearsum)]

I'm not sure what's up with this machine, nagios shows it as not having run Buildbot for 5+ days. Based on the contents of /builds/slave it doesn't appear to have _ever_ run Buildbot:
[cltbld@linux-ix-slave16 slave]$ ls -l /builds/slave/
total 102508
-rw-r--r-- 1 cltbld cltbld       966 Sep 10 14:03 buildbot.tac.off
-rw-rw-r-- 1 cltbld cltbld 104857600 Sep 20 07:48 myfile.dat

John, do you know anything about this machine?

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

I looked at this machine this morning. Seemed to me that it didn't make the transition to production completely (production ssh keys, staging dirs unclobbered, buildbot.tac.off pointing to pm03). 

When I clobbered all the build directories it was pretty slow, and there were errors removing some .hg dirs. So I booted to runlevel 1 and ran fsck over it - there were no errors found. Same for the other partitions. I'm not sure if the errors were as a result of a hard reboot during the network fun, or if this machine has an issue. Any objections to throwing it back on staging for a few runs ?

Comment 2

[[e-mail] (:bhearsum)]

None from me.

Comment 3

[Lukas Blakk [:lsblakk] use ?needinfo]

I'm running this in staging right now.  Interestingly it was run in staging before, Sept 8-10th and did successful builds.  I'll move this back to production tomorrow if all looks well with it.

Comment 4

[Lukas Blakk [:lsblakk] use ?needinfo]

It's in production now, looks to be working fine on several branch builds.