879287 - (bld-lion-r5-071) bld-lion-r5-071 problem tracking

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task, P3)

Description

[Phil Ringnalda (:philor)]

It has lost the ability to run make check, hitting bug 874642 like bld-lion-r5-042 and bld-lion-r5-085 before it. I wanted to say that we should just get it reimaged and put it back in service to see whether that's all it takes to fix this problem, but I see by bug 876832 that so far that hasn't gone well with 085. Maybe this one (or 042, which has fallen through the cracks and not gotten a reimage bug) will fare better.

Comment 1

[Armen [:armenzg]]

Disabled.

Comment 2

[Aki Sasaki (not active)]

I've done the post-imaging steps, but this is currently failing puppet's screen resolution step.

Either the r5 lion builders need a dongle for some reason, or the check shouldn't be running.  Either way this needs followup.

Comment 3

[Aki Sasaki (not active)]

The above resolution issue had to do with cltbld not being logged in.
Essentially:

* I noticed cltbld had an old pw
* changed cltbld's pw via ssh
* neglected to change cltbld's autologon settings
* ???
* profit!

Fixed the pw+autologin via vnc, noticed puppet finished successfully but didnt' launch buildbot.  Followed https://wiki.mozilla.org/ReleaseEngineering/How_To/Set_Up_a_Freshly_Imaged_Slave#Bld-Lion-r5_--_buildbot_startup_failures_on_Lion .

Now connected and taking jobs.

Comment 4

[Ryan VanderMeulen [:RyanVM]]

Burned 5 of its last 12 jobs, the most recent failure being this:
https://tbpl.mozilla.org/php/getParsedLog.php?id=47100295&tree=Mozilla-Central

Disabled.

Comment 5

[Phil Ringnalda (:philor)]

Reenabled and rebooted.

Comment 6

[Release Engineering SlaveAPI Service]

Attempting SSH reboot...Failed.
Attempting PDU reboot...Failed.
Filed IT bug for reboot (bug 1124811)

Comment 7

[Phil Ringnalda (:philor)]

Hasn't taken any job for two days, hasn't taken a real job (an actual build, not a repack, not a fuzzing job) for five days.

Comment 8

[Release Engineering SlaveAPI Service]

Attempting SSH reboot...Failed.
Attempting PDU reboot...Failed.
Filed IT bug for reboot (bug 1244657)

Comment 9

[Vlad Ciobancai [:vladC] [:vciobancai]]

slave has been resolved by DCops. we will monitor the slave to see if is working OK

Comment 10

[Vlad Ciobancai [:vladC] [:vciobancai]]

By mistake I rebooted the slave and the following jobs failed http://buildbot-master82.bb.releng.scl3.mozilla.com:8001/builders/OS%20X%2010.7%20mozilla-inbound%20build/builds/4359 . Sorry for mistake

Comment 11

[Release Engineering SlaveAPI Service]

Attempting SSH reboot...Failed.
Attempting PDU reboot...Failed.
Filed IT bug for reboot (bug 1311029)

Comment 12

[Andrei Obreja [:aobreja NOT AVAILABLE][:buildduty]]

Back online and taking jobs.