1068427 - (bld-lion-r5-059) bld-lion-r5-059 problem tracking

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task)

Comment 1

[Release Engineering SlaveAPI Service]

Attempting SSH reboot...Failed.
Attempting PDU reboot...Failed.
Filed IT bug for reboot (bug 1068428)

Comment 2

[Release Engineering SlaveAPI Service]

Attempting SSH reboot...Failed.
Attempting PDU reboot...Failed.
Filed IT bug for reboot (bug 1133069)

Comment 3

[Release Engineering SlaveAPI Service]

Attempting SSH reboot...Failed.
Attempting PDU reboot...Failed.
Filed IT bug for reboot (bug 1134183)

Comment 4

[Ryan VanderMeulen [:RyanVM]]

Burning B2G desktop builds. The log below is failure #2 today for this slave.
https://treeherder.mozilla.org/logviewer.html#?job_id=6728401&repo=mozilla-inbound

Comment 5

[Phil Ringnalda (:philor)]

Reenabled with a new disk.

Comment 6

[Jordan Lund (:jlund)]

discovered this failing during release automation. looks like it has a history of losing connection almost during each run. network? shutting down?

Comment 7

[Andrei Obreja [:aobreja NOT AVAILABLE][:buildduty]]

Back online and taking jobs.

Comment 8

[Phil Ringnalda (:philor)]

Failing make check with Permission denied: '/Volumes/Stub', which just means it needs to be reimaged. Disabled.

Comment 9

[Andrei Obreja [:aobreja NOT AVAILABLE][:buildduty]]

Back online after have been re-imaged.

Comment 10

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Creates busted builds: https://secure.pub.build.mozilla.org/builddata/reports/slave_health/slave.html?name=bld-lion-r5-059
Clicked the 'Shutdown buildbot' button (that's how it gets disabled, isn't it?)

Comment 11

[Phil Ringnalda (:philor)]

Disabled is actually "click the button next to the word 'Slavealloc', which then lets you click the checkbox next to 'Enabled?' to uncheck it, then click the other button to the right of the one next to 'Slavealloc' to commit that change, then click the reboot button to make it notice it should be disabled right away." Shutdown just tells it to not take another job after it finishes the one it is running, but leaves it able to take another job if, say, someone reboots every slave that hasn't taken a job for a few days.

Comment 12

[Andrei Obreja [:aobreja NOT AVAILABLE][:buildduty]]

Back online after re-image.

Comment 13

[Release Engineering SlaveAPI Service]

Attempting SSH reboot...Failed.
Attempting PDU reboot...Failed.
Filed IT bug for reboot (bug 1365063)

Comment 14

[Andrei Obreja [:aobreja NOT AVAILABLE][:buildduty]]

mini replaced with decommissioned t-yosemite-r5-0060.test.releng.scl3.mozilla.com. inventory updated and reimaged done by DCops team.
back online and taking jobs.