Closed Bug 799799 (talos-mtnlion-r5-022) Opened 13 years ago Closed 11 years ago

talos-mtnlion-r5-022 problem tracking

Categories

(Infrastructure & Operations Graveyard :: CIDuty, task)

Product:
Infrastructure & Operations Graveyard
Component:
CIDuty
Platform:
x86_64
macOS
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: rail, Unassigned)

References

Details

(Whiteboard: [buildduty][capacity])

No description provided.
Depends on: 799800
Rebooted via PDU.
Status: NEW → RESOLVED
Closed: 13 years ago
No longer depends on: 799800
Resolution: --- → FIXED
Back in production.
Product: mozilla.org → Release Engineering
Attempting SSH reboot...Failed. Attempting PDU reboot...Failed. Filed IT bug for reboot (bug 1105297)
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Status: REOPENED → RESOLVED
Closed: 13 years ago → 11 years ago
QA Contact: armenzg → bugspam.Callek
Resolution: --- → FIXED
Hitting lots of timeouts and slave health looks very poor overall (~50%).
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Re-imaged and returned to production.
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED
Timing out jobs left and right again :(
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Re-imaged and returned to production. I recognize the potential futility here, but we're hardware-constrained.
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED
Slave health is poor (again) and it's burning jobs consistently in ways that I've never seen before, a la. https://treeherder.mozilla.org/ui/logviewer.html#?job_id=470127&repo=mozilla-aurora Disabled. Please decomm.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
http://mozilla-releng-blobs.s3.amazonaws.com/blobs/mozilla-aurora/sha512/07e2ea9c39d537fe846d528800de414f1aada2ffea47e9d1f1150891cf092deb851293cf65e8b5eb22d3a3b3c6cea6ed2ebdc5561b6e55b68bfb0296d85f3694 is more instructive than the log, what with the Mac OS X can't repair the disk "Macintosh HD" You can still open or copy files on the disk, but you can't save changes to files on the disk. Back up the disk and reformat it as soon as you can. dialog displayed. Rather than leaping to decomm, let's just replace the disk which we know is bad whether or not diagnostics admit it.
Depends on: 1115251
Reenabled, good luck little slave.
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED
Exact same thing again. Disabled, we need to either blind-replace the disks or decomm, and we cannot decomm a 10.8 slave.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Reenabled with new disks.
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED
This slave was one of two (talos-mtnlion-r5-015 being the other) to hit bug 1072044. Hopefully the new disks takes care of it.
Product: Release Engineering → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.