Closed Bug 735735 (talos-r3-fed-068) Opened 12 years ago Closed 10 years ago

talos-r3-fed-068 problem tracking

Categories

(Infrastructure & Operations Graveyard :: CIDuty, task, P3)

Product:
Infrastructure & Operations Graveyard
Component:
CIDuty
Platform:
x86
Linux
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: mozilla, Unassigned)

References

(
URL
)

Details

(Whiteboard: [buildduty][capacity][buildslaves]) 

Needs a reboot.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Depends on: 740493
OS: Mac OS X → Linux
Priority: -- → P3
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Depends on: 740798
No longer depends on: 740798
Getting close to a reimage ?
Depends on: 742433
grey screen | rebooted
Status: REOPENED → RESOLVED
Closed: 12 years ago → 12 years ago
Resolution: --- → FIXED
Status: RESOLVED → REOPENED
Depends on: 750039
Resolution: FIXED → ---
Status: REOPENED → RESOLVED
Closed: 12 years ago → 12 years ago
Resolution: --- → FIXED
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
grey screen | rebooted
Status: REOPENED → RESOLVED
Closed: 12 years ago → 12 years ago
Resolution: --- → FIXED
PING down again:
[52] talos-r3-fed-068.build.scl1:PING is CRITICAL: PING CRITICAL - Packet loss = 100%
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Depends on: 751687
grey screen | rebooted
Back in production.
Status: REOPENED → RESOLVED
Closed: 12 years ago → 12 years ago
Resolution: --- → FIXED
Please reboot again.
Status: RESOLVED → REOPENED
Depends on: 757272
Resolution: FIXED → ---
The reboot got this back in production.
Status: REOPENED → RESOLVED
Closed: 12 years ago → 12 years ago
Resolution: --- → FIXED
Please reboot.
Status: RESOLVED → REOPENED
Depends on: 760958
Resolution: FIXED → ---
grey screen | rebooted
Status: REOPENED → RESOLVED
Closed: 12 years ago → 12 years ago
Resolution: --- → FIXED
Please reboot.
Depends on: 762545
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
grey screen | rebooted
Up and running
Status: REOPENED → RESOLVED
Closed: 12 years ago → 12 years ago
Resolution: --- → FIXED
Status: RESOLVED → REOPENED
Depends on: 765229
Resolution: FIXED → ---
Status: REOPENED → RESOLVED
Closed: 12 years ago → 12 years ago
Resolution: --- → FIXED
Please reboot.
Status: RESOLVED → REOPENED
Depends on: 768520
Resolution: FIXED → ---
Back in production.
Status: REOPENED → RESOLVED
Closed: 12 years ago → 12 years ago
Resolution: --- → FIXED
Status: RESOLVED → REOPENED
Depends on: 780207
Resolution: FIXED → ---
Depends on: 781818
Depends on: 785854
Back in production.
Status: REOPENED → RESOLVED
Closed: 12 years ago → 12 years ago
Resolution: --- → FIXED
Reboot needed.
Status: RESOLVED → REOPENED
Depends on: 803076
Resolution: FIXED → ---
Depends on: 804830
Status: REOPENED → RESOLVED
Closed: 12 years ago → 12 years ago
Resolution: --- → FIXED
Status: RESOLVED → REOPENED
Depends on: 808500
Resolution: FIXED → ---
Did 100s of jobs since bug 808500 but is down again, reboot time.
Depends on: 810713
Taking jobs.
Status: REOPENED → RESOLVED
Closed: 12 years ago → 12 years ago
Resolution: --- → FIXED
Needs a reboot, not available on the network.
Status: RESOLVED → REOPENED
Depends on: 814216
Resolution: FIXED → ---
Status: REOPENED → RESOLVED
Closed: 12 years ago → 12 years ago
Resolution: --- → FIXED
Seeing this a lot this morning:
https://tbpl.mozilla.org/php/getParsedLog.php?id=17973936&tree=Mozilla-Esr17

exceptions.OSError: [Errno 30] Read-only file system: '/home/cltbld/talos-slave/test-pgo/build'
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Disabled in slavealloc, and I went in and forced a reboot as well.

Not a single pass in the last 400 runs
(In reply to Justin Wood (:Callek) from comment #22)
> and I went in and forced a reboot as well.

Umm, odd- Seems that *not a single* login attempt, with all versions of the password, including trying as root is working
Depends on: 822073
Blocks: 826863
Requested to be re-imaged in bug 826863.
Do not forget to enable on slavealloc when we get to set it up.
Remember that it might have issues and it might burn jobs even after re-imaging.
Diagnosis might be required at that point.
Back in production.
Status: REOPENED → RESOLVED
Closed: 12 years ago → 11 years ago
Resolution: --- → FIXED
Depends on: 830154
Reboot needed (bug 830154)
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED
needs a reboot
Status: RESOLVED → REOPENED
Depends on: 833807
Resolution: FIXED → ---
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED
No longer blocks: 826863
Depends on: 826863
and reboot needed ,

Host Status:	
  DOWN  
 (for 1d 5h 21m 27s)
Status Information:	PING CRITICAL - Packet loss = 100%
Status: RESOLVED → REOPENED
Depends on: 836660
Resolution: FIXED → ---
And down again.
Depends on: 837526
Confirmed its back up again.
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED
...and back down
Status: RESOLVED → REOPENED
Depends on: 839427
Resolution: FIXED → ---
Depends on: 842538
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED
Status: RESOLVED → REOPENED
Depends on: 847772
Resolution: FIXED → ---
Green (and a couple orange) in production.
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED
Status: RESOLVED → REOPENED
Depends on: 894430
Resolution: FIXED → ---
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED
Depends on: 896449
No longer depends on: 896449
Product: mozilla.org → Release Engineering
Status: RESOLVED → REOPENED
Depends on: 912545
Resolution: FIXED → ---
Back in production.
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED
Last job Oct 3
Status: RESOLVED → REOPENED
Depends on: 929220
Resolution: FIXED → ---
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED
Attempting SSH reboot...Failed.
Filed IT bug for reboot (bug 962752)
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Status: REOPENED → RESOLVED
Closed: 11 years ago → 10 years ago
Resolution: --- → FIXED
Attempting SSH reboot...Failed.
Filed IT bug for reboot (bug 973952)
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Back in production.
Status: REOPENED → RESOLVED
Closed: 10 years ago → 10 years ago
Resolution: --- → FIXED
Attempting SSH reboot...Failed.
Filed IT bug for reboot (bug 977789)
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Status: REOPENED → RESOLVED
Closed: 10 years ago → 10 years ago
Resolution: --- → FIXED
Attempting SSH reboot...Failed.
Filed IT bug for reboot (bug 988921)
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Attempting SSH reboot...Failed.
Filed IT bug for reboot (bug 994768)
Disabled in slavealloc due to lack of jobs. Should be back ready for service anyway
Status: REOPENED → RESOLVED
Closed: 10 years ago → 10 years ago
Resolution: --- → FIXED
Product: Release Engineering → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.