Closed Bug 1258714 Opened 8 years ago Closed 8 years ago

Windows builds not starting

Categories

(Infrastructure & Operations Graveyard :: CIDuty, task)

Product:
Infrastructure & Operations Graveyard
Component:
CIDuty
Type:
task
Priority:
Not set
Severity:
blocker

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: aryx, Unassigned)

References

Details

Windows builds scheduled in the last 2 hours didn't start (only looked at inbound and fx-team).
It looks like we generated a new AMI for both try and build last night, so I'm not sure if something there went wrong. Could one of you (mark, q, rob) please take a look ASAP?
Flags: needinfo?(rthijssen)
Flags: needinfo?(q)
Flags: needinfo?(mcornmesser)
Looks like the puppet run succeeded but launched instances weren't running runner/buildbot. deregistered amis and mass terminated spot instances. Will take an hour or so for new instances to replace them.
Flags: needinfo?(rthijssen)
Just fyi for markco: the puppet run on y-2008 also succeeded and those instances are running runner/buildbot correctly. Not sure why b is different.
For b-2008, it doesn't seem like Puppet ran on it this morning: 

https://foreman.pub.build.mozilla.org/hosts/b-2008-ec2-golden.build.releng.use1.mozilla.com/reports
Flags: needinfo?(mcornmesser)
Yep, sorry my bad. That was caused by a cert download failure which wasn't detected in userdata.
Flags: needinfo?(q)
Situation seems to be back to normal. Thank you for fixing it.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
See Also: → 1258202
Aryx, FYI these kinds of bugs are best filed in Release Engineering :: Buildduty.
Component: Release Automation → Buildduty
QA Contact: bhearsum → bugspam.Callek
Product: Release Engineering → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.