Closed Bug 1146188 Opened 9 years ago Closed 9 years ago

TaskCluster builds mostly dying while getting 500s from quay.io

Categories

(Taskcluster :: General, defect)

Product:
Taskcluster
Component:
General
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: philor, Unassigned)

References

Details

Looks like it got better after a couple of hours.

Probably still a https://wiki.mozilla.org/Sheriffing/Job_Visibility_Policy violation, though.
Severity: blocker → normal
Depends on: 1146197
A few lines from: http://status.quay.io/
--------------------
Completed - The database upgrade has completed and service should be running normally. We thank you for your patience during this upgrade process. 
Mar 22, 18:03 EDT

In progress - We're beginning the second stage of the upgrade process. We expect an intermittent downtime of no more than 20-30 minutes during this period. 
Mar 22, 17:01 EDT
...

Scheduled - A database upgrade will occur from 4PM - 6PM on Sunday, March 22nd, 2015. We expect some intermittent periods of downtime during this upgrade. 
Mar 17, 13:13 EDT
--------------------

That might explain it, though my timezone math would have to be off by one hour :)

Anyways, this only makes me more interested in looking at using S3 for storing docker images.
We're not using quay anymore, right? There's still the matter of bug 1147867 where a lot of these failures were also getting dumped, but I think this can be closed now.
Blocks: 1080265
Flags: needinfo?(jlal)
Closing this bug as it is now being tracked mostly in 1165759 and 1147867.
Status: NEW → RESOLVED
Closed: 9 years ago
Flags: needinfo?(jlal)
Resolution: --- → FIXED
Also, the issue is more of being able to retry failed pulls with any repository, not specific to just quay.  Bug 1170999 hopes to help with that.
Component: TaskCluster → General
Product: Testing → Taskcluster
Target Milestone: --- → mozilla41
Resetting Version and Target Milestone that accidentally got changed...
Target Milestone: mozilla41 → ---
Version: Trunk → unspecified
You need to log in before you can comment on or make changes to this bug.