1307509 - bundle sidecar images on the docker-worker AMI

Status: RESOLVED FIXED

(Taskcluster :: Workers, defect)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

Many of the failures to link sidecar containers come from failures to *download* the sidecar containers.

If they are available on-disk, rather than being downloaded, then we can eliminate that class of failure.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

this bug is the work required for bug 1307493, there are no updates here in 3.5 months and bug 1307493 is the root cause for many of our top intermittents.

:jhford, were you planning on working on this and bug 1307510 in the short term?

Comment 2

[Greg Arndt [:garndt]]

We are planning on bundling the side car images which is something I attempted late last week and picking back up this week.  Original attempts to bundle them with the AMI proved to be unsuccessful.  I had to revert the changes here: https://github.com/taskcluster/docker-worker/pull/277

Comment 3

[Greg Arndt [:garndt]]

Attachment: docker-worker PR 279

Comment 4

[[github robot]]

Commit pushed to master at https://github.com/taskcluster/docker-worker

https://github.com/taskcluster/docker-worker/commit/f02cd887da1097a483ccce10e2e9b74183f79a4f
Merge pull request #279 from gregarndt/load_sidecar

Bug 1307509 - Load sidecar images during startup

Comment 5

[Greg Arndt [:garndt]]

Comment on attachment 8828433 [details] [review]
docker-worker PR 279

carrying over approval from github

Comment 6

[Greg Arndt [:garndt]]

New amis have been generated and rolling out to gecko-t-linux-* worker types currently:

--> hvm-builder: AMIs were created:

us-east-1: ami-7cce396a
us-west-1: ami-da9eccba
us-west-2: ami-3da41a5d
--> pv-builder: AMIs were created:

us-east-1: ami-90cc3b86
us-west-1: ami-409ecc20
us-west-2: ami-34a41a54

Comment 7

[Greg Arndt [:garndt]]

AMIs for builders and gecko-decision (will roll out tomorrow):
--> pv-builder: AMIs were created:

us-east-1: ami-4424d352
us-west-1: ami-be9fcdde
us-west-2: ami-e2a11f82
--> hvm-builder: AMIs were created:

us-east-1: ami-f325d2e5
us-west-1: ami-989ccef8
us-west-2: ami-119c2271

Comment 8

[Greg Arndt [:garndt]]

AMIs have been deployed.  Looking over the last set of failures from the last couple of days, the errors do not seem to be related to sidecars