1299025 - Intermittent [taskcluster:error] Pulling docker image {"path":"public/image.tar","type":"task-image","taskId":"QO|Ws|U9|Uz|EN|Hf"} has failed: Error loading docker image. Could not download artifact ... after 5 attempt(s). Error: connect ETIMEDOUT

Status: RESOLVED DUPLICATE of bug 1302596

(Taskcluster :: General, defect)

Description

[Treeherder Bug Filer]

Filed by: philringnalda [at] gmail.com

https://treeherder.mozilla.org/logviewer.html#?job_id=2737471&repo=autoland

https://queue.taskcluster.net/v1/task/GebW_9_ATAGMBYj6R6ZmKA/runs/0/artifacts/public%2Flogs%2Flive_backing.log

https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/GebW_9_ATAGMBYj6R6ZmKA/runs/0/artifacts/public%2Flogs%2Flive_backing.log&only_show_unexpected=1

Comment 2

[Greg Arndt [:garndt]]

There's not much to do with this bug because we are already retrying 5 times to download the file.  Sometimes machines and/or s3 have hiccups that prevent file transfer and I think this is one of those cases.  If we don't see this ongoing over the next couple of days we could close it.

Comment 3

[Intermittent Failures Robot]

12 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* fx-team: 6
* mozilla-inbound: 4
* autoland: 2

Platform breakdown:
* linux64: 12

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1299025&startday=2016-08-29&endday=2016-09-04&tree=all

Comment 4

[Phil Ringnalda (:philor)]

The reason we keep plaguing you with bugs like this is because we want to wear you down the same way we wore down releng with buildbot, so that when you see something like this you'll instead say "sometimes machines and/or s3 have hiccups, so after we fail like this 5 times within a run, we should set RETRY on the run so we can try again on a different machine a few minutes later."

Comment 5

[Greg Arndt [:garndt]]

There is no need to try to wear us down, we definitely want to fix any issues if we can.  in this case if the worker marked that run as needing to be retried, there is nothing to say that the same worker won't pick up the task again.  Perhaps that's still enough to get around this, I don't know.

Comment 6

[Greg Arndt [:garndt]]

Also, we could add to the worker the ability to detect that maybe something is going on and in crease the duration at which it will claim tasks in hopes other workers will claim more.