Open Bug 1635905 Opened 4 years ago Updated 4 days ago

Intermittent Exception: Download failed, no more retries!

Categories

(Release Engineering :: General, defect, P5)

Tracking

(Not tracked)

REOPENED

People

(Reporter: intermittent-bug-filer, Unassigned)

References

(Regression)

Details

(Keywords: intermittent-failure, regression)

Filed by: btara [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=301078510&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/NieaMDwQQHOy4ACnJz0gag/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/NieaMDwQQHOy4ACnJz0gag/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1


[setup 2020-05-06T18:43:45.007Z] MOZ_FETCHES_DIR is /builds/worker/fetches
[fetches 2020-05-06T18:43:45.007Z] fetching artifacts
[fetches 2020-05-06T18:43:45.008Z] executing ['/usr/bin/python3', '-u', '/builds/worker/bin/fetch-content', 'task-artifacts']
[fetches 2020-05-06T18:43:45.140Z] attempt 1/5
[fetches 2020-05-06T18:43:45.140Z] Downloading http://taskcluster/api/queue/v1/task/ZadATFIhS_WavQqhvuOHog/artifacts/project/gecko/android-sdk/android-sdk-linux.tar.xz to /builds/worker/fetches/android-sdk-linux.tar.xz
[fetches 2020-05-06T18:43:45.140Z] attempt 1/5
[fetches 2020-05-06T18:43:45.140Z] Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/ZUuako8_QCCgm_hmawQ6fg/artifacts/public/build/minidump_stackwalk.tar.xz to /builds/worker/fetches/minidump_stackwalk.tar.xz
[fetches 2020-05-06T18:43:45.141Z] Downloading http://taskcluster/api/queue/v1/task/ZadATFIhS_WavQqhvuOHog/artifacts/project/gecko/android-sdk/android-sdk-linux.tar.xz
[fetches 2020-05-06T18:43:45.142Z] Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/ZUuako8_QCCgm_hmawQ6fg/artifacts/public/build/minidump_stackwalk.tar.xz
[fetches 2020-05-06T18:43:45.142Z] attempt 1/5
[fetches 2020-05-06T18:43:45.142Z] Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/ITwxSM_KR62t_vWLyQo3sA/artifacts/public/build/fix-stacks.tar.xz to /builds/worker/fetches/fix-stacks.tar.xz
[fetches 2020-05-06T18:43:45.142Z] Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/ITwxSM_KR62t_vWLyQo3sA/artifacts/public/build/fix-stacks.tar.xz
[fetches 2020-05-06T18:43:45.254Z] https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/ITwxSM_KR62t_vWLyQo3sA/artifacts/public/build/fix-stacks.tar.xz resolved to 1214328 bytes with sha256 598e04e60dedd85a2b6ea2a23b419142087d1ce00187617c440b67bdd5f75c0f in 0.112s
[fetches 2020-05-06T18:43:45.254Z] Extracting /builds/worker/fetches/fix-stacks.tar.xz to /builds/worker/fetches
[fetches 2020-05-06T18:43:45.261Z] https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/ZUuako8_QCCgm_hmawQ6fg/artifacts/public/build/minidump_stackwalk.tar.xz resolved to 983480 bytes with sha256 2460f18b076e19f365035466b83cef7c793ec05539713f26a5b00beb1cbac2b7 in 0.119s
[fetches 2020-05-06T18:43:45.261Z] Extracting /builds/worker/fetches/minidump_stackwalk.tar.xz to /builds/worker/fetches
[fetches 2020-05-06T18:43:45.390Z] /builds/worker/fetches/minidump_stackwalk.tar.xz extracted in 0.129s
[fetches 2020-05-06T18:43:45.390Z] /builds/worker/fetches/fix-stacks.tar.xz extracted in 0.136s
[fetches 2020-05-06T18:43:45.390Z] Removing /builds/worker/fetches/fix-stacks.tar.xz
[fetches 2020-05-06T18:43:45.390Z] Removing /builds/worker/fetches/minidump_stackwalk.tar.xz
[fetches 2020-05-06T18:44:00.952Z] Download failed: Remote end closed connection without response
[fetches 2020-05-06T18:44:00.955Z] sleeping for 60.00s (attempt 1/5)
[fetches 2020-05-06T18:45:01.302Z] attempt 2/5
[fetches 2020-05-06T18:45:01.314Z] Downloading http://taskcluster/api/queue/v1/task/ZadATFIhS_WavQqhvuOHog/artifacts/project/gecko/android-sdk/android-sdk-linux.tar.xz to /builds/worker/fetches/android-sdk-linux.tar.xz
[fetches 2020-05-06T18:45:01.314Z] Downloading http://taskcluster/api/queue/v1/task/ZadATFIhS_WavQqhvuOHog/artifacts/project/gecko/android-sdk/android-sdk-linux.tar.xz
[fetches 2020-05-06T18:45:19.774Z] Download failed: <urlopen error [Errno 113] No route to host>
[fetches 2020-05-06T18:45:19.776Z] sleeping for 91.00s (attempt 2/5)
[fetches 2020-05-06T18:46:51.085Z] attempt 3/5
[fetches 2020-05-06T18:46:51.121Z] Downloading http://taskcluster/api/queue/v1/task/ZadATFIhS_WavQqhvuOHog/artifacts/project/gecko/android-sdk/android-sdk-linux.tar.xz to /builds/worker/fetches/android-sdk-linux.tar.xz
[fetches 2020-05-06T18:46:51.122Z] Downloading http://taskcluster/api/queue/v1/task/ZadATFIhS_WavQqhvuOHog/artifacts/project/gecko/android-sdk/android-sdk-linux.tar.xz
[fetches 2020-05-06T18:46:54.309Z] Download failed: <urlopen error [Errno 113] No route to host>
[fetches 2020-05-06T18:46:54.312Z] sleeping for 134.00s (attempt 3/5)
[fetches 2020-05-06T18:49:08.564Z] attempt 4/5
[fetches 2020-05-06T18:49:08.628Z] Downloading http://taskcluster/api/queue/v1/task/ZadATFIhS_WavQqhvuOHog/artifacts/project/gecko/android-sdk/android-sdk-linux.tar.xz to /builds/worker/fetches/android-sdk-linux.tar.xz
[fetches 2020-05-06T18:49:08.628Z] Downloading http://taskcluster/api/queue/v1/task/ZadATFIhS_WavQqhvuOHog/artifacts/project/gecko/android-sdk/android-sdk-linux.tar.xz
[fetches 2020-05-06T18:49:12.138Z] Download failed: <urlopen error [Errno 113] No route to host>
[fetches 2020-05-06T18:49:12.147Z] sleeping for 202.50s (attempt 4/5)
[fetches 2020-05-06T18:52:34.846Z] attempt 5/5
[fetches 2020-05-06T18:52:35.021Z] Downloading http://taskcluster/api/queue/v1/task/ZadATFIhS_WavQqhvuOHog/artifacts/project/gecko/android-sdk/android-sdk-linux.tar.xz to /builds/worker/fetches/android-sdk-linux.tar.xz
[fetches 2020-05-06T18:52:35.083Z] Downloading http://taskcluster/api/queue/v1/task/ZadATFIhS_WavQqhvuOHog/artifacts/project/gecko/android-sdk/android-sdk-linux.tar.xz
[fetches 2020-05-06T18:52:38.385Z] Download failed: <urlopen error [Errno 113] No route to host>
[fetches 2020-05-06T18:52:38.595Z] Traceback (most recent call last):
[fetches 2020-05-06T18:52:38.597Z]   File "/builds/worker/bin/fetch-content", line 681, in <module>
[fetches 2020-05-06T18:52:38.620Z]     sys.exit(main())
[fetches 2020-05-06T18:52:38.620Z]   File "/builds/worker/bin/fetch-content", line 677, in main
[fetches 2020-05-06T18:52:38.620Z]     return args.func(args)
[fetches 2020-05-06T18:52:38.622Z]   File "/builds/worker/bin/fetch-content", line 611, in command_task_artifacts
[fetches 2020-05-06T18:52:38.623Z]     fetch_urls(downloads)
[fetches 2020-05-06T18:52:38.623Z]   File "/builds/worker/bin/fetch-content", line 482, in fetch_urls
[fetches 2020-05-06T18:52:38.623Z]     f.result()
[fetches 2020-05-06T18:52:38.623Z]   File "/usr/lib/python3.5/concurrent/futures/_base.py", line 405, in result
[fetches 2020-05-06T18:52:38.639Z]     return self.__get_result()
[fetches 2020-05-06T18:52:38.639Z]   File "/usr/lib/python3.5/concurrent/futures/_base.py", line 357, in __get_result
[fetches 2020-05-06T18:52:38.639Z]     raise self._exception
[fetches 2020-05-06T18:52:38.639Z]   File "/usr/lib/python3.5/concurrent/futures/thread.py", line 55, in run
[fetches 2020-05-06T18:52:38.639Z]     result = self.fn(*self.args, **self.kwargs)
[fetches 2020-05-06T18:52:38.639Z]   File "/builds/worker/bin/fetch-content", line 461, in fetch_and_extract
[fetches 2020-05-06T18:52:38.649Z]     download_to_path(url, dest_path, sha256=sha256, size=size)
[fetches 2020-05-06T18:52:38.649Z]   File "/builds/worker/bin/fetch-content", line 241, in download_to_path
[fetches 2020-05-06T18:52:38.649Z]     raise Exception("Download failed, no more retries!")
[fetches 2020-05-06T18:52:38.649Z] Exception: Download failed, no more retries!
[fetches 2020-05-06T18:52:39.008Z] removing /builds/worker/fetches
[fetches 2020-05-06T18:52:39.016Z] finished
[taskcluster 2020-05-06 18:52:44.193Z] === Task Finished ===
[taskcluster 2020-05-06 18:52:47.144Z] Artifact "public/logs/" not found at "/builds/worker/workspace/logs/"
[taskcluster 2020-05-06 18:52:47.997Z] Artifact "public/test_info/" not found at "/builds/worker/workspace/build/blobber_upload_dir/"
[taskcluster 2020-05-06 18:52:48.838Z] Unsuccessful task run with exit code: 1 completed in 551.301 seconds

Bug 1624649 (pushed today) introduced this particular download (android-sdk-linux). It usually works.

Regressed by: 1624649

If related to packet.net (as in bug 1521378, possibly), keep in mind that wcosta is making excellent progress migrating from packet.net to aws.

Summary: Intermittent Android Exception: Download failed, no more retries! → Intermittent Exception: Download failed, no more retries!
Severity: normal → S3
Status: NEW → RESOLVED
Closed: 10 months ago
Resolution: --- → INCOMPLETE
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Status: REOPENED → RESOLVED
Closed: 10 months ago4 months ago
Resolution: --- → INCOMPLETE
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Status: REOPENED → RESOLVED
Closed: 4 months ago2 months ago
Resolution: --- → INCOMPLETE
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
You need to log in before you can comment on or make changes to this bug.