1596011 - Perma Can't download from https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/project.mobile.reference-browser.v2.raptor.latest.arm/artifacts/public/target.apk to /builds/task_1573614169/workspace/build/target.apk!

Status: RESOLVED FIXED

(Testing :: Raptor, defect, P2)

Description

[Treeherder Bug Filer]

Filed by: csabou [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=275937636&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/ODf4MjOwQTOs2lJhpRh5rA/runs/0/artifacts/public/logs/live_backing.log

---

[task 2019-11-13T03:15:24.027Z] 03:15:24 WARNING - Server returned status 404 HTTP Error 404: Not Found for https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/project.mobile.reference-browser.v2.raptor.latest.arm/artifacts/public/target.apk
[task 2019-11-13T03:15:24.027Z] 03:15:24 INFO - retry: attempt #5 caught HTTPError exception: HTTP Error 404: Not Found
[task 2019-11-13T03:15:24.027Z] 03:15:24 FATAL - Can't download from https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/project.mobile.reference-browser.v2.raptor.latest.arm/artifacts/public/target.apk to /builds/task_1573614169/workspace/build/target.apk!
[task 2019-11-13T03:15:24.027Z] 03:15:24 FATAL - Caught exception: HTTP Error 404: Not Found
[task 2019-11-13T03:15:24.027Z] 03:15:24 FATAL - Caught exception: HTTP Error 404: Not Found
[task 2019-11-13T03:15:24.027Z] 03:15:24 FATAL - Caught exception: HTTP Error 404: Not Found
[task 2019-11-13T03:15:24.027Z] 03:15:24 FATAL - Caught exception: HTTP Error 404: Not Found
[task 2019-11-13T03:15:24.027Z] 03:15:24 FATAL - Caught exception: HTTP Error 404: Not Found
[task 2019-11-13T03:15:24.027Z] 03:15:24 FATAL - Running post_fatal callback...
[task 2019-11-13T03:15:24.027Z] 03:15:24 FATAL - Exiting -1
[task 2019-11-13T03:15:24.027Z] 03:15:24 INFO - Running post-action listener: _resource_record_post_action
[task 2019-11-13T03:15:24.027Z] 03:15:24 INFO - Running post-action listener: find_tests_for_coverage
[task 2019-11-13T03:15:24.027Z] 03:15:24 INFO - Running post-action listener: find_tests_for_verification
[task 2019-11-13T03:15:24.027Z] 03:15:24 INFO - Running post-action listener: set_extra_try_arguments
[task 2019-11-13T03:15:24.027Z] 03:15:24 INFO - Running post-action listener: setup_coverage_tools
[task 2019-11-13T03:15:24.027Z] 03:15:24 INFO - [mozharness: 2019-11-13 03:15:24.026947Z] Finished download-and-extract step (failed)
[task 2019-11-13T03:15:24.411Z] 03:15:24 INFO - Running post-run listener: _resource_record_post_run
[task 2019-11-13T03:15:24.411Z] cleanup
[task 2019-11-13T03:15:24.411Z] + cleanup
[task 2019-11-13T03:15:24.411Z] + local rv=255
[task 2019-11-13T03:15:24.411Z] + [[ -s /builds/worker/.xsession-errors ]]
[task 2019-11-13T03:15:24.411Z] + false
[task 2019-11-13T03:15:24.411Z] + exit 255
[task 2019-11-13T03:15:24.411Z]
[task 2019-11-13T03:15:24.411Z] netstat -aop
[task 2019-11-13T03:15:24.411Z] Active Internet connections (servers and established)

Comment 1

[Tom Prince [:tomprince]]

It looks like some time ago, the index changed to project.mobile.fenix.v2.performance-test.latest (with multiple platforms built by a single task), but that wasn't noticed because the old index still existed, though it was out of date. With the move to the new firefox-ci cluster, that out-of-date index is no longer present in the cluster, so this started failing, rather than providing out-of-date results.

Comment 2

[Intermittent Failures Robot]

33 failures in 4207 pushes (0.008 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 32
• autoland: 1

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 11
• android-hw-p2-8-0-android-aarch64: 10
• android-hw-g5-7-0-arm7-api-16: 11
• android-em-7-0-x86_64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1596011&startday=2019-11-11&endday=2019-11-17&tree=all

Comment 3

[Octavian Negru]

:igoldan, do you know where the new link must be put?

Comment 4

[Florin Strugariu [:Bebe]]

https://searchfox.org/mozilla-central/source/taskcluster/ci/test/raptor-fenix.yml#20

Comment 5

[Octavian Negru]

Thx, I will change the line in that file.

Comment 6

[Octavian Negru]

Attachment: Bug 1596011 - Perma Can't download apk

Comment 7

[Intermittent Failures Robot]

39 failures in 5444 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

This is the #28 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 39

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 13
• android-hw-p2-8-0-arm7-api-16: 13
• android-hw-g5-7-0-arm7-api-16: 13

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1596011&startday=2019-11-18&endday=2019-11-24&tree=all

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e0a7daa7a397
Perma Can't download apk r=perftest-reviewers,igoldan,jlorenzo

Comment 9

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/e0a7daa7a397

Comment 10

[Daniel Varga [:dvarga]]

This doesn't seem to be fixed: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&searchStr=pgo-raptor-tp6m-1-refbrow-cold-e10s%2Crap-refbrow%28tp6m-c-1%29&fromchange=92ff4d99f814bab4ddcb5ad7077e2c407b41c9d0

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=278355876&repo=mozilla-central&lineNumber=522

Comment 11

[Octavian Negru]

Attachment: Bug 1596011 - Perma can't download apk

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3d2b14d80fb3
Perma can't download apk r=perftest-reviewers,AlexandruIonescu,jlorenzo

Comment 13

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/3d2b14d80fb3

Comment 14

[Intermittent Failures Robot]

20 failures in 4331 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 20

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 3
• android-hw-p2-8-0-android-aarch64: 3
• android-hw-p2-8-0-arm7-api-16: 14

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1596011&startday=2019-11-25&endday=2019-12-01&tree=all