1493997 - Android coverage test tasks are failing to download android-sdk

Status: RESOLVED FIXED

(Testing :: Code Coverage, defect)

Description

[Marco Castelluccio [:marco]]

Attachment: live_backing.tar.xz

It seems like the tooltool download somehow also contains an extra argument (related to fetch):
/usr/bin/python2.7 -u /builds/worker/workspace/mozharness/external_tools/tooltool.py --url http://relengapi/tooltool/ fetch -m /builds/worker/workspace/build/tooltool.tt -o -c /builds/worker/tooltool-cache public/build/grcov.tar.xz@MRA1Xy3GTfmU70d9TysMpw

Comment 1

[Geoff Brown [:gbrown]]

Is this related to https://searchfox.org/mozilla-central/rev/881a3c5664ede5e08ee986d76433bc5c4b5680e6/testing/mozharness/mozharness/mozilla/tooltool.py#92 maybe?

Comment 2

[Andrew Halberstadt [:ahal]]

Came here to post that link :). That looks like the only plausible explanation to me.

Comment 3

[Tom Prince [:tomprince]]

Attachment: Bug 1493997: [taskgraph] Use fetches directly for grcov artifacts; r?ahal

Comment 4

[Tom Prince [:tomprince]]

Attachment: Bug 1493997: [mozharness] Ensure that `MOZ_TOOLCHAINS` isn't specified if `mach` isn't available; r?ahal

Comment 5

[Tom Prince [:tomprince]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=0b92a9bf474469d64d6a95ae3b624035878cd070

Comment 6

[Andrew Halberstadt [:ahal]]

Comment on attachment 9011853 [details]
Bug 1493997: [mozharness] Ensure that `MOZ_TOOLCHAINS` isn't specified if `mach` isn't available; r?ahal

Andrew Halberstadt [:ahal] has approved the revision.

Comment 7

[Andrew Halberstadt [:ahal]]

Comment on attachment 9011852 [details]
Bug 1493997: [taskgraph] Use fetches directly for grcov artifacts; r?ahal

Andrew Halberstadt [:ahal] has approved the revision.

Comment 8

[Marco Castelluccio [:marco]]

Comment on attachment 9011852 [details]
Bug 1493997: [taskgraph] Use fetches directly for grcov artifacts; r?ahal

Marco Castelluccio [:marco] has approved the revision.

Comment 9

[Tom Prince [:tomprince]]

Attachment: Bug 1493997 : [taskgraph] Add support for fetching toolchains via fetch-content; r?gps

Comment 10

[Gregory Szorc [:gps]]

Comment on attachment 9011971 [details]
Bug 1493997 : [taskgraph] Add support for fetching toolchains via fetch-content; r?gps

Gregory Szorc [:gps] has approved the revision.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/44644323d6b1
[taskgraph] Add support for fetching toolchains via fetch-content; r=gps
https://hg.mozilla.org/integration/autoland/rev/dc276e11f0f4
[taskgraph] Use fetches directly for grcov artifacts; r=ahal,marco

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8fe27b9b1bec
[mozharness] Ensure that `MOZ_TOOLCHAINS` isn't specified if `mach` isn't available; r=ahal

Comment 13

[Tom Prince [:tomprince]]

Attachment: Bug 1493997: [taskgraph] Add missinng `)` in comment.

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/44ec48f51186
[taskgraph] Add missinng `)` in comment.

Comment 15

[Andreea Pavel [:apavel]]

This is actively being worked on, no need to add stockwell tag.

Comment 16

[Intermittent Failures Robot]

33 failures in 553 pushes (0.06 failures/push) were associated with this bug yesterday.

Repository breakdown:
* mozilla-central: 33

Platform breakdown:
* android-hw-p2-8-0-android-aarch64: 1
* android-em-4-3-armv7-api16-ccov: 32

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1493997&startday=2018-09-25&endday=2018-09-25&tree=all

Comment 17

[Arthur Iakab [arthur_iakab]]

https://hg.mozilla.org/mozilla-central/rev/44644323d6b1
https://hg.mozilla.org/mozilla-central/rev/dc276e11f0f4
https://hg.mozilla.org/mozilla-central/rev/8fe27b9b1bec
https://hg.mozilla.org/mozilla-central/rev/44ec48f51186

Comment 18

[Intermittent Failures Robot]

36 failures in 2773 pushes (0.013 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 1
* mozilla-inbound: 2
* mozilla-central: 33

Platform breakdown:
* android-em-4-3-armv7-api16-ccov: 32
* android-hw-p2-8-0-android-aarch64: 1
* linux64: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1493997&startday=2018-09-24&endday=2018-09-30&tree=all

Comment 19

[Tom Prince [:tomprince]]

https://hg.mozilla.org/releases/mozilla-esr60/rev/70f54b5ee8c9
https://hg.mozilla.org/releases/mozilla-esr60/rev/6c6140f1f8b0