1391114 - Make `mach artifact toolchain` validate toolchain downloads

Status: RESOLVED FIXED

(Firefox Build System :: General, enhancement)

Description

[Mike Hommey [:glandium]]

That's something I planned to do, but two build recent failures related to that made me just do it now:

https://treeherder.mozilla.org/logviewer.html#?job_id=123714327&repo=autoland
https://treeherder.mozilla.org/logviewer.html#?job_id=123705733&repo=autoland

Comment 1

[Mike Hommey [:glandium]]

Attachment: Bug 1391114 - Make `mach artifact toolchain` validate toolchain downloads.

Tooltool manifests contain digests that have been used to validate
tooltool downloads. Toolchain artifacts don't benefit from that, and as
a result, an incomplete download can be considered as finished, and
unpack fail after that, without retrying, even with --retry.

Fortunately, the chain of trust artifacts do contains digests for
taskcluster artifacts, as long as the jobs that created the artifacts
have chain of trust enabled.

As of now, the goal is not cryptographic validation of the download, but
to ensure that we got the complete file, and to trigger a new download
if we haven't.

Review commit: https://reviewboard.mozilla.org/r/169362/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/169362/

Comment 2

[Gregory Szorc [:gps]]

Comment on attachment 8898065 [details]
Bug 1391114 - Make `mach artifact toolchain` validate toolchain downloads.

https://reviewboard.mozilla.org/r/169362/#review174718

So hacky. Just reimplement tooltool already :)

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fd74c81eee5a
Make `mach artifact toolchain` validate toolchain downloads. r=gps

Comment 4

[Mike Hommey [:glandium]]

(In reply to Gregory Szorc [:gps] from comment #2)
> Comment on attachment 8898065 [details]
> Bug 1391114 - Make `mach artifact toolchain` validate toolchain downloads.
> 
> https://reviewboard.mozilla.org/r/169362/#review174718
> 
> So hacky. Just reimplement tooltool already :)

Actually, I just wish taskcluster would deal with that on its own.

Comment 5

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/fd74c81eee5a

Comment 6

[Intermittent Failures Robot]

2 failures in 949 pushes (0.002 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 2

Platform breakdown:
* linux64-noopt: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1391114&startday=2017-08-14&endday=2017-08-20&tree=all