578752 - Verify downloaded build artifacts on test runs

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect, P4)

Description

[John Ford [:jhford] CET/CEST Berlin Time]

Once we have checksum files generated and uploaded to FTP, we should verify these checksums on the slave.

I propose that we do:
-download the build, test and symbols using current code paths
-run a script that:
  -takes artifact download urls as arguments for retries
  -verifies the checksums for each file downloaded
  -on failure will (up to X number of retries):
    -notify through email relevant parties of cache corruption
    -clear the cache
    -download file again
    -re-verify the file
-unpack files using current code paths

Comment 1

[Armen [:armenzg]]

We don't have the cache now as we use to at that time.
Shall we dupe bug 665559 against this one?
Are you planning on doing this on the build machine or the test machine?

Comment 2

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to comment #1)
> We don't have the cache now as we use to at that time.

That is correct.

> Shall we dupe bug 665559 against this one?

That is for repacks, this is for tests.  We could use the same logic for both, and if that's the case, lets modify this bug to cover both and dupe 665559 against this bug

> Are you planning on doing this on the build machine or the test machine?

It wouldn't be useful to do on the build machine as they are the source of these tarballs in the first place.  We should be verifying artifacts for repacks and tests.

Comment 3

[Armen [:armenzg]]

(In reply to comment #2)
> > Are you planning on doing this on the build machine or the test machine?
> 
> It wouldn't be useful to do on the build machine as they are the source of
> these tarballs in the first place.  We should be verifying artifacts for
> repacks and tests.

My reasoning was that if the build job knows that what it just uploaded to ftp is corrupted we could prevent sendchanges from happening but your approach should work too.

Comment 4

[Chris Cooper [:coop] (he/him)]

I agree that checksum-ing would be an improvement here. Let's make sure that makes it into the mozharness-equiv of this bug.