1392138 - Thunderbird 52.3.0 SHA512 differences

Status: RESOLVED FIXED

(Release Engineering :: Release Automation, defect)

Description

[Nick Thomas [:nthomas] (UTC+12)]

From https://support.mozilla.org/en-US/questions/1172029:

Hi, i downloaded thunderbird for linux from here: https://ftp.mozilla.org/pub/thunderbird/releases/52.3.0/linux-x86_64/de/

when checking the sha512 i always get the c37dfd965e1a59f5c62dd849078e927752aedd6590a00aed155c53cafb0912d5a8178b4e235e996f93a99bd0f168c534b9253a7877b62c574d5ea8cc1682fb73 thunderbird-52.3.0.tar.bz2

but in the sha512 on your server the value is diffrent 010b56755f7107e32df25481a2febc7d0468bd27a6ec5521af32d80382c96653a2989a70e2dd7350452ebfb25f5691a161ff5fd74228b9b3670475942f7b0f6b linux-x86_64/de/thunderbird-52.3.0.tar.bz2

i try downloading for more then 10 times now on diffrent systems but the result is always the same.

please check the files you provide to your users the didnt match!!!

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

We did three 'repack_2/10 on linux64' jobs.

The first job failed - https://archive.mozilla.org/pub/thunderbird/candidates/52.3.0-candidates/build1/logs/release-comm-esr52-linux64_repack_2-bm70-build1-build0.txt.gz - in the da locale (ie before de).

The second job completed fine - https://archive.mozilla.org/pub/thunderbird/candidates/52.3.0-candidates/build1/logs/release-comm-esr52-linux64_repack_2-bm73-build1-build2.txt.gz. The de installer has a hash of 010b56755f7107e32df25481a2febc7d0468bd27a6ec5521af32d80382c96653a2989a70e2dd7350452ebfb25f5691a161ff5fd74228b9b3670475942f7b0f6b, ie this is the value in the SHA512SUMS file. The checksums file was signed at 2017-08-15 09:16 (this happens just prior to uploading the installer etc).

A third repack also ran and was fine - https://archive.mozilla.org/pub/thunderbird/candidates/52.3.0-candidates/build1/logs/release-comm-esr52-linux64_repack_2-bm73-build1-build3.txt.gz. The hash is c37dfd965e1a59f5c62dd849078e927752aedd6590a00aed155c53cafb0912d5a8178b4e235e996f93a99bd0f168c534b9253a7877b62c574d5ea8cc1682fb73, ie what the CDN is serving. The checksums file for this was signed at 2017-08-15 12:05:54.

So we have the last installer, but not the last-installer data in SHA512SUMS.

The job which generates SHA512SUMS was https://archive.mozilla.org/pub/thunderbird/candidates/52.3.0-candidates/build1/logs/release-comm-esr52-thunderbird_checksums-bm73-build1-build1.txt.gz. It started at 2017-08-15 09:04:01 and completed 24 seconds later.

So the issue is that the third repack replaced the files for de after the checksums job ran.

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

The hashes for the update files and the xpi are likely to be wrong too. And also dsb, which followed de in the the chunk. Interestingly this doesn't affect ca, cs, cy, or da, which preceded those locales in the chunk.

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

Ah, the log timestamps leave of the timezone. Looking at the buildbot masters, the 2nd job was an auto-retry after the network disconnect in the first job (may have gotten outbid on the AWS spot instance). It ran between Tue Aug 15 05:41:40 2017 and 06:21 Pacific.

The third job was retriggered by an unknown human and ran between 08:31 and 09:10. The checksums job did run between 09:04:01 and 09:04:25. So were just a little unlucky with the timing that the third job wasn't needed, but when it ran it replaced just a couple of locales.

Anyway, the solution is to rerun the checksums job, copy it into releases/52.3.0/, and ask CloudOps to purge the CDNs.

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

There were also failures in linux64 10/10 and linux 7/10.

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Diff of SHA512SUMS file

Hashes were updated for:
linux-i686/nb-NO/thunderbird-52.3.0.tar.bz2
linux-i686/nl/thunderbird-52.3.0.tar.bz2
linux-i686/nn-NO/thunderbird-52.3.0.tar.bz2
linux-i686/pa-IN/thunderbird-52.3.0.tar.bz2
linux-i686/pl/thunderbird-52.3.0.tar.bz2
linux-i686/pt-BR/thunderbird-52.3.0.tar.bz2

linux-x86_64/de/thunderbird-52.3.0.tar.bz2
linux-x86_64/dsb/thunderbird-52.3.0.tar.bz2

mar and xpi file generation must be reproducible since they haven't changed.

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

https://archive.mozilla.org/pub/thunderbird/releases/52.3.0/SHA512SUMS has been refreshed, along with the gpg signature SHA512SUMS.asc. Until Bug 1392141 is resolved the CDNs may serve the previous copy of those.

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

All done here. Ton, could you please update the support posts, thanks.

Comment 9

[Ton [:Tonnes]]

Will do. Thanks for fixing!