1260986 - Thunderbird 38.7.1 SHA512SUMS file is incorrect for one l10n chunk on mac

Status: RESOLVED WONTFIX

(Release Engineering :: Release Requests, defect)

Description

[Nick Thomas [:nthomas] (UTC+12)]

For the mac locales is, it, ja-JP-mac, ko, lt, nb-NO, the hashes for the updates are different in the update server than in the SHA512SUMS file (see output). That script is only checking updates but it's likely the dmg's have the same issue.

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: output

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

Here's what I think happened
* chunk 6 has locales is,it,ja-JP-mac,ko,lt,nb-NO - matching the set in attachment 8736607 [details]
* the first l10n repack attempt fails to sign the dmgs
https://archive.mozilla.org/pub/thunderbird/candidates/38.7.1-candidates/build1/logs/release-comm-esr38-macosx64_repack_6-bm84-build1-build1.txt.gz
* a second attempt runs with error between 2016-03-23 14:33 and 16:01 Pacific
https://archive.mozilla.org/pub/thunderbird/candidates/38.7.1-candidates/build1/logs/release-comm-esr38-macosx64_repack_6-bm84-build1-build2.txt.gz
* the automation continues on, running the checksums builder at 16:08, taking just a minute
https://archive.mozilla.org/pub/thunderbird/candidates/38.7.1-candidates/build1/logs/release-comm-esr38-thunderbird_checksums-bm91-build1-build0.txt.gz
* the l10n repack chunk runs again, between 18:32 and 20:02. Looks like the rebuild button was pressed on a buildbot master. This overwrites the files in the candidates directory, and they have new hashes because of various timestamps inside the dmg.

To fix this up we can run the checksums job again, and do a little S3 magic to copy the new file over from the candidates directory to releases.

Comment 3

[Kim Moir [:kmoir] ET]

I tried to rerun the job but it failed

07:17:48     INFO - Running main action method: collect_individual_checksums
07:17:48     INFO - Connecting to S3
07:17:48     INFO - Connecting to bucket net-mozaws-prod-delivery-archive
07:17:49    FATAL - Uncaught exception: Traceback (most recent call last):
07:17:49    FATAL -   File "/builds/slave/tb-rel-c-esr38-tb_sums-0000000/scripts/mozharness/base/script.py", line 1693, in run
07:17:49    FATAL -     self.run_action(action)
07:17:49    FATAL -   File "/builds/slave/tb-rel-c-esr38-tb_sums-0000000/scripts/mozharness/base/script.py", line 1635, in run_action
07:17:49    FATAL -     self._possibly_run_method(method_name, error_if_missing=True)
07:17:49    FATAL -   File "/builds/slave/tb-rel-c-esr38-tb_sums-0000000/scripts/mozharness/base/script.py", line 1576, in _possibly_run_method
07:17:49    FATAL -     return getattr(self, method_name)()
07:17:49    FATAL -   File "scripts/scripts/release/generate-checksums.py", line 148, in collect_individual_checksums
07:17:49    FATAL -     bucket = self._get_bucket()
07:17:49    FATAL -   File "scripts/scripts/release/generate-checksums.py", line 141, in _get_bucket
07:17:49    FATAL -     self.bucket = conn.get_bucket(self.bucket_name)
07:17:49    FATAL -   File "/builds/slave/tb-rel-c-esr38-tb_sums-0000000/build/venv/lib/python2.7/site-packages/boto/s3/connection.py", line 502, in get_bucket
07:17:49    FATAL -     return self.head_bucket(bucket_name, headers=headers)
07:17:49    FATAL -   File "/builds/slave/tb-rel-c-esr38-tb_sums-0000000/build/venv/lib/python2.7/site-packages/boto/s3/connection.py", line 535, in head_bucket
07:17:49    FATAL -     raise err
07:17:49    FATAL - S3ResponseError: S3ResponseError: 403 Forbidden

Does having updates disabled also mean we can't push to s3?

Comment 4

[Kim Moir [:kmoir] ET]

talked to bhearsum and will hold off making any changes until change freeze is over

Comment 5

[Kim Moir [:kmoir] ET]

So I tried to run this today bit it ran for 38.7.2 since we have built that since then 38.7.1.  So should I run this as
scripts/scripts/release/generate-checksums.py --stage-product thunderbird --version 38.7.1 etc. manually or does it matter since we will release 38.7.2 soon?

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

If you are time constrained then it's OK to WONTFIX.