1085026 - AUS manifest reporting wrong hash value for MAR files on Oak, Ash, and Maple

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Brian R. Bondy [:bbondy]]

On Oak, it turns out that the manifest gives the wrong SHA value for the advertised MAR file.

The entry says:
hashFunction="sha512" hashValue="4d4b1c293862ec10fc886c2dd548bd125efe89a8" (full manifest below)

But the real SHA512 hash for the mar file mentioned is:
adbbfb44cccb060ed3e71623a0e3ef5fcc87ee3d24b4cc728e77a3b5e63ff3161552b253a6115d37c4dda62a8d4267d5e33d5d3d18c2e280b87819709db5ce24

After trying out a bunch of different things, I found out that the hash value it gives is the SHA1 hash, and not the SHA512 hash.


At the time of my testing, the manifest URL is located at:
https://aus4.mozilla.org/update/3/Firefox/36.0a1/20141017040201/WINNT_x86-msvc/en-US/nightly-oak/Windows_NT%206.3.0.0%20%28x64%29/default/default/update.xml?force=1

Which produced this manifest:
<updates>
<update type="minor" displayVersion="36.0a1" appVersion="36.0a1" platformVersion="36.0a1" buildID="20141018040203">
<patch type="complete" URL="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/10/2014-10-16-13-55-55-oak/firefox-36.0a1.en-US.win32.complete.mar" hashFunction="sha512" hashValue="4d4b1c293862ec10fc886c2dd548bd125efe89a8" size="47787586"/>
</update>
</updates>

Comment 1

[Brian R. Bondy [:bbondy]]

More on this: x64 builds correctly give and advertise SHA512, but x86 builds continue to say SHA512, but really contain SHA1s.

Comment 2

[Brian R. Bondy [:bbondy]]

Adding needinfo just to make sure you see this, especially in case it affects something more than just oak.

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

Might be fallout from mach mozharness.

Comment 4

[[e-mail] (:bhearsum)]

Based on this I suspect that Nick is right. Note that the complete looks like a sha1, and the partial looks like a sha512:
05:13:20     INFO - Dumping config to /builds/slave/oak-lx-ntly-000000000000000000/balrog_props.json.
05:13:20     INFO - {'properties': {'appName': 'Firefox',
05:13:20     INFO -                 'appVersion': '36.0a1',
05:13:20     INFO -                 u'basedir': u'/builds/slave/oak-lx-ntly-000000000000000000',
05:13:20     INFO -                 u'branch': u'oak',
05:13:20     INFO -                 u'buildername': u'Linux oak nightly',
05:13:20     INFO -                 u'buildid': u'20141021040207',
05:13:20     INFO -                 u'buildnumber': 5,
05:13:20     INFO -                 u'builduid': u'53022887de78458893362f59a0d45e36',
05:13:20     INFO -                 u'completeMarFilename': u'firefox-36.0a1.en-US.linux-i686.complete.mar',
05:13:20     INFO -                 u'completeMarHash': u'225d0e1932917ecb3b4b7c71a8d9ac06610efb94',
05:13:20     INFO -                 u'completeMarSize': 52378590,
05:13:20     INFO -                 u'completeMarUrl': u'http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/10/2014-10-21-04-02-07-oak/firefox-36.0a1.en-US.linux-i686.complete.mar',
05:13:20     INFO -                 'got_revision': u'02550df60211',
05:13:20     INFO -                 'hashType': 'sha512',
05:13:20     INFO -                 u'jsshellUrl': u'http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/oak-linux/1413889327/jsshell-linux-i686.zip',
05:13:20     INFO -                 u'master': u'http://buildbot-master94.srv.releng.use1.mozilla.com:8001/',
05:13:20     INFO -                 'num_ctors': 71,
05:13:20     INFO -                 u'packageUrl': u'http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/oak-linux/1413889327/firefox-36.0a1.en-US.linux-i686.tar.bz2',
05:13:20     INFO -                 'partialMarFilename': 'firefox-36.0a1.en-US.linux-i686.partial.20141021040207-20141021040207.mar',
05:13:20     INFO -                 'partialMarHash': '68e2bf83a00721ebaf65d1d5bd10500e8a3b45d1cee4483632a083fea59bb3d030396991e30db36f678684c4d0231bf5851b432a2db563fd7894ace747bc393c',
05:13:20     INFO -                 'partialMarSize': 6618,
05:13:20     INFO -                 u'platform': u'linux',
05:13:20     INFO -                 'previous_buildid': '20141021040207',
05:13:20     INFO -                 u'product': u'firefox',
05:13:20     INFO -                 u'project': u'',
05:13:20     INFO -                 u'repository': u'',
05:13:20     INFO -                 u'revision': u'02550df602110228ea56505b532ceddbc250c5a6',
05:13:20     INFO -                 u'scheduler': u'oak nightly',
05:13:20     INFO -                 u'slavename': u'bld-linux64-spot-059',
05:13:20     INFO -                 'sourcestamp': '02550df60211',
05:13:20     INFO -                 u'symbolsUrl': u'http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/oak-linux/1413889327/firefox-36.0a1.en-US.linux-i686.crashreporter-symbols.zip',
05:13:20     INFO -                 'testresults': [('num_ctors', 'num_ctors', 71, '71')],
05:13:20     INFO -                 u'testsUrl': u'http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/oak-linux/1413889327/firefox-36.0a1.en-US.linux-i686.tests.zip'}}
05:13:20     INFO - Calling Balrog submission script
05:13:20     INFO - retry: Calling <bound method FxDesktopBuild.run_command of <__main__.FxDesktopBuild object at 0x1d6d490>> with args: (['python', '/builds/slave/oak-lx-ntly-000000000000000000/build/tools/scripts/updates/balrog-submitter.py', '--build-properties', '/builds/slave/oak-lx-ntly-000000000000000000/balrog_props.json', '--api-root', 'https://aus4-admin.mozilla.org', '--username', 'ffxbld', '-t', 'nightly', '--credentials-file', '/builds/slave/oak-lx-ntly-000000000000000000/oauth.txt', '--verbose'],), kwargs: {}, attempt #1
05:13:20     INFO - Running command: ['python', '/builds/slave/oak-lx-ntly-000000000000000000/build/tools/scripts/updates/balrog-submitter.py', '--build-properties', '/builds/slave/oak-lx-ntly-000000000000000000/balrog_props.json', '--api-root', 'https://aus4-admin.mozilla.org', '--username', 'ffxbld', '-t', 'nightly', '--credentials-file', '/builds/slave/oak-lx-ntly-000000000000000000/oauth.txt', '--verbose']
05:13:20     INFO - Copy/paste: python /builds/slave/oak-lx-ntly-000000000000000000/build/tools/scripts/updates/balrog-submitter.py --build-properties /builds/slave/oak-lx-ntly-000000000000000000/balrog_props.json --api-root https://aus4-admin.mozilla.org --username ffxbld -t nightly --credentials-file /builds/slave/oak-lx-ntly-000000000000000000/oauth.txt --verbose
05:13:20     INFO -  Balrog request to https://aus4-admin.mozilla.org/releases/Firefox-oak-nightly-20141021040207
05:13:20     INFO -  Data sent: None
05:13:20     INFO -  Starting new HTTPS connection (1): aus4-admin.mozilla.org
05:13:21     INFO -  "HEAD /releases/Firefox-oak-nightly-20141021040207 HTTP/1.1" 200 0
05:13:21     INFO -  Balrog request to https://aus4-admin.mozilla.org/releases/Firefox-oak-nightly-20141021040207/builds/Linux_x86-gcc3/en-US
05:13:21     INFO -  Data sent: {'product': u'Firefox', 'hashFunction': u'sha512', 'alias': 'null', 'schema_version': 4, 'data_version': '2', 'copyTo': '["Firefox-oak-nightly-latest"]', 'version': u'36.0a1', 'data': '{"buildID": "20141021040207", "platformVersion": "36.0a1", "displayVersion": "36.0a1", "appVersion": "36.0a1", "completes": [{"fileUrl": "http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/10/2014-10-21-04-02-07-oak/firefox-36.0a1.en-US.linux-i686.complete.mar", "hashValue": "225d0e1932917ecb3b4b7c71a8d9ac06610efb94", "from": "*", "filesize": 52378590}]}'}

Comment 5

[Brian R. Bondy [:bbondy]]

Yep, just to clarify in comment 1 I mentioned it was an sha1, but I did also write code to verify 100% that it is 100% an sha1 but the manifest says sha512 for the type.

Comment 6

[Michael Shal [:mshal]]

Attachment: sha512

Use sha512 for the mar hashes.

Comment 7

[Brian R. Bondy [:bbondy]]

So was this affecting only oak or also other things? Partial being 512 but full being sha1.

Comment 8

[Jordan Lund (:jlund)]

(In reply to Brian R. Bondy [:bbondy] from comment #7)
> So was this affecting only oak or also other things? Partial being 512 but
> full being sha1.

so far, the code that would cause this bug has only been rolled out to all twig branches bar fx-team and alder. 

So this will also affect Maple and Ash (ash is a testing/staging branch). The other branches don't run nightlies so they are unaffected.

Apologies for any inconveniences this caused.

Comment 9

[Brian R. Bondy [:bbondy]]

No problem, thanks for fixing!

Comment 10

[Michael Shal [:mshal]]

inbound: https://hg.mozilla.org/integration/mozilla-inbound/rev/2d393d07f218
try: https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=434c88299059

Comment 11

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/2d393d07f218

Comment 12

[Michael Shal [:mshal]]

aurora uplift for bug 1093897: https://hg.mozilla.org/releases/mozilla-aurora/rev/14de6cbff9c2

Comment 13

[Michael Shal [:mshal]]

beta uplift for bug 1093897: https://hg.mozilla.org/releases/mozilla-beta/rev/72d8ba95b2db