853036 - use lowercase sha512 in all snippets / balrog blobs

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[[e-mail] (:bhearsum)]

The current AUS server uses a lowercase "sha512" in snippets for the hash function. We should do the same for Balrog snippets to make comparisons easier.

This might involve something in the balrog code itself, or maybe just something in the client code.

Comment 1

[[e-mail] (:bhearsum)]

Looks like we actually aren't even consistent in the snippets on aus3.m.o. It's looking like we use the lowercase for nightlies and uppercase for releases. Clearly they both work, so let's just start using uppercase everywhere, since that's what Balrog is already using.

Comment 2

[[e-mail] (:bhearsum)]

Attachment: buildbot-configs

Turns out there's a bunch of places where we have 'sha512'. All of these end up doing one or more of the following with it:
* Printing it directly to the snippet
* Using it with Python's hashlib:
>>> hashlib.new('SHA512')
<SHA512 HASH object @ 0x7f5343249c10>
>>> 
* Using it with openssl:
➜  balrog git:(master) openssl dgst -SHA512 Makefile 
SHA512(Makefile)=xxxxxxx

I've run these patches through checkconfig but I haven't done any real staging tests. I think they're safe enough to not bother with that?

Comment 3

[[e-mail] (:bhearsum)]

Attachment: buildbotcustom

Comment 4

[[e-mail] (:bhearsum)]

Attachment: mozharnes

Comment 5

[[e-mail] (:bhearsum)]

Attachment: tools

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 727328 [details] [diff] [review]
buildbot-configs

Looks like it should be fine without a staging release.

Comment 7

[Armen [:armenzg]]

Merged change(s) from this bug and I reconfigured the masters.

Comment 8

[[e-mail] (:bhearsum)]

This broke at least Android.

Comment 9

[[e-mail] (:bhearsum)]

Comment on attachment 727331 [details] [diff] [review]
tools

I confirmed that only Android is broken. I'm only going to back out the parts that affect it.

Comment 10

[[e-mail] (:bhearsum)]

I filed bug 853823 to get the Android updater to be case insensitive, but we still need a workaround here, as I don't anticipate that patch being accepted as a backport.

Comment 11

[[e-mail] (:bhearsum)]

(In reply to Ben Hearsum [:bhearsum] from comment #10)
> I filed bug 853823 to get the Android updater to be case insensitive, but we
> still need a workaround here, as I don't anticipate that patch being
> accepted as a backport.

Seems like the right thing to do here is to switch everything to lowercase instead, including whatever is setting hashFunction in Balrog (which might be done by hand right now...). The alternative is to wait for bug 832454 to be on aurora for a couple of weeks and then re-land the existing patches. That would be cause any users without the patch to get stuck on their current build. We only have 5,500 users on Fennec nightly/aurora.

Comment 12

[Nick Thomas [:nthomas] (UTC+12)]

There seems to be lots of usage out there for both sha512 & SHA512. In the metrics dashboard, about two thirds of aurora users are on builds from the last two weeks. Unknown if users on even older are even moving anywhere, but I guess we should use lower case to not strand them.

Comment 13

[[e-mail] (:bhearsum)]

Comment on attachment 727328 [details] [diff] [review]
buildbot-configs

Backing out per comments #11 and 12.

Comment 14

[[e-mail] (:bhearsum)]

Attachment: switch balrog to lowercase

Turns out that our only references in the Balrog code itself are in docs/tests. There's no references to hashFunction in the Balrog client code yet (bug 763004 will be adding one, though.)

I'm going to be manually adjusting the -latest blobs to change their hashFunction to "sha512" today. That plus this patch will finish off this bug.

Comment 15

[[e-mail] (:bhearsum)]

Updated the Balrog blobs. We're now serving lowercase from aus4-dev: https://aus4-dev.allizom.org/update/3/Firefox/14.0a1/20111108031146/Linux_x86_64-gcc3/en-US/nightly/Linux%203.0.0-12-generic%20%28GTK%202.24.6%29/default/default/update.xml?force=1

Comment 16

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 729499 [details] [diff] [review]
switch balrog to lowercase

zomg, huge. I'm going to assume you got them all!

Comment 17

[[github robot]]

Commit pushed to master at https://github.com/mozilla/balrog

https://github.com/mozilla/balrog/commit/7bfb2eb10b0a3e91e65e0367288f46abfc03d970
bug 853036: use lowercase sha512 in all snippets / balrog blobs. r=nthomas

Comment 18

[[e-mail] (:bhearsum)]

This went into production yesterday. Snippets are lowercased again: https://aus3.mozilla.org/update/1/Firefox/3.6.23pre/2011032403/Linux_x86_64-gcc3/en-US/nightly/update.xml?force=1