497956 - Thunderbird 2.0.0.6 and older fail integrity check for Tb2.0.0.22 update, and then fail to apply

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P1)

Description

[Al Billings [:abillings - ex-MoCo]]

When checking for updates in Thunderbird 2.0 in the betatest channel, TB downloads the full update for 2.0.0.22, which then fails the integrity check. It then immediately attempts to download it again and gets stuck waiting to apply the download.

This was verified in multiple locales on Linux, Win32, and OS X. Full updates to 2.0.0.22 from TB 2.0.0.18 and 2.0.0.19 apply correctly and Build has verified that the bits appear to be correct and exactly the same in all instances of full updates. It is suspected that this is a server problem when TB 2.0 requests updates.

Steps to Reproduce
1. Install TB 2.0
2. Edit the channel-prefs.js file in /defaults/pref under TB to have the channel changed from "release" to "betatest"
3. Start TB and check for updates

Result: http://img3.imageshack.us/img3/2538/picture2rfu.png - the status keeps animated and the user is never prompted to restart to apply the download (which means it isn't an issue applying the update after download).

Comment 1

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

additional info:

1) all the build logs files are error free.
2) that the sha512 of the mar files:
thunderbird-2.0.0.22.en-US.linux-i686.complete.mar
thunderbird-2.0.0.22.ja-JP-mac.mac.complete.mar 
...do match the snippets, as expected.

Note: these same exact mar files are used when doing updates from
TB2.0.0.19->2.0.0.22, and that worked fine. This leads us to believe the actual
mar and snippet files are correct. We dont know why they seem to download in some cases, but not others.

mrz: Is it possible there is some change in how files are being served? This is
before beta, so no files shared out to mirrors yet, all are on local stage. Are
there any recent changes to apache/webheads/cache/etc that might cause this?

ss: are there any recent changes to updater that might be involved?

Comment 2

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

raising severity, as this is blocking TB2.0.0.22 going to beta today.

Comment 3

[[e-mail] (:bhearsum)]

Taking this back to RelEng.

The problem lies in the snippets. This was the first release on this branch that used sha512 sums in the snippets. Apparently earlier version of tb don't support sha512 hashes. We need to regenerate all of the snippets (not the MARs though, those are valid).

Comment 4

[Al Billings [:abillings - ex-MoCo]]

Thunderbird 2.0.0.6 will update to 2.0.0.22 without issues. TB 2.0.0.9 (the next release despite version numbers) does not update. So the problem occurred after the 2.0.0.6 release.

Comment 5

[Al Billings [:abillings - ex-MoCo]]

Oops, backwards. 2.0.0.6 does NOT update but 2.0.0.9 does.

Comment 6

[Daniel Veditz [:dveditz]]

This is due to bug 383390 which was fixed in 1.8.1.8

Prior to the fix nsICryptoHash truncated to 32 bytes which is wrong for sha384 and sha512

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

Will be fixed by reverting to UPDATE_PACKAGING_R7 and regenerating snippets.

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

New snippets (using SHA1) now available on the betatest channel. Snippets for beta and release pushes generated and staged.

Notes
https://wiki.mozilla.org/Releases/Thunderbird_2.0.0.22/BuildNotes#Fix_update_snippets