468975 - Sign Thunderbird 3 Beta 1 Build 3 complete MARs

Status: RESOLVED FIXED

(Mozilla Messaging Graveyard :: Release Engineering, defect, P3)

Description

[Philippe M. Chiasson (:gozer)]

Files are ready to be signed, and have been massaged into the new directory layout.

http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/3.0b1-candidates/build3

Note, these are just the complete MARs, as the previous one were busted (bug 468488)

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

Hmm, I may be guilty of not thinking hard enough when I said this was possible. How were these mar files created ? For them to be useful, all the files enclosed bitwise identical to what you shipped in the installer.

Comment 2

[Philippe M. Chiasson (:gozer)]

I redeployed the build archives to the build boxes, and re-ran:

$> make -C $OBJDIR/mail/locales installers-$locale MOZ_MAKE_COMPLETE_MAR=1

for each locale and copied the MARs over.

FYI, build archives are complete tarballs of whatever was left from running the original builds, absolutely everything included. They were unpacked in the same location as when the original builds were made, and on the same machines.

Should be bitwise identical packaged files, no?

Is there an easy way to test this ?

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

OK, I mis-thought that the en-US mar was also busted. So the difficulty is that each locale has a localized uninstall/helper.exe which is already signed, and the scripts assume they are updating both the installer and the mar at the same time. Really we want to fish out the already signed helper.exe and use that. I'll have a look if they can be convinced to work.

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: mar utilty for linux

I haven't had time to get to this as I hoped, and won't do until the new year. In case you want to take it and try yourself, the basic process is

for each locale
  unpack installer with 7zip
  copy nonlocalized/* firefox/
  repeat for localized/, optional/

  make dir to unpack mar
  unpack complete update with unwrap_full_update.pl [1] ...
    and mar [attached with this]
  diff firefox/ and unwrapped mar
  substitute files for files that differ

  recreate mar with make_full_update.sh [2]
done

Extra points for verifying that any files replaced have the same hash in the unpacked mar as in the *unsigned* installer. Also note there are supposed to be some files not in the mar, eg channel-prefs.js, you can pick a random Firefox release to verify the complete list.


[1] http://mxr.mozilla.org/seamonkey/source/tools/update-packaging/unwrap_full_update.pl
[2] http://mxr.mozilla.org/seamonkey/source/tools/update-packaging/make_full_update.sh

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

Oh, and we have a copy of both signed and unsigned builds on the signing machine if you need a copy uploaded somewhere.

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

I started doing a comparison of mar files with the archives/installers for each platform, and as a result I'm not sure this is worth doing. There's quite a bit to fix, and the shortest path may be to just give everyone using b1 a complete update to b2.

On all three platforms
* chrome/${locale}.jar differs, probably because zip includes timestamps. The actual files inside the jar don't appear to differ
* defaults/pref/all-l10n.js differs in the comment lines, where the path changed

On linux only
* ordering in chrome.manifest differs. Really this file should not be there at all (bug 472431)

On win32 only
* All the dll's differ, which is a result of building the mars using the unsigned binaries from the original build

All these things would have to be resolved for the 3.1 complete mars to reflect what people have installed, and hence be able to generate a useful partial update.

Comment 7

[Mark Banner (:standard8)]

Well we definitely need to do updates from b1 to b2 I think - we need to test the mechanism before we get to release.

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

I'm proposing only doing complete updates, rather than no updates at all.

Comment 9

[Philippe M. Chiasson (:gozer)]

(In reply to comment #8)
> I'm proposing only doing complete updates, rather than no updates at all.

Just double-checking to be sure. So you mean we'd do complete updates from alpha*/beta1 => beta2, and could do beta2 => beta3 with partial updates ?

Comment 10

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #9)
> Just double-checking to be sure. So you mean we'd do complete updates from
> alpha*/beta1 => beta2, and could do beta2 => beta3 with partial updates ?

Right (assuming there are no further problems with complete mar generation for b2 and b3).

Comment 11

[Mark Banner (:standard8)]

(In reply to comment #10)
> (In reply to comment #9)
> > Just double-checking to be sure. So you mean we'd do complete updates from
> > alpha*/beta1 => beta2, and could do beta2 => beta3 with partial updates ?
> 
> Right (assuming there are no further problems with complete mar generation for
> b2 and b3).

Is there a way we could test this or simulate it now before we release b2? Is it worth doing that? From the looks of it, we need to fix a TB version of bug 472431 at least before b2.

Comment 12

[Philippe M. Chiasson (:gozer)]

Can't we try this by trying to provide complete updates from the 3.0 alphas to 3.0 beta1 ?

Comment 13

[Nick Thomas [:nthomas] (UTC+12)]

You're getting en-US testing from nightly updates, but it was broken locales which caused the problem for b1. Without spinning up a test release I'm not sure how you would test this ahead of time.

Reassigning to gozer for whatever testing you guys want to do.

Comment 14

[Philippe M. Chiasson (:gozer)]

The complete MARs for Beta 2 were produced correctly and even signed, so I consider this issue closed. Over to bug 471584.