459878 - Tracking bug: do major update from FF3.0.x to FF3.5 to make sure it all works

Status: VERIFIED FIXED

(Release Engineering :: General, defect, P3)

Description

[juan becerra [:juanb]]

We should generate a set of major updates prior to FF3.1 release, so QA has time to test and report problems before FF3.1 is shipped.

Comment 1

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Yep, we did this for FF2->FF3.0, and it was very useful. (see bug#394046 for details).

Triaging to Future for now, until we get through some other stuff first.

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

Lets aim for 3.0.4 to 3.1b2, waiting for those releases.

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

Gonna be from 3.0.5 to 3.1b2.

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: New update generation and verification configs

This adds a patcher config, and three verify configs, for a test major update. We'll never push release or beta channels live for this, just betatest and releasetest. The locale list for the verify configs is 3.0.5 less da, gl, ku, mk, mn, oc, sr, and th, based on diffing shipped-locales. Using CVS since we haven't switched Fx3.0.x releases over yet (need to verify all is well in bug 467310).

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Tweak update generation and verification configs

I committed this followup fix for attachment 355358 [details] [diff] [review], as I'd 
* left out si and sl from the list of 3.0.5 locales in moz19-branch-major-update-patcher2.cfg
* hadn't tidied up the ja/ja-JP-mac per platform in update verify, so we were trying to test ja-JP-mac on linux and win32, and ja on Mac

Generated and tested the first time before these were landed.

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Results for first run - win32

win32 looks fine to me. The wikipedia-en-CN.xml searchplugin issue is already fixed, and Axel has a patch for the install.rdf/chrome.manifest issue up on bug 472431. --> No further action required on this platform, on this test.

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Results for first run - linux

Only a leftover libjemalloc.so is new here (compared to win32), since that moved into firefox-bin. We'll need to add that to removed-files.in.

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Results for first run - mac

Mac is, as ever, the problem child when it comes to packaging, because we don't control it with a manifest. There are 2 files, and 2 directories removed between 3.0 and 3.1 that need cleaning up.

Comment 9

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Fix up removed-files.in and unix/packages-static (first run)

This is against mozilla-central for baking there, with the goal to get it to mozilla-1.9.1 before 3.1b3 freezes. I'm hoping to test that the updater can handle the space in plugins/Default Plugin.plugin/ soon.

Comment 10

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #9)
> I'm hoping to test that the updater can handle the space in 
> plugins/Default Plugin.plugin/ soon.

This worked fine when I modified a partial update for a Shiretoko nightly - the update.manifest encloses the path in spaces. It does leave behind some empty dirs which the updater can't do anything about, but about:plugins reports only a Default Plugin with "File name: DefaultPlugin.plugin".

Comment 11

[Nick Thomas [:nthomas] (UTC+12)]

This bug encompasses the prep. work to make sure we're able to offer a major update promptly when 3.1 is done. Requesting blocking for it.

Comment 12

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 357623 [details] [diff] [review]
Fix up removed-files.in and unix/packages-static (first run)

Baking on mozilla-central: http://hg.mozilla.org/mozilla-central/rev/501b8fce34fa

Comment 13

[Nick Thomas [:nthomas] (UTC+12)]

Works fine in my testing of mac and linux, and no bugs have been filed by nightly users. Ready for mozilla-1.9.1.

Comment 14

[Nick Thomas [:nthomas] (UTC+12)]

Bugzilla is not configured for approval1.9.1 in this component, so CC beltzner for comment based approval. There has been no fallout filed against this patch, which removes files no longer in Firefox 3.1 when updating from 3.0.x.

Comment 15

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #14)
> Bugzilla is not configured for approval1.9.1 in this component, so CC beltzner
> for comment based approval. There has been no fallout filed against this patch,
> which removes files no longer in Firefox 3.1 when updating from 3.0.x.

Referring to attachment 357623 [details] [diff] [review] here.

Comment 16

[Reed Loden [:reed]]

(In reply to comment #14)
> Bugzilla is not configured for approval1.9.1 in this component, so CC beltzner
> for comment based approval.

That's because somebody messed up the original flag. I've fixed it. Request away!

Comment 17

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 357623 [details] [diff] [review]
Fix up removed-files.in and unix/packages-static (first run)

http://hg.mozilla.org/releases/mozilla-1.9.1/rev/0fb1364ef059

Comment 18

[Mike Beltzner [:beltzner, not reading bugmail]]

We don't block on meta bugs; if there are unique issues, please nominate them.

Comment 19

[Nick Thomas [:nthomas] (UTC+12)]

Okie doke. 

Waiting for 3.1b3 now (--> P3 in the RelEng scheme)

Comment 20

[Al Billings [:abillings - ex-MoCo]]

I also took a look at the 3.0.5 to 3.1b2 updates that Nick generated on all platforms and I did not find any problems.

I have noticed that using a profile with passwords saved in 3.1 doesn't have those passwords accessible in 3.0 so I assume we did something with how we save passwords in 3.1.

Comment 21

[Justin Dolske [:Dolske]]

Yes. We're using mozStorage for passwords now (bug 288040), so FF3.0 won't see any passwords added in FF3.1.

Comment 22

[Nick Thomas [:nthomas] (UTC+12)]

We'll generate a 3.0.7 to 3.1b3 test update as we get close to shipping 3.1b3.

Comment 23

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Update generation and verification configs for 3.0.7 --> 3.1b3

Bumps buildID's, locale lists (no change between 3.0.5 and 3.0.7; 3.1b2 to 3.1b3 is -cy and +da,fa,gl,ku,mk,mn,oc,sr,th,vi, but not all are available for 3.0.7), and switches d/l url to ftp.m.o to avoid lottery of mirror speed.

Comment 24

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 366479 [details] [diff] [review]
Update generation and verification configs for 3.0.7 --> 3.1b3

See comment #23 for notes on this patch

Comment 25

[Nick Thomas [:nthomas] (UTC+12)]

Promised QA that this would be ready for them Thursday.

Comment 26

[Chris Cooper [:coop] (he/him)]

Comment on attachment 366479 [details] [diff] [review]
Update generation and verification configs for 3.0.7 --> 3.1b3

fa and vi are missing from the platform major.cfgs, but I'm assuming that's because they're not in 3.0.x (according to comment 23 and a quick check of moz19-branch-major-update-patcher2.cfg).

Comment 27

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 366479 [details] [diff] [review]
Update generation and verification configs for 3.0.7 --> 3.1b3

(In reply to comment #26)
> (From update of attachment 366479 [details] [diff] [review])
> fa and vi are missing from the platform major.cfgs, but I'm assuming that's
> because they're not in 3.0.x

Exactly. Given the time it to create patches with that sort of exception it's probably just easier to put in all 3.0.x locales and check failures after the verify completes.

Comment 28

[Nick Thomas [:nthomas] (UTC+12)]

3.0.7 --> 3.1b3 major update test is available on betatest channel. Automated checks are running and the first couple of locales look good.

Comment 29

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Results for 3.0.7 --> 3.1b3 run - win32

No big issues here, just a couple of locales with mucked up searchplugins (bug 483093, bug 483095).

Comment 30

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Results for 3.0.7 --> 3.1b3 run - linux

No new problems since last time, and libjemalloc.so is fixed up.

Comment 31

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Results for 3.0.7 --> 3.1b3 run - mac

No problems here, and we successfully fixed up a few issues.

No actions to take on packaging manifests this time. Will check 3.5b4pre a few days before code freeze to see if any files got removed but not in removed-files.in.

Comment 32

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Explore directories in only one side of diff when doing update verify

This is a better version of attachment 304252 [details] [diff] [review] - it recursively shows the contents of directories that diff reports with "Only in ...".

Comment 33

[Tony Chung [:tchung]]

hey nick, why was this tracking bug deprioritized to P3?

Comment 34

[Nick Thomas [:nthomas] (UTC+12)]

hey tony, there's nothing further for me to do at this moment (see comment #31). If you find any issues at this point it seems very likely they will be filed against Firefox itself.

Comment 35

[Nick Thomas [:nthomas] (UTC+12)]

And I probably should have said that P2/P3 mean something different for RelEng compared to Firefox/platform bugs. Roughly, P2 is working on it in the next week, P3 is longer term.

Comment 36

[Tony Chung [:tchung]]

okay sounds good.  I thought RelEng's Px definitions would be different than devs, so i'm glad you clarified.

Comment 37

[Nick Thomas [:nthomas] (UTC+12)]

I compared en-US builds for 3.1b3 against today's nightlies. Files only found in one build are: 
* old-homepage-default.properties is only in the 3.1b3 builds via http://mxr.mozilla.org/mozilla1.9.1/source/other-licenses/branding/firefox/locales/Makefile.in#52
* linux nightly now has an updater.png from bug 473337
* mac nightly now has a components/necko_wifi.xpt from bug 479898 (shows up here because mac doesn't combine xpts like other platforms)

So nothing removed in nightlies that needs handling.

Comment 38

[Nick Thomas [:nthomas] (UTC+12)]

Going to refresh this for 3.0.10 to 3.5b4.

Comment 39

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Update generation and verification configs for 3.0.10 --> 3.5b4

In the update config: update versions and buildIDs, the locale list for 3.5b4, and use release copy of files in all locations. The "details" location isn't live yet, s/www/www-trunk.stage/ if you're curious. Going to ftp.m.o is a convenience for testing (avoiding slow mirrors) which will go away at 3.5 final.

In the verify config: update versions and buildIDs, add cy to locale list (we now have a 3.5 locale for every 3.0.x locale), switch to release location for "to" declaration. Sorry, I know this is a pain, it's easier if you use XCode's Filemerge (opendiff on the command line) between a pristine and patched copy of the files, it highlights the chars that changed.

Comment 40

[[e-mail] (:bhearsum)]

Comment on attachment 375982 [details] [diff] [review]
Update generation and verification configs for 3.0.10 --> 3.5b4

Everything looks good to me.

Comment 41

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 375982 [details] [diff] [review]
Update generation and verification configs for 3.0.10 --> 3.5b4

 Checking in moz19-branch-major-update-patcher2.cfg;
 new revision: 1.4; previous revision: 1.3
and 
 http://hg.mozilla.org/build/tools/rev/1dacbbe5613e

Comment 42

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Results for 3.0.10 --> 3.5b4 run - win32

No problems here, just gl needing to fix up their searchplugin list still (bug 483093)

Comment 43

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Results for 3.0.10 --> 3.5b4 run - linux

No problems for us to fix here, just gl searchplugins as noted above.

Comment 44

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Results for 3.0.10 --> 3.5b4 run - mac

This is also fine.

Comment 45

[Nick Thomas [:nthomas] (UTC+12)]

Al, do you want a 3.0.11/12 --> 3.5b99 test path set up ?

Comment 46

[Al Billings [:abillings - ex-MoCo]]

No. We only want to do this for the RC so, if the RC becomes the release, it will already be tested. We're snowed under with TB 2.0.0.22 right now so I expect that we will have an RC shortly after we're done.

Comment 47

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Update generation and verification configs for 3.0.11 --> 3.5rc2 build2

r? for a sanity check. We lost mn and tr on the 3.5 branch, so we're going to get update verify errors there. Figured leaving the complete of 3.0.11 locales in the update verify was the best way forward.

Comment 48

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Results for 3.0.11 --> 3.5rc2 build2 run - all platforms

Look fine to me. We've removed deprecated searchplugins in the past but it's too late to do that for 3.5.

Comment 49

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 383873 [details] [diff] [review]
Update generation and verification configs for 3.0.11 --> 3.5rc2 build2

moz19-branch-major-update-patcher2.cfg: new revision: 1.5; previous revision: 1.4
build/tools: committed changeset 305:bb0bb3c44011

Note to self: Deliberately removed prettyVersion from 3.5rc2 block. We'll have to change this config if 3.5rc2 becomes 3.5 final, moving url to betatest-url and then url points to bouncer.

Comment 50

[Marco Bonardo [:mak]]

if the work here is finished, could you please mark status1.9.1 accordingly? or fixed1.9.1 keyword at least. i'm querying bugzilla for unfinished 1.9.1 bugs and this is still marked as unfinished. or any other way to mark that we are done here, that i can query on.