590953 - do major updates from 3.6x to 4.0 to make sure it all works

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[juan becerra [:juanb]]

We need major updates generated to test, before out beta period is done.

Comment 1

[Aki Sasaki (not active)]

This will need to become a blocker to a beta release bug.
Joduinn says definitely by the final beta; preferably the one-before-final beta.

Comment 2

[Mike Taylor [:bear]]

Assigning it to the beta6 bug as a blocker.

Comment 3

[juan becerra [:juanb]]

It's already beta 8 and there are a few things we'd like to start looking into, like add-ons and Sync data. These two features are almost done for Fx4 but they may need the sort of information we'll find during one of these testing cycles. I'm hoping beta 9.

Comment 4

[Benjamin Smedberg]

joduinn, do you have somebody in mind for this unassigned blocker?

Comment 5

[LegNeato]

I'll grab this. As discussed, we will be doing this for beta 10.

TBD if we will test all the new snippet functionality (suppressing we page,
popping infobar, etc) or only focus on current functionality parity.

Comment 6

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #4)
> joduinn, do you have somebody in mind for this unassigned blocker?


Beta9 schedule got moved up, so we'll be doing this as part of beta10. See linked bugs. 

(We've done this "verify MU works before shipping a major release" in FF3.0 and onwards. Beta10 is the soonest we can do this, and also gives us the most time to react to any surprises we find.)

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

Christian, in the past we've done a full set of locales from the previous branch(es) to a beta, then run the RelEng update verify to shakes out any packaging fixes we need to make (depracated files etc). Meanwhile QA test the human interaction side. 

I'm happy to generate a major update in the existing style, and you can let me know what you want to do for the newer snippet functionality. We'll have to serve those off static snippets rather than AUS.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+1]]

As responsible QA person for the Add-ons Manager I would really like to see major updates happening for beta10 so we can deeply test the major update part. Otherwise we would run into an RC if blockers get identified.

Comment 9

[Nick Thomas [:nthomas] (UTC+12)]

To do spot-testing you can pretend to be 4.0b8 and get offered a major update to 4.0b10. Create a new pref called app.update.url.override and set it to (eg mac)
https://aus2.mozilla.org/update/3/%PRODUCT%/4.0b8/20101214165004/Darwin_x86-gcc3-u-i386-x86_64/%LOCALE%/beta/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/update.xml

Obviously different buildID and build target for other platforms. Channel is also changed from release to beta.

Comment 10

[Nick Thomas [:nthomas] (UTC+12)]

First go at a comparison of updated 3.6.13 vs 4.0b10 on mac finds these files that we don't remove
Contents/MacOS/plugins/JavaEmbeddingPlugin.bundle/Contents/Info.plist
Contents/MacOS/plugins/JavaEmbeddingPlugin.bundle/Contents/MacOS/JavaEmbeddingPlugin
Contents/MacOS/plugins/JavaEmbeddingPlugin.bundle/Contents/MacOS/JavaEmbeddingPlugin.policy
Contents/MacOS/plugins/JavaEmbeddingPlugin.bundle/Contents/PkgInfo
Contents/MacOS/plugins/JavaEmbeddingPlugin.bundle/Contents/Resources/English.lproj/InfoPlist.strings
Contents/MacOS/plugins/JavaEmbeddingPlugin.bundle/Contents/Resources/Java/JavaEmbeddingPlugin.jar
Contents/MacOS/plugins/MRJPlugin.plugin/Contents/Info.plist
Contents/MacOS/plugins/MRJPlugin.plugin/Contents/MacOS/MRJPlugin
Contents/MacOS/plugins/MRJPlugin.plugin/Contents/MacOS/MRJPlugin.jar
Contents/MacOS/plugins/MRJPlugin.plugin/Contents/MacOS/MRJPlugin.policy
Contents/MacOS/plugins/MRJPlugin.plugin/Contents/MacOS/MRJPlugin.properties
Contents/MacOS/plugins/MRJPlugin.plugin/Contents/PkgInfo
Contents/MacOS/plugins/MRJPlugin.plugin/Contents/Resources/English.lproj/InfoPlist.strings
Contents/MacOS/plugins/MRJPlugin.plugin/Contents/Resources/MRJPlugin.rsrc
Contents/MacOS/searchplugins/answers.xml
Contents/MacOS/searchplugins/creativecommons.xml
and some empty directories that the updater isn't allowed to delete. Those two searchplugins may still be there on purpose, based on earlier decisions to not remove anything existing users might be using.

Comment 11

[(not currently active) Ted Mielczarek]

Josh, Steven, should we be removing these files?

Comment 12

[Josh Aas]

Contents/MacOS/plugins/JavaEmbeddingPlugin.bundle
Contents/MacOS/plugins/MRJPlugin.plugin

Anything with these path components can be deleted - we don't need the JavaEmbeddingPlugin or MRJPlugin.

We should do with the search plugins whatever we are doing on other platforms.

Comment 13

[Nick Thomas [:nthomas] (UTC+12)]

I have generated a MU from 3.6.14 build1 and 4.0b10 on our staging systems, and run our normal update verify. The logs are at 
  http://people.mozilla.com/~nthomas/3.6.14build1_4.0b10_MU/

The main issues are the JavaEmbeddingPlugin.bundle & MRJPlugin.plugin mentioned above, locales not removing chrome/ab-CD.{jar,manifest}, and figuring out what to do with searchplugin removals in 4.0. Hoping to work up a packaging fix in time for tomorrow.

Comment 14

[Nick Thomas [:nthomas] (UTC+12)]

Linux differences:

For all locales (where ab-CD is substituted for af)
Only in source/firefox/chrome: af.jar
Only in source/firefox/chrome: af.manifest

Empty dirs (or contain only empty dirs):
Only in source/firefox/defaults: autoconfig
Only in source/firefox/defaults: profile
Only in source/firefox: greprefs
Only in source/firefox: modules
Only in source/firefox: plugins
Only in source/firefox: res

Expected differences in channel control file:
diff -r source/firefox/defaults/pref/channel-prefs.js target/firefox/defaults/pref/channel-prefs.js
1,2c1,2
< //@line 2 "/builds/slave/rel-192-lnx-bld/build/browser/app/profile/channel-prefs.js"
< pref("app.update.channel", "release");
---
> //@line 2 "/builds/slave/rel-cen-lnx-bld/build/browser/app/profile/channel-prefs.js"
> pref("app.update.channel", "beta");

Locale specific:
af, bn-IN, el, en-GB, en-US, is:
       Only in source/firefox/searchplugins: answers.xml
       Only in source/firefox/searchplugins: creativecommons.xml
ca:    Only in source/firefox/searchplugins: yahoo-ca.xml
cz:    Only in source/firefox/searchplugins: mall-cz.xml
es-CL, es-ES, it, kn, ko, sv-SE
       Only in source/firefox/searchplugins: creativecommons.xml
eu:    Only in source/firefox/searchplugins: answers.xml
ku:    Only in source/firefox/searchplugins: google.xml
lt,tr: Only in source/firefox/searchplugins: answers.xml
       Only in source/firefox/searchplugins: creativecommons.xml
       Only in source/firefox/searchplugins: yahoo.xml
nl:    Only in source/firefox/searchplugins: yahoo-nl.xml
ru:    Only in source/firefox/searchplugins: torgmailru.xml
sq:    Only in source/firefox/searchplugins: amazondotcom.xml
       Only in source/firefox/searchplugins: answers.xml
       Only in source/firefox/searchplugins: creativecommons.xml

Those plugin changes are probably all productization changes. Look OK Axel ? I'm assuming we'll leave those there like we have for previous major updates.

Only thing to action for Linux is the chrome/ab-CD.{manifest,jar}.

Comment 15

[Nick Thomas [:nthomas] (UTC+12)]

By way of explanation, 'source/' is 3.6.14 which has been updated, and 'target/' is an unpacked copy of 4.0b10. Anything Only in source/ is a 3.6.14 file which the update hasn't removed. Empty directories we have to leave there because we don't allow the updater to remove them.

Win32 is much the same as linux, except that omni.jar and chrome.manifest live in the root of the install dir on windows, so we get this
Contents of source/bin/chrome dir only in source or target
2043194  230 -rw-r--r--   1 cltbld   Administrators   470854 Jan  19:06 source/bin/chrome/af.jar
2043195    1 -rw-r--r--   1 cltbld   Administrators     1116 Jan  19:06 source/bin/chrome/af.manifest
instead of the 'Only in source/firefox/chrome'. Looks like similar locale differences for searchplugins (shouldn't be any platform dependence there).

Comment 16

[Nick Thomas [:nthomas] (UTC+12)]

Mac is much the same as linux & win32, the prefix is source/Firefox.app/Contents/MacOS instead. We need to clean up chrome/ab-CD.{jar,manifest} and the files in plugins/{JavaEmbeddingPlugin.bundle,MRJPlugin.plugin}.

Comment 17

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #15)
> Win32 is much the same as linux, except that omni.jar and chrome.manifest live
> in the root of the install dir on windows, 

Actually, it's because we have chrome/icons/ on linux so there's still something left in chrome/ after all the jars and manifests move into omni.jar. That's not true on mac and windows so we get different output from diff.

Comment 18

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Packaging fixes (from 3.6.14build1 --> 4.0b10 test)

rs, this adds all the locales we will ship for 3.6.14, plus the single locale we ship on 3.5 but not 3.6 (mn). I've verified it covers all locales we've shipped at any time on 3.0, 3.5 and 3.6. Also removes some directories on mac (DefaultPlugin.plugin) that just causes warnings when making the complete mar.

Comment 19

[Nick Thomas [:nthomas] (UTC+12)]

The 3.6.14build1 --> 4.0b10 major update test is published on production update servers, using the betatest and releasetest channels (at least until we do 3.6.15 or another 3.6.14).

Bug 630085 for setting up the release automation to generate and test automagically.

Comment 20

[Robert Strong (they/them - no direct email)]

Nick, just a heads up... I have a patch for empty directory removal via software update in bug 386760. It will probably land after Firefox 4. Will finish the review of this patch momentarily.

Comment 21

[Robert Strong (they/them - no direct email)]

Comment on attachment 508291 [details] [diff] [review]
Packaging fixes (from 3.6.14build1 --> 4.0b10 test)

Looks fine. Though this is for testing we shouldn't be updating the Mac locales affected by bug 628829 except for nightly builds after bug 628829 landed. Updates can be turned back on for them after Firefox 3.6.15 / 3.5.18 are released.

Comment 22

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 508291 [details] [diff] [review]
Packaging fixes (from 3.6.14build1 --> 4.0b10 test)

Thanks Rob.

http://hg.mozilla.org/mozilla-central/rev/74860484ed4d

I'll let QA know to ignore mac updates for bn-IN, kn, ml, and te. Unlike the point releases, we don't have any mechanism to not generate updates for those that automatically (the patcher config is created from scratch each time for major updates). We can clean up manually if the timing doesn't work and 3.6.15 comes after 4.0 final.

Comment 23

[Axel Hecht [:Pike]]

Re the l10n differences with search plugins, we're not solid on l10n search plugins, so there may be changes to removed-files.in that we'd still want.

Comment 24

[Nick Thomas [:nthomas] (UTC+12)]

Is that being tracked in a bug I can CC to ?

Comment 25

[mattburles]

waiting for 3.6.14 release before this blocker becomes fixed?

Comment 26

[Nick Thomas [:nthomas] (UTC+12)]

I've done a 3.6.14 build2 -> 4.0b11 build3 MU run, and the update verify results show
* empty directories left over (expected)
* channel-prefs.js change similar to comment #14 (expected)
* slightly different searchplugin leftovers, compared to comment #14 also have
 hi-IN: Only in source/firefox/searchplugins: amazondotcom.xml
 rm:    Only in source/firefox/searchplugins: creativecommons.xml
 and should have said 'ar' instead of 'af'. (assuming this is expected)

So the issues with the ab-CD.{jar,manifest} and the obsolete mac plugin files are resolved correctly. 

I've published the snippets for QA to test and any issues found can be filed separately. It's a known issue that there isn't a billboard isn't present, bug 633060 is filed to get that done. Bug 630085 will modify the release automation to be able to generate MU snippets automatically.

Comment 27

[Steven Michaud [:smichaud] (Retired)]

I just got prompted to update to FF 4 on a PPC Mac running FF 3.6.16.  But FF 4 won't work on PPC Macs, so this shouldn't have happened.

I don't know if there's already a bug on this.

Comment 28

[Steven Michaud [:smichaud] (Retired)]

> I don't know if there's already a bug on this.

I couldn't find one, so I've just opened bug 644438.

Comment 29

[[e-mail] (:bhearsum)]

(In reply to comment #27)
> I just got prompted to update to FF 4 on a PPC Mac running FF 3.6.16.  But FF 4
> won't work on PPC Macs, so this shouldn't have happened.
> 
> I don't know if there's already a bug on this.

Just to clarify, bug 644438 is about getting an update offer via the first run page, not via Check for Updates (which is what this bug is about).