Closed Bug 308575 Opened 19 years ago Closed 19 years ago

First nightly update fails for installer builds if it is a partial (binary patch) - xpicleanup is not stripped

Categories

(Toolkit :: Application Update, defect)

x86
Linux
defect
Not set
normal

Tracking

()

VERIFIED FIXED

People

(Reporter: nthomas, Assigned: chase)

References

Details

(Keywords: verified1.8)

Attachments

(2 files, 1 obsolete file)

User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b4) Gecko/20050910 Firefox/1.4 Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b4) Gecko/20050910 Firefox/1.4 Trying to bring a just-installed older nightly up to date fails to finish a partial update, instead requiring a complete update. Subsequent partial updates apply successfully. Reproducible: Always Steps to Reproduce: 1. Install the 2003091105 nightly from http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2005-09-13-06-mozilla1.8/firefox-1.4.en-US.linux-i686.installer.tar.gz to ~/firefox1.5 2. Modify firefox1.5/defaults/pref/channel-prefs.js so that the channel is nightly instead of beta 3. Start Firefox, go to Help > Check for Updates, let Firefox download the update and restart. Actual Results: A Software Update dialog is hidden the main browser window with the message "The partial update could not be applied". Following the prompts to download and apply the complete update works fine. Then partial updating from 0912 to 0913 works ok. Expected Results: The partial update should apply successfully. Also tried starting with 0912 and 0913, giving identical results. Distribution is Ubuntu 05.04.
This is the console output after setting the following prefs to true: app.update.log.{Checker,Downloader,General,UI:CheckingPage, UI:DownloadingPage,UI:LicensePage,UpdateManager} The log covers the initial failed partial, successful complete and one subsequently successful partial.
(In reply to comment #0) > Steps to Reproduce: > 1. Install the 2003091105 nightly from > http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2005-09-13-06-mozilla1.8/firefox-1.4.en-US.linux-i686.installer.tar.gz > to ~/firefox1.5 That's a link to the 0913 build. So in some cases you can partial update from 0912 to 0913 and in others you can't? Can you list the full build IDs that you are testing?
Arrgh, silly typo on the url. BuildID's and links to the directories I got installers from: 2005091105 http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2005-09-11-07-mozilla1.8/ 2005091205 http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2005-09-12-06-mozilla1.8/ 2005091305 http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2005-09-13-06-mozilla1.8/ On those three builds I didn't see a partial update work if it was the first update.
I investigated this bug some, and it looks like the MAR file is indeed bad. The MAR file contains a patch for xpicleanup that claims that the size of xpicleanup before patching should be 21368 bytes, but in fact it is 142252 bytes. I'm talking about this MAR file: firefox-1.4.en-US.linux-i686.partial.2005091305-2005091405.mar I started with the nightly build from: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2005-09-13-06-mozilla1.8/ I don't know if it explains the problem or not, but that MAR file says it maps a build from 2005091305, but the builds in that directory are from the 06 hour. That discrepancy probably doesn't explain this bug though. Marking confirmed. Reassigning to Chase.
Assignee: nobody → chase
Status: UNCONFIRMED → NEW
Ever confirmed: true
(In reply to comment #4) > I investigated this bug some, and it looks like the MAR file is indeed bad. The > MAR file contains a patch for xpicleanup that claims that the size of xpicleanup > before patching should be 21368 bytes, but in fact it is 142252 bytes. Okay, I'll take a look. > I'm talking about this MAR file: > firefox-1.4.en-US.linux-i686.partial.2005091305-2005091405.mar > > I started with the nightly build from: > http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2005-09-13-06-mozilla1.8/ Did you try to install 0912 and then partial update from 0912 to 0913, then from 0913 to 0914? Based on what you say, partial update from 0913 to 0914 should always fail. But Nick says that he's not able to partial update from 0913 to 0914 if it's his first update. Nick, can you partial update from 0913 to 0914 as long as you haven't just installed 0913? > I don't know if it explains the problem or not, but that MAR file says it maps a > build from 2005091305, but the builds in that directory are from the 06 hour. > That discrepancy probably doesn't explain this bug though. That's not unusual. The build ID and directory hour don't come from the same source.
> Did you try to install 0912 and then partial update from 0912 to 0913, then > from 0913 to 0914? Based on what you say, partial update from 0913 to 0914 > should always fail. No, I started with the 0913 build, and then attempted to partial update to the 0914 build. I suspect that anyone coming from the 0913 build to the 0914 build would experience the same problem. > That's not unusual. The build ID and directory hour don't come from the same > source. OK. I sort of figured that ;-)
(In reply to comment #6) > No, I started with the 0913 build, and then attempted to partial update to the > 0914 build. I suspect that anyone coming from the 0913 build to the 0914 build > would experience the same problem. Darin: can you file a new bug on the bogus partial update file?
(In reply to comment #5) > ... Nick, can you partial update from 0913 to 0914 as long as you > haven't just installed 0913? Starting from 0912, the partial fails but a complete gives 0913. Partial to 0914 works like a charm.
It seems that the xpicleanup in the installer is not stripped (139KB) while the one served in the complete update is (21KB). I verified that the complete update replaces xpicleanup with the stripped version, which allows subsequent partials to work. The 0912 archive (rather than installer) has the stripped xpicleanup, and copying this file into the installer dir allows partials to work first time. Spot checking of installer builds (0831, 0905, 0909, 0911, 0912) reveals unstripped xpicleanup for some time.
Summary: First nightly update fails if it is a partial (binary patch) → First nightly update fails for installer builds if it is a partial (binary patch) - xpicleanup is not stripped
Wow. This indicates something very odd with our talkback symbol stripping process.
(In reply to comment #7) > Darin: can you file a new bug on the bogus partial update file? Based on recent data please ignore my request to file that bug, Darin. Here's what I think is happening: The MAR is built from the compressed build and the compressed and installer builds on Linux differ enough that the partial update only successfully applies to compressed builds. Nick tries to update, the partial update fails, it falls back to the complete update which successfully applies. When Nick restarts, he's effectively running the compressed version which is what one needs on Linux for the partial updates to apply successfully. They work from then on.
Another reason to stop shipping the linux installer.
Update on Linux Fx branch builds exhibits this bug. But even the full install doesn't apply correctly. The update mechanism goes through all the motions of a valid update (download updates, restart Fx, apply updates). But in the end, the build ID remains the same as before update was attempted. note: I use installer builds. This seems blocker serious to me. Is that this bug or should I file another?
Testing Fx branch update on Windows 1003 to 1004 fails with this bug. Finds partial update, downloads it, restarts Fx, fails to apply partial update, downloads full update, restarts Fx, appears to apply Full update with progress dialog going to compkletion, Fx starts then check buid ID in About: it's still from 1003.
Flags: blocking1.8b5?
This just happened to me today when upgrading from Windows Firefox 10/03 to Firefox 10/04. Same symptoms Tracy saw, except for me the full update did apply correctly.
(In reply to comment #15) > This just happened to me today when upgrading from Windows Firefox 10/03 to > Firefox 10/04. Same symptoms Tracy saw, except for me the full update did apply > correctly. Ditto.
Oddly enough, the Thunderbird incremental update to 10/04 worked fine.
(In reply to comment #14) > Testing Fx branch update on Windows 1003 to 1004 fails with this bug. Finds > partial update, downloads it, restarts Fx, fails to apply partial update, > downloads full update, restarts Fx, appears to apply Full update with progress > dialog going to compkletion, Fx starts then check buid ID in About: it's still > from 1003. Fwiw, I see the same, but I've never used an installer build, only zip builds and possibly a few automatic full-updates.
(In reply to comment #14) I see the same behavior. Partial update fails, full update appears to happen but when checking the UA string, I am still on a 10/03 build. WinXP/SP2
Updating my 20051003 build, I saw this: 1. Initial update attempt found 490k partial update and tried to apply, but failed on restart. 2. Second update attempt switched to downloading 6mb full update, which succeeded as I am now using today's build: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b5) Gecko/20051004 Firefox/1.4.1 So something is still broken with partial update, and a few people seem to be having problems with the full update as well.
Bob and Jay, we need to be able to reproduce both cases. Can you two get together to work out what's different about your installations and procedures for how to manifest both of these results? This got worse, I believe, after I ran 'cvs update -dP tools/update-packaging/' on the partial patch packaging system yesterday per bug 310715. That command: 1. Added common.sh. 2. Moved make_incremental_update.sh from 1.4 to 1.6. Changes to make_full_update.sh may have occurred but since that script is not used on this system, I didn't pay attention to it.
I think the most recent reports are separate issue, since the outcome of the first complete update is different to the original diagnosis. Also, the original bug was not seen on win32 - I had checked (but not reported) earlier that xpicleanup.exe was the same size in installer and updated builds and that his bug did not occur. Perhaps bug 311099 would be better than morphing this bug.
The original reported bug is happening on Windows with 100% reproducibility. But I agree this should not morph into the second problem I reported. Bug 311099 is what I am seeing, if you filter out the reporters connection problems. I'll comment there and cleanup summary.
we want to make sure this works for beta1->beta2.
Flags: blocking1.8b5? → blocking1.8b5+
Keywords: qawanted
This patch makes the regexp for xpicleanup less restrictive, bringing it into line with those for .so and -bin. It produces a Firefox installer with stripped xpicleanup in my build, but I don't know the consequences for other toolkit apps.
Attachment #198692 - Flags: superreview?(bsmedberg)
Attachment #198692 - Flags: review?(darin)
For clarity, attachment 198692 [details] [diff] [review] is for the original report rather than the broken update comments of 2005-10-04. Which of these does the blocking1.5b5+ flag apply to? The patch should mean that fresh installs of 1.5b2 can update on the beta channel using only a partial update. Current 1.5b1 installs may need a custom partial update, made against an unstripped xpicleanup instead of the stripped version used for nightly builds. Alternatively, 1.5b1 users could apply the complete update when the partial fails (a release note would be helpful!).
Asa, beta1 -> beta2 incremental is just not going to work if people installed a linux installer. And I think we should just stop shipping the linux installers altogether, as we've discussed.
Flags: blocking1.8b5+ → blocking1.8rc1+
Comment on attachment 198692 [details] [diff] [review] Don't be quite so picky about finding xpicleanup What about other executables like 'updater' and 'mozilla-xremote-client' It seems like this patch is not sufficient.
Attachment #198692 - Flags: review?(darin) → review-
I've checked that all the binaries installed by the installer are stripped when using this patch, and that this matches the content of the plain archive. Using "diff -r" to compare the unpacked xpis from the installer with the archive's content gives: libsoftokn3.chk different; .autoreg and removed-files are only present in the archive. This patch is really a short term fix. A better solution would be to not maintain separate strip lists for the archive (packager.mk,line 263 on) and the installer (makexpi.pl as patched). Currently, the prometheus tinderbox (and presumably others) builds the installer and then the archive. By reversing this order, the already stripped dist/{firefox,thunderbird} can be used to build the installer.
Attachment #198692 - Attachment is obsolete: true
Attachment #198918 - Flags: superreview?(benjamin)
Attachment #198918 - Flags: review?(darin)
Whiteboard: [needs SR bsmedberg, review darin]
Attachment #198918 - Flags: review?(darin) → review+
Attachment #198692 - Flags: superreview?(bsmedberg)
Depends on: 311969
Attachment #198918 - Flags: superreview?(benjamin) → superreview+
Comment on attachment 198918 [details] [diff] [review] Also include mozilla-xremote-client and updater in strip list [checked in] Please land on the trunk. Requesting approval for landing on the branch after the fix has been verified there.
Attachment #198918 - Flags: approval1.8rc1?
Whiteboard: [needs SR bsmedberg, review darin] → needs 1 day bake on trunk to verify fix, land on branch after verified fixed there
fixed-on-trunk
Status: NEW → RESOLVED
Closed: 19 years ago
Resolution: --- → FIXED
who can verify this on the trunk?
(In reply to comment #32) > who can verify this on the trunk? Asa, coop or I will respin prometheus's Firefox branch build to generate another Linux nightly. You can verify this by downloading today's earlier morning build and updating to the later build once it and its partial patch are available.
(In reply to comment #33) > Asa, coop or I will respin prometheus's Firefox branch build to generate another s/branch/trunk
(In reply to comment #32) > who can verify this on the trunk? The respin has completed and the partial patch should be available. Download http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2005-10-20-05-trunk/firefox-1.6a1.en-US.linux-i686.installer.tar.gz, install it, then check for an update. The partial patch should be downloaded and installed without failure.
Verified fixed per comment #35
Comment on attachment 198918 [details] [diff] [review] Also include mozilla-xremote-client and updater in strip list [checked in] thanks for verifying the fix Nick.
Attachment #198918 - Flags: approval1.8rc1? → approval1.8rc1+
(In reply to comment #35) > The respin has completed and the partial patch should be available. > > Download > http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2005-10-20-05-trunk/firefox-1.6a1.en-US.linux-i686.installer.tar.gz, > install it, then check for an update. The partial patch should be downloaded > and installed without failure. FYI, these were instructions on how to verify this bug is fixed. Nick, in comment 36 were you saying you were able to run these steps with success?
v.fixed on Trunk. Partial update worked going from this morning's build to the new build (Talkback ID: 2005102011).
Status: RESOLVED → VERIFIED
For comment #36 I used the steps and build in comment #35. The partial update applied successfully. I also confirmed that the previous trunk nightly - 2005101904 - did not apply the partial.
Whiteboard: needs 1 day bake on trunk to verify fix, land on branch after verified fixed there → fix is attachment 198918, needs landing on branch by chase
Comment on attachment 198918 [details] [diff] [review] Also include mozilla-xremote-client and updater in strip list [checked in] Landed on the 1.8 branch. Checking in toolkit/mozapps/installer/makexpi.pl; /cvsroot/mozilla/toolkit/mozapps/installer/makexpi.pl,v <-- makexpi.pl new revision: 1.4.10.1; previous revision: 1.4 done
Attachment #198918 - Attachment description: Also include mozilla-xremote-client and updater in strip list → Also include mozilla-xremote-client and updater in strip list [checked in]
Fixed on the 1.8 branch.
Keywords: fixed1.8
Whiteboard: fix is attachment 198918, needs landing on branch by chase
Product: Firefox → Toolkit
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: