Last Comment Bug 598647 - Implement MSI installer
: Implement MSI installer
Status: NEW
:
Product: Firefox
Classification: Client Software
Component: Installer (show other bugs)
: unspecified
: x86 Windows 7
: -- enhancement with 21 votes (vote)
: ---
Assigned To: Nobody; OK to take it and work on it
:
Mentors:
Depends on:
Blocks: MSI
  Show dependency treegraph
 
Reported: 2010-09-22 08:09 PDT by Kyle Huey [:khuey] (khuey@mozilla.com)
Modified: 2015-06-11 04:59 PDT (History)
27 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
Configure bits for MSI installer (3.34 KB, patch)
2010-10-29 16:14 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
mitchell.field: review+
Details | Diff | Splinter Review
Part 2: Provide branding info to WiX (12.40 KB, patch)
2010-10-29 16:15 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
khuey: review+
Details | Diff | Splinter Review
Part 3: Generate Product Code at build time (549 bytes, patch)
2010-10-29 16:16 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
khuey: review+
Details | Diff | Splinter Review
Part 4: Reflect build system variables into WiX (985 bytes, patch)
2010-10-29 16:16 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
khuey: review+
Details | Diff | Splinter Review
Part 5: Implement a basic en-US Firefox MSI (12.60 KB, patch)
2010-10-29 16:17 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
robert.strong.bugs: review+
Details | Diff | Splinter Review
Part 6: Write the product code to disk for later use (1.67 KB, patch)
2010-10-29 16:18 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
khuey: review+
Details | Diff | Splinter Review
Part 7: Look pretty in Add/Remove Programs (5.78 KB, patch)
2010-10-29 16:19 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
khuey: review+
Details | Diff | Splinter Review
Part 8: Tell Windows we know what we're doing with UAC. (864 bytes, patch)
2010-10-29 16:22 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
khuey: review+
Details | Diff | Splinter Review
Part 9: Export brand.dtd to dist/branding (4.24 KB, patch)
2010-10-29 16:58 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
khuey: review+
Details | Diff | Splinter Review
Part 10: Use BrandFullNameInternal to get the right name in official builds (2.25 KB, patch)
2010-10-29 16:58 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
robert.strong.bugs: review+
Details | Diff | Splinter Review
Part 11: Calculate the firefox.exe Windows version from the app version, not the Gecko version (2.00 KB, patch)
2010-10-29 16:59 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
mitchell.field: review+
Details | Diff | Splinter Review
Part 12: Add localized shorcuts (9.91 KB, patch)
2010-10-30 02:59 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
khuey: review+
Details | Diff | Splinter Review
Part 13: Add a custom action DLL to replace AppAssocReg (7.30 KB, patch)
2010-10-30 03:00 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
khuey: review+
Details | Diff | Splinter Review
Part 14: Integrate custom action DLL into the installer (3.70 KB, patch)
2010-10-30 03:00 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
khuey: review+
Details | Diff | Splinter Review
Part 15: Registry/shell integration (9.91 KB, patch)
2010-10-30 03:01 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
robert.strong.bugs: review-
Details | Diff | Splinter Review
Part 16: Upload completed MSI (3.35 KB, patch)
2010-10-30 03:02 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
ted: review+
Details | Diff | Splinter Review
Part 17: Move all existing strings into the DTD for l10n (4.19 KB, patch)
2010-10-30 06:46 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
l10n: review+
robert.strong.bugs: feedback+
Details | Diff | Splinter Review
Part 18: Use a shortened product version to get the ProductCodes just right. (1.01 KB, patch)
2010-10-30 07:18 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
khuey: review+
Details | Diff | Splinter Review
Part 19: Teach the updater about MSI installations (4.61 KB, patch)
2010-10-30 07:18 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
robert.strong.bugs: review-
Details | Diff | Splinter Review
Part 20: Add patch tools to configure (2.40 KB, patch)
2010-10-30 09:04 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
mitchell.field: review+
Details | Diff | Splinter Review
Part 12: Add localized shortcut (6.84 KB, patch)
2011-03-24 15:43 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
khuey: review+
Details | Diff | Splinter Review
Part 21: Omit CRT directives from the custom action DLL. (1.10 KB, patch)
2011-03-24 15:46 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
robert.strong.bugs: review+
Details | Diff | Splinter Review
Part 22: Bind files into the WiX output file (1.29 KB, patch)
2011-03-24 16:13 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
robert.strong.bugs: review+
nthomas: feedback+
Details | Diff | Splinter Review
Part 23: Upload the wixpdb (742 bytes, patch)
2011-03-24 16:17 PDT, Kyle Huey [:khuey] (khuey@mozilla.com)
ted: review+
Details | Diff | Splinter Review

Description Kyle Huey [:khuey] (khuey@mozilla.com) 2010-09-22 08:09:36 PDT
When I have a go at this again soon I'm going to use this bug rather than the ridiculously long/noisy Bug 231062.
Comment 1 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-29 16:14:15 PDT
Created attachment 487077 [details] [diff] [review]
Configure bits for MSI installer
Comment 2 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-29 16:15:31 PDT
Created attachment 487079 [details] [diff] [review]
Part 2: Provide branding info to WiX

This patch was r+'d by rs in Bug 231062.
Comment 3 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-29 16:16:17 PDT
Created attachment 487080 [details] [diff] [review]
Part 3: Generate Product Code at build time

This was also r+'d by rs in Bug 231062.
Comment 4 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-29 16:16:54 PDT
Created attachment 487081 [details] [diff] [review]
Part 4: Reflect build system variables into WiX

This patch was r+'d by jimm in Bug 231062.
Comment 5 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-29 16:17:45 PDT
Created attachment 487082 [details] [diff] [review]
Part 5: Implement a basic en-US Firefox MSI

Parts of this have been previously reviewed, but it's changed enough that I think we should start from scratch on this one.
Comment 6 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-29 16:18:46 PDT
Created attachment 487083 [details] [diff] [review]
Part 6: Write the product code to disk for later use

This was r+'d by jimm in Bug 231062.
Comment 7 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-29 16:19:15 PDT
Created attachment 487084 [details] [diff] [review]
Part 7: Look pretty in Add/Remove Programs

Ditto
Comment 8 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-29 16:22:22 PDT
Created attachment 487087 [details] [diff] [review]
Part 8: Tell Windows we know what we're doing with UAC.
Comment 9 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-29 16:58:25 PDT
Created attachment 487096 [details] [diff] [review]
Part 9: Export brand.dtd to dist/branding

This was reviewed by ted and Pike in Bug 231062.
Comment 10 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-29 16:58:56 PDT
Created attachment 487097 [details] [diff] [review]
Part 10: Use BrandFullNameInternal to get the right name in official builds
Comment 11 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-29 16:59:46 PDT
Created attachment 487098 [details] [diff] [review]
Part 11: Calculate the firefox.exe Windows version from the app version, not the Gecko version
Comment 12 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-30 02:59:52 PDT
Created attachment 487149 [details] [diff] [review]
Part 12: Add localized shorcuts

This patch has r=Mitch,ted,Pike in Bug 231062.
Comment 13 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-30 03:00:30 PDT
Created attachment 487150 [details] [diff] [review]
Part 13: Add a custom action DLL to replace AppAssocReg

This patch has r=jimm in Bug 231062.
Comment 14 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-30 03:00:56 PDT
Created attachment 487151 [details] [diff] [review]
Part 14: Integrate custom action DLL into the installer

Same as Part 13
Comment 15 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-30 03:01:31 PDT
Created attachment 487152 [details] [diff] [review]
Part 15: Registry/shell integration
Comment 16 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-30 03:02:27 PDT
Created attachment 487153 [details] [diff] [review]
Part 16: Upload completed MSI
Comment 17 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-30 06:46:18 PDT
Created attachment 487161 [details] [diff] [review]
Part 17: Move all existing strings into the DTD for l10n
Comment 18 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-30 07:18:05 PDT
Created attachment 487164 [details] [diff] [review]
Part 18: Use a shortened product version to get the ProductCodes just right.

This has r=Mitch in Bug 231062.
Comment 19 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-30 07:18:53 PDT
Created attachment 487165 [details] [diff] [review]
Part 19: Teach the updater about MSI installations

Adds an 'msiexec' command to the updater that lets us pass an MSI file out to Windows Installer.
Comment 20 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-30 09:04:47 PDT
Created attachment 487174 [details] [diff] [review]
Part 20: Add patch tools to configure
Comment 21 Robert Strong [:rstrong] (use needinfo to contact me) 2010-10-30 13:45:07 PDT
I have a few concerns with making the updater binary part of the msi process. A few of the concerns off the top of my head are.

1. the updater requires that Firefox is not running and MSI's don't have that requirement.
2. It adds complexity to the updater which is unnecessary.
3. releng packaging will require creating the msi then creating a mar containing the msi. Though minor it is an additional step that should be unnecessary.
4. it locks us in to only applying the msi when the app is restarted which should not be necessary with an msi.

We already use the update service to distribute apk files for Android and I would think it would be possible to do the same for msi / msp files.
Comment 22 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-10-30 14:05:02 PDT
OK, I'll take a look at what we do for Android.
Comment 23 Ted Mielczarek [:ted.mielczarek] 2010-11-04 07:30:44 PDT
Comment on attachment 487153 [details] [diff] [review]
Part 16: Upload completed MSI

>diff --git a/toolkit/mozapps/installer/packager.mk b/toolkit/mozapps/installer/packager.mk
>--- a/toolkit/mozapps/installer/packager.mk
>+++ b/toolkit/mozapps/installer/packager.mk
>@@ -669,16 +669,17 @@ QUOTED_WILDCARD = $(if $(wildcard $(subs
> CHECKSUM_ALGORITHM = 'sha512'
> 
> # This variable defines where the checksum file will be located
> CHECKSUM_FILE = "$(DIST)/$(PKG_PATH)/$(PKG_BASENAME).checksums"
> 
> UPLOAD_FILES= \
>   $(call QUOTED_WILDCARD,$(DIST)/$(PACKAGE)) \
>   $(call QUOTED_WILDCARD,$(INSTALLER_PACKAGE)) \
>+  $(call QUOTED_WILDCARD,$(DIST)/$(PKG_BASENAME).msi) \

If you use this in more than one place you might want to define MSI_PACKAGE or something like that.
Comment 24 Robert Strong [:rstrong] (use needinfo to contact me) 2010-11-04 17:02:16 PDT
Comment on attachment 487152 [details] [diff] [review]
Part 15: Registry/shell integration

Note: we will likely be removing the program directory from the start menu and only having a single shortcut in the root of the start menu. It will probably be easier if the msi did this from day one so cleanup doesn't have to happen when the change is made. There are a couple of bugs that need to be fixed for this to happen though.
Comment 25 Robert Strong [:rstrong] (use needinfo to contact me) 2010-11-04 17:04:18 PDT
Comment on attachment 487152 [details] [diff] [review]
Part 15: Registry/shell integration

btw: you named this patch "Part 15: Registry/shell integration" when all it really does is add the shortcuts. Is there already a patch for "Registry/shell integration"?
Comment 26 Kyle Huey [:khuey] (khuey@mozilla.com) 2010-11-04 17:07:21 PDT
(In reply to comment #25)
> Comment on attachment 487152 [details] [diff] [review]
> Part 15: Registry/shell integration
> 
> btw: you named this patch "Part 15: Registry/shell integration" when all it
> really does is add the shortcuts. Is there already a patch for "Registry/shell
> integration"?

Bah, I uploaded Part 12 as Part 12 and Part 15.
Comment 27 Robert Strong [:rstrong] (use needinfo to contact me) 2010-11-04 18:53:47 PDT
Comment on attachment 487152 [details] [diff] [review]
Part 15: Registry/shell integration

Minusing patch 15 due to comment #26 and comment #24 applies to patch 12 attachment #487149 [details] [diff] [review]
Comment 28 Robert Strong [:rstrong] (use needinfo to contact me) 2010-11-05 20:20:31 PDT
Regarding the Registry/shell integration we are planning on removing dde support in bug 491947. The one hassle is Win2K. I think we shouldn't support MSI's with Win2K and just disable it for MSI's.
Comment 29 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-03-24 15:43:30 PDT
Created attachment 521666 [details] [diff] [review]
Part 12: Add localized shortcut

Updated to the Firefox 4 behavior of one toplevel shortcut.
Comment 30 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-03-24 15:46:27 PDT
Created attachment 521668 [details] [diff] [review]
Part 21: Omit CRT directives from the custom action DLL.

The custom action DLL cannot rely on an appropriate version of the CRT already being installed.
Comment 31 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-03-24 16:13:55 PDT
Created attachment 521675 [details] [diff] [review]
Part 22: Bind files into the WiX output file

Asking for nthomas for feedback on the approach here, not the actual code.

There are two ways to do "partial" MSI updates in WiX.  You can generate the updates from a pair of the .msi packages themselves, or from the .wixpdb files that the toolset outputs.  If we use the .msi files, we have to do administrative installations to unpack them to generate the partials.  If we use the .wixpdbs we can avoid that, at the cost of having to have another type of file around.

It's basically a tradeoff between code complexity if we use .msis and some disk space on ftp if we use the .wixpdbs.  I'd like to use the wixpdbs, but I'd like nthomas's feedback on this.

If we use the wixpdbs, we need to bind the files into them (see http://windows-installer-xml-wix-toolset.687559.n2.nabble.com/error-PYRO0103-The-system-cannot-find-the-file-UI-Icons-appicon-ico-if-appicon-ico-is-in-my-latest-wb-td4600799.html for the type of error we hit otherwise)
Comment 32 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-03-24 16:17:41 PDT
Created attachment 521678 [details] [diff] [review]
Part 23: Upload the wixpdb

If we go with the wixpdb method, we'll need to upload the wixpdbs.
Comment 33 Ted Mielczarek [:ted.mielczarek] 2011-03-25 06:05:00 PDT
Comment on attachment 521678 [details] [diff] [review]
Part 23: Upload the wixpdb

Fine with me.
Comment 34 Nick Thomas [:nthomas] 2011-03-28 20:18:19 PDT
Comment on attachment 521675 [details] [diff] [review]
Part 22: Bind files into the WiX output file

So we would need to retain the .wixpdb's between releases but not actually ship them to end users ? If so, we can keep them in firefox/nightly/$version-candidates/ on the ftp server, and avoid the mirrors having to carry that weight in firefox/releases/. That would be fine so long as the .wixpdb's are reasonably sized, so how large are they, and the msi's ?

Something we'll need to consider is signing the windows binaries for releases. The current process is 
 * build en-US, package
 * repack locales, package
 * sign files inside the installers and complete mars, and the installers themselves
 * generate partial mars
So we'd need to adjust the signing scripts to unpack the files in a msi, sign them or replace with signed files, and put them back together. Possibly sign the msi, and generate the partial msi updates afterwards. Bug 509158 may change that process, making signed bits available on the build machine - probably life much simpler for you - but there's isn't a timeline there yet.
Comment 35 Nick Thomas [:nthomas] 2011-03-28 20:19:23 PDT
Comment on attachment 521675 [details] [diff] [review]
Part 22: Bind files into the WiX output file

Oops, setting feedback? again until khuey answers.
Comment 36 Kyle Huey [:khuey] (khuey@mozilla.com) 2011-03-29 03:22:46 PDT
The MSI installer itself is ~25MB uncompressed (that'll drop significantly once we turn on compression) and the wixpdb is about 12-13 MB.  Note that we only have to keep the wixpdbs around for any binary we want to create a partial update to/from, we can always create full updates (the MSI itself is the full update)

(In reply to comment #34)
> Something we'll need to consider is signing the windows binaries for releases.
> The current process is 
>  * build en-US, package
>  * repack locales, package
>  * sign files inside the installers and complete mars, and the installers
> themselves
>  * generate partial mars
> So we'd need to adjust the signing scripts to unpack the files in a msi, sign
> them or replace with signed files, and put them back together. Possibly sign
> the msi, and generate the partial msi updates afterwards. Bug 509158 may change
> that process, making signed bits available on the build machine - probably life
> much simpler for you - but there's isn't a timeline there yet.

Yeah catlee and I have been talking about this.  He's rewriting a good chunk of our packaging code as part of that effort.
Comment 37 Robert Strong [:rstrong] (use needinfo to contact me) 2011-04-14 11:22:31 PDT
Comment on attachment 521675 [details] [diff] [review]
Part 22: Bind files into the WiX output file

I think wixpdb would be the way to go
Comment 38 Nick Thomas [:nthomas] 2011-04-28 17:47:43 PDT
Comment on attachment 521675 [details] [diff] [review]
Part 22: Bind files into the WiX output file

Another GB per set of release builds isn't going to break the bank.

Note You need to log in before you can comment on or make changes to this bug.