Closed Bug 231062 (MSI) Opened 21 years ago Closed 7 years ago

Provide Firefox MSI package

Categories

(Firefox :: Installer, enhancement)

x86
Windows 2000
enhancement
Not set
normal

Tracking

()

RESOLVED WONTFIX
Future
Tracking Status
blocking2.0 --- -

People

(Reporter: Bugzilla-alanjstrBugs, Unassigned)

References

(Depends on 2 open bugs, Blocks 2 open bugs, )

Details

Attachments

(22 files, 35 obsolete files)

7.60 KB, patch
jimm
: review+
Details | Diff | Splinter Review
10.07 KB, patch
robert.strong.bugs
: review+
Details | Diff | Splinter Review
8.71 KB, patch
jimm
: review+
Details | Diff | Splinter Review
4.43 KB, patch
robert.strong.bugs
: review+
Details | Diff | Splinter Review
413 bytes, patch
robert.strong.bugs
: review+
Details | Diff | Splinter Review
871 bytes, patch
jimm
: review+
Details | Diff | Splinter Review
9.31 KB, patch
jimm
: review+
Details | Diff | Splinter Review
1.75 KB, patch
jimm
: review+
Details | Diff | Splinter Review
4.25 KB, patch
jimm
: review+
Details | Diff | Splinter Review
746 bytes, patch
jimm
: review+
Details | Diff | Splinter Review
4.48 KB, patch
ted
: review+
Pike
: feedback+
Details | Diff | Splinter Review
9.96 KB, patch
Mitch
: review+
Details | Diff | Splinter Review
4.80 KB, patch
khuey
: review+
Details | Diff | Splinter Review
6.27 KB, patch
Mitch
: review+
Details | Diff | Splinter Review
6.68 KB, patch
Mitch
: review+
Details | Diff | Splinter Review
10.64 KB, patch
Mitch
: review+
Details | Diff | Splinter Review
1.17 KB, patch
jimm
: review+
Details | Diff | Splinter Review
7.20 KB, patch
jimm
: review+
Details | Diff | Splinter Review
6.40 KB, patch
jimm
: review+
Details | Diff | Splinter Review
14.79 KB, patch
Details | Diff | Splinter Review
4.54 KB, patch
Details | Diff | Splinter Review
7.63 KB, application/xml
Details
Bug 52052 was for Seamonkey and WONTFIX'd.  There are lots of good reasons to
use .msi on Win32, including corporate deployment.

This would also ease an upgrade path, uninstallation, selecting shortcut
locations, and maybe even file associations.
An alternative is to use NSIS (see http://seb.mozdev.org/firebird/ ).  Either
would allow a net-installer.
We should ask apache.org how they do it
Discussion starts at 
23:00 < alanjstr> anyone know anything about making .msi files?
I think we need to implement this on Windows, because with newer MS compilers
the C/C++ runtimes are no longer system files and have to be distributed with
the application. This means that in the long term, when we move away from VC6
we're going to run into a chicken and egg problem, as the current installer
won't run without the C/C++ runtimes.

IMO, this is something that we'll probably *have* to do in the long term.
The fact that installing software across Active Directory (Microsofts domain
technology) requires MSI's, makes me think that this is quite an important thing
to have done. It makes pushing out software to users (100s or 1000s) extremely easy.
Summary: Firebird installer for Windows should use the MSI (Microsoft Installer) → Firefox installer for Windows should use the MSI (Microsoft Installer)
The wontfix on bug 52052 in part explains why NS7.1 never got installed in our
building: computing support people don't necessarily have any interest in
browsing functionality beyond "good enough" for normal users.  Combined with
disinterest among administrators and managers, and you get software stasis. 
Making software distribution easy and automatable for support folk is nearly
essential, and in a Windows environment, that means MSI.
Blocks: 241532
http://sourceforge.net/projects/wix/

Perhaps WiX would be a good way to do this?  I'm not sure how much its status as
CPL matters, if any (it looks like the CPL only applies to distribution of code,
not product use).  It would seem to be a good way to go, as it's been used
before by Microsoft and has command line tools that would fit into the command
line based build process the Mozilla tree uses.  (I haven't worked with it,
tested it, or gone much beyond reading the blurb about it on the WiX site, so I
might be wrong about its utility for this bug.)
An MSI package would be very helpful for administrators, as it simplifies the
software installation process for thousands of machines to a couple minutes
(stick the MSI on a network share and assign it via an ActiveDirectory policy to
computers, all with a couple mouse clicks).

Although not very powerful, Visual Studio.NET's built-in Windows Installer
package support appears sufficient for Firefox. I created my own Firefox MSI
without any major MSI-related problems.
*** Bug 246625 has been marked as a duplicate of this bug. ***
Requesting blocking 1.0 as there have been several threads on this recently on
Mozillazine and it does seem like this would help a lot with corporate rollouts.
Comment #10 suggests that this might have an easyish resolution.
Flags: blocking-aviary1.0?
not going to block the release for this, but its something I really want to see
as an option (how is another issue entirely)
Flags: blocking-aviary1.0? → blocking-aviary1.0-
I would suggest that the best way to do this, is produce a STiX XML file in the
installer scripts. This really wouldn't be that bad, but would need reasonable
knowledge of STiX. I have had a quick look at it, and might try and do it at
some stage in the near future.
To clarify: would the existing installer be replaced by an MSI, or would the MSI
be in addition to the existing installer? (If the latter, the summary is a
little misleading.)
Since Win9x, and probably NT4, don't support .msi natively, I'm not sure if we'd
want to take the replacement approach.  However, those are in the minority, and
we do also provide zip builds.  We could probably get some input from apache.org
on how to do this.
replacement isn't necessary, but this is something that corporate types need/want

based on http://support.microsoft.com/default.aspx?scid=kb;EN-US;292539, the
Windows Installer Engine isn't part of the base install of 95/98/NT, but is for
ME/2k/XP, however it can be downloaded (and a number of people likely already
have it).  For the target audience utilizing MSI, we can safely assume they
already have it.
Summary: Firefox installer for Windows should use the MSI (Microsoft Installer) → Provide Firefox MSI package
it's easy enough to get or make available msi for 9x/nt4. nt3.51 is a different 
story. these days a bunch commercial vendors include the msi engine with their 
product (especially because some os's have the original msi engine and people 
tend to want the first point release).
Can anyone think of a reason to not use .msi?  NT 3.51?  Zip!
Anyone who has installed a recent version of MS Office (I think Office 2000 and
newer) will already have MSI support on their computer. Even those on Win9x. The
same is probably true for a lot of other MS Apps as well, but I don't have a
list of names. Point is that we can be pretty certain that almost everybody will
have MSI support already. Though of course, until we drop support for Win9x/NT
(which is probably a few years away), we will still need to provide the installer.

In any case, before we start, we need to decide what version of MSI to use.
Windows 2000 comes with 1.1, ME with 1.2, and XP with 2.0. Therefore I think we
have to use MSI version 1.1, so that we can support all Windows 2000
installations without having to upgrade to 2.0. I'm not sure what their
respective featuresets are, but it would certainly complicate things further if
we wanted to use something in the 2.0 featureset. Either we use 1.1 or 2.0.
Using 1.2 doesn't make any sense.
I've seen a model where two (downloadable) installers are offered: one assumes a
correct version of MSI and one that includes the MSI engine as part of the
package.  I imagine software distributed on CD just includes at least the needed
version of MSI and does what needs to be done to get it done.
W2K SP4 (or maybe it was SP3) provides MSI 2.0 on that platform.
DownloadDownload Windows Installer 2.0 for Windows 2000 and Windows NT 4.0 now

DownloadDownload Windows Installer 2.0 for Windows ME, Windows 98 Second
Edition, Windows 98, and Windows 95 now

http://support.microsoft.com/default.aspx?scid=kb;EN-US;292539
VS.NET can generate installer executables that check and install/upgrade Windows
Installer as necessary. The raw MSI package can be offered for people who know
they have the correct version installed to cut down on the download size.
The problem with including the Windows Installer package within the installer is
that the installer size would jump up from 4.7MB to somewhere in the region of
6-7MB. This is unacceptable as a long-term solution.
I certainly think that the existing installer works perfectly, the MSI version
should be downloadable for network administrators. STiX takes an XML file, and
the stuff you want to package and produces a MSI file. This would fit into how
the installer is currently built well. If we bootstrap it on there the
advantages are pretty easy to see.
I have also built an msi file for Firefox.  We use it at my college.  The 0.9.3
version is available at:

ftp://anon:anon@www.lakeland.cc.il.us/wlilly/Mozilla%20Firefox%200.9.3.msi

I would like to help anyone that developing this and am open to any suggestions
concerning my msi file.  I believe that this is one of the biggest things
keeping corporate people from using Firefox.  Ease of deployment thru Active
Directory is huge when you consider installing to thousands of machines.  Also
file associations and default browser settings can be done during the install
fairly easily if the correct registry settings are known.
Do you have a shell script that builds the .msi, or did you manually step
through some gui.
Unfortunately I had to step thru some GUI.  I have no idea how to shell script
an msi as the only shell scripting I do is for net administration.  The only
thing that I can provide that comes even close to what you are asking is that I
have a Firefox template that I use when I build the MSI so I don't have to
recreate the GUI portions.
Well, go ahead and attach what you've got, please.
(In reply to comment #30)
> Well, go ahead and attach what you've got, please.

Can I attach a zip file with the project?  Also it is an InstallShield
Adminstudio project and thus will be useless to most people.  If someone has a
scripting method to create an msi file for this I can provide the dialogs and
such that I use.  This bug is my first posting in bugzilla so I am a newbie to
this.  I am willing to help as much as my limited knowledge can help.  I love
both Firefox and Thunderbird and hope to see some of the corporate features
added that I need to help get support for them where I work.  Collaborative
calendaring and msi support are the big ones for us. Although of course I have
taken care of the msi thing although I will be the first to admit that my skills
with msi's are at best mediocre.
Yes, you can attach zips.
I cannot attach as it is to big.  A little over 1 MB.  I could put it out on an
ftp server if that will suffice.
(In reply to comment #27)
> I have also built an msi file for Firefox.  We use it at my college.  The 
0.9.3
> version is available at:
> ftp://anon:anon@www.lakeland.cc.il.us/wlilly/Mozilla%20Firefox%200.9.3.msi

I can't express how grateful I am for this. I hope there will be official MSI 
packages of future FireFox versions, it's essential for deployment in large 
organizations such as the school I work at. I'd like to help but I'm not that 
proficient with Windows Installer yet. Please have a look at 
http://www.appdeploy.com/ for more topics related to MSI and application 
deployment.
MSI would really be a dramatic help even for small organizations.  One of the
prime drivers away from Internet Explorer is concern about security issues.  If
Firefox had an MSI installer, an administrator could automatically push browser
patches to every desktop.  Without an MSI installer, this responsibility falls
to the users -- this is a deal-breaker in many cases.

(In reply to comment #9)
> http://sourceforge.net/projects/wix/
> 
> Perhaps WiX would be a good way to do this?  I'm not sure how much its status as
> CPL matters, if any (it looks like the CPL only applies to distribution of code,
> not product use).  It would seem to be a good way to go, as it's been used
> before by Microsoft and has command line tools that would fit into the command
> line based build process the Mozilla tree uses.  (I haven't worked with it,
> tested it, or gone much beyond reading the blurb about it on the WiX site, so I
> might be wrong about its utility for this bug.)

It looks like this may be a decent option indeed.  You create an xml file with
installation instructions.  It seems as if you can even design dialogs...

Some examples are available here:
http://cvs.sourceforge.net/viewcvs.py/wix/wix/doc/examples/

Does the installer do anything besides literally copy files over?  I mean, aside
from showing the license and etc.

-[Unknown]
Ok, created a makemsi script for making an firefox msi with enUS from the zip 
file.  It can be deployed via AD, doesn't require admin to do a first run, and 
installs flash.  it can also set itself up as the default browser (whole 
system).  A few questions, the zip file is vastly different from the .exe/  
There's xpt files, some dll files, .js, etc...., what's going on?  relevant to 
functionality?

I am also having problems using the same procedure to produce localized 
versions.  I tried to make a ja-JP version using the xpi file from the 
Japanese group, and I couldn't get firefox to integrate it.  Directions at 
http://texturizer.net/firefox/localize.html 'How to make a localized Firefox 
build' doesn't work.  Author doesn't respond to email.  
They're just compiled slightly differently.  The stuff is in there.  We don't
have permission to distribute Flash, so take that out.  Texturizer.net is kinda
old.  Search mozilla.org for localization information.
Thanks for the tips.  Regarding distributing Flash, I believe it is allowed 
for firefox.  I have a license to do so for my software which Macromedia 
sugguested back in April, 2004.  It is called a 'distribution agreement', 
located here 
http://www.macromedia.com/support/shockwave/info/licensing/license.html
It is per product so a new agreement will need to obtained.  

The enabling clause is 2)a)iii)
'through the Internet to end users, solely as a part of or with, with its 
Licensee Product (such as bundled in Licensee¡¦s installer which, in turn, is 
downloaded from the Internet).'

This means the installer needs to incorporate the Flash components which the 
MSI does.  There are some marketing considerations though, like displaying the 
Flash enabled logo, etc. 

Comments welcome.
What I'm suggesting is that you meet the minimal current requirement: a
scriptable way to create an MSI package.  Bug 229590 deals with whatever plugins
we may bundle.
Eric's port works perfectly on my system. I was able to install and uninstall it
easily and it went smoothly. Much much better than the current system. All my
extensions work fine. Flash works fine, right out of the box.

I recommend this MSI method for 1.0 final.
Tests for my MSI's are being tracked here.

http://forums.mozillazine.org/viewtopic.php?t=138033
Draconpern's msi installer with the Fx 2004-10-01 build also works without flaws
as best I can tell.
(In reply to comment #37)
> Ok, created a makemsi script for making an firefox msi with enUS from the zip 
> file.

Is this script available somewhere for inspection/testing?
(In reply to comment #44)
> (In reply to comment #37)
> > Ok, created a makemsi script for making an firefox msi with enUS from the zip 
> > file.
> 
> Is this script available somewhere for inspection/testing?

http://forums.mozillazine.org/viewtopic.php?p=846746#846746
Here's someone else doing what sounds like a great job. He set and claims to
have achieved the following goals:

1. Deploy FF using Active Directory GPO/Intellimirror
2. Make FF the default browser
3. Set a default profile for new users including correct extensions, shortcuts
and preferences
4. The default profile is set without user intervention or interaction (no
import settings dialog)
5. Add a shortcut to FF on the desktop and Start menu
6. Include necessary plugins (Java and Flash)

http://www.mikrotuki.org/ffguide.pdf
A deployment guide would be great!  But that's a separate issue. 
yuu
OpenOffice.Org 2.0 (beta) now uses MSI for installing...
I've been able to make my own .msi Firefox install.
The Biggest issue I ran into is that Firefox is *very* specific about the string
it looks for to make sure it's the default browser down to being case sensitive.

This makes it somewhat more difficult (possibly depending on the MSI packager
used) to make Firefox the default browser no matter what directory it is
installed to.

The files used to create my .msi are also available are my site:
http://www.webheat.co.uk/firefox.php
Flags: blocking-aviary1.0- → blocking-aviary1.0?
Flags: blocking-aviary1.0? → blocking-aviary1.0+
Assignee: bugs → cmp
Mike Dixon,  I wonder if we could plug what you have done into one of the
tinderbox systems and have it kick out an msi package every day?

what do you think it would take to make that happen?

chris h.
I don't think we can make 1.0 on this.  The status still shows as NEW, and 
probably too close to 1.0.
I am releasing .msi's of non-nightly's in the the forum
http://forums.mozillazine.org/viewtopic.php?t=138033

Wouldn't it be ideal if we could sync the msi installer with the regular
installer? Maybe dynamically generate the script that creates the msi from
browser/installer/windows/packages-static. Having a shared set of registry
entries that need to be set might be more of a problem.
(In reply to comment #53)
> Wouldn't it be ideal if we could sync the msi installer with the regular
> installer? Maybe dynamically generate the script that creates the msi from
> browser/installer/windows/packages-static.

Yup, that would be ideal, except currently, all the registry info in the 
mozilla installer are hard coded (AFAIK). Worse, there are options that 
firefox sets which are hard to figure out (eg lots of LXR searches, no Xref 
available). For example, I am working on my msi bug where even if you install 
it with 'set as default browser', it still asks you if you want to set it. 
Grrr.
(In reply to comment #53)
> Wouldn't it be ideal if we could sync the msi installer with the regular
> installer? Maybe dynamically generate the script that creates the msi from
> browser/installer/windows/packages-static. 
My intention would be that this would eventually replace the current windows
installer.  I think that we'd be able to avoid several of the UI bugs that the
current installer has.

> Having a shared set of registry
> entries that need to be set might be more of a problem.
You lost me there.  


(In reply to comment #54)
> Yup, that would be ideal, except currently, all the registry info in the 
> mozilla installer are hard coded (AFAIK). 
Instead of looking at the installer, try looking at the Options dialog where
that button launches.  Maybe this link will help:
http://lxr.mozilla.org/aviarybranch/search?string=nsIShellService
I found it by searching for the Check Now button.


The current installer does a number of things - including using the xpinstall
engine which will eventually keep track of installed packages and so on, making
it hard to obsolete unless you know all the requirements and features. No one
knows all of this just yet so we can't "replace" anything. 
I don't know much about creating MSI files, but the Office Developer Package and
Deployment wizard doesn't include Windows Installer 2.0.

When installed on a 98 machine (for example) without Windows Installer 2.0 it
pops up a dialog telling the user to download it and directs them to
http://support.microsoft.com/default.aspx?scid=kb;EN-US;292539 (IIRC).

This saves adding Windows Installer to the package.
This is not going to block the 11/9 launch however Chase will continue to work
on it as it is still very important. The msi installer may trickle in before, at
the same time or after the 1.0 release. 
Flags: blocking-aviary1.0+ → blocking-aviary1.0-
Priority: -- → P2
Target Milestone: --- → Firefox1.1
I would like to add that mysql now uses http://sourceforge.net/projects/wix/ to
deploy mysql on windows, so someone might want to look into how they do it and
if they have the scripts/xml files available to the public that can be used as a
reference
(In reply to comment #56)
> The current installer does a number of things - including using the xpinstall
> engine which will eventually keep track of installed packages and so on, making
> it hard to obsolete unless you know all the requirements and features. No one
> knows all of this just yet so we can't "replace" anything. 

Can you gather a list of requirements and features for this project to meet? 
What needs to be installed so that we can call the xpinstall engine from MSI?

(In reply to comment #59)
> I would like to add that mysql now uses http://sourceforge.net/projects/wix/ to
> deploy mysql on windows, so someone might want to look into how they do it and
> if they have the scripts/xml files available to the public that can be used as a
> reference

wix was first mentioned in comment #9, and I agree that it sounds like a great
way to go.  Documentation in their packages is a .chm file.  The process looks
pretty easy and of course we can dynamically generate the necessary XML files.
for an unoffical MSI packaging for 1.0 see

http://forums.mozillazine.org/viewtopic.php?t=138033

These were built using MakeMSI
mscott also saw this post...

http://forums.mozillazine.org/viewtopic.php?t=166949&sid=09ad647ea29533eaa5aee89a2d0d1319


tuaris
Joined: 02 Jul 2004
	PostPosted: Nov Thu 18th 2004 2:40pm     Thunderbird MSI
I created a MSI package using InstallShield X for use with Active Directory
group policies. I made the project files available at
http://www.danielmorante.com/index.php it's one zip file for Firefox, and
another for Thunderbird. They include all project files needed for building and
editing.

The only problem I'm having is with Advertised shortcuts, I can't get the
installer to "create" certian registry settings and shortcuts on a per-user
basis. If someone could help me out it would be great.

You are free to use this MSI project anyway you wish, jus give me credit and
send back and improvements, changes or fixes.
Blocks: 271208
Assuming there are no blockers on license issues and it's a better solution, we
should add WiX to the list of candidates for MSI packaging further down the
road.  For now, I'll use whatever information is available in the links listed
in this bug to work on generating a MSI package on our build systems.
Will all the win32 builds be switching to an MSI based installer, or will it be
an alternative download?

Also how feasible is it for an MSI to support a multi-region installer (bug
#269869)?
> 
> Also how feasible is it for an MSI to support a multi-region installer (bug
> #269869)?

you can use ORCA to export the UI strings into a text file, edit those and
import them back in.  The importing part can be scripted (at least with MakeMSI).

Attached file My 1.0-abAB firefox MakeMSI script (obsolete) —
This is my current FireFox MakeMSI script.
there's a com interface to msi files which lets you use sql to query / update
the msi tables (via database views).

http://msdn.microsoft.com/library/en-us/msi/setup/installer_object.asp
Blocks: 271351
Assuming there are no additional complications involved in making an MSI for
Thunderbird, can this bug be morphed to include Thunderbird? It seems daft to
file a new bug if there are no Thunderbird-specific issues.
(In reply to comment #68)
> Assuming there are no additional complications involved in making an MSI for
> Thunderbird, can this bug be morphed to include Thunderbird? It seems daft to
> file a new bug if there are no Thunderbird-specific issues.

There would be Thunderbird-specific issues.  If you need to confirm this, just
look at the contents of mozilla/mail/installer/windows/mail.jst (46K) and
mozilla/browser/installer/windows/browser.jst (26K) -- the contents are entirely
different, and these files are only some of the files that would need to be to
some degree ported to whatever MSI creation method is used, so this really can't
(and shouldn't, if we want to get any sort of traction here) be morphed.
(In reply to comment #69)
> (In reply to comment #68)
> > Assuming there are no additional complications involved in making an MSI for
> > Thunderbird, can this bug be morphed to include Thunderbird? It seems daft to
> > file a new bug if there are no Thunderbird-specific issues.
> 
> There would be Thunderbird-specific issues.
OK then; I've filed bug 274624.
Attached patch Use $(srcdir) for make-msi.pl (obsolete) — Splinter Review
When making objdir builds, perl can't find msi/make-msi.pl because it's being
run in the object tree.  Telling it to use $(srcdir) works around this and it
finds the right file.  This patch does not fix the whole bug, of course :D 
Also, since I don't have makemsi, I can't actually test to see if the script
works...  Just that it runs.  $(SRC_DIR) (cygwin paths) did not work for me.
 
(Submitting because a change to .../installer/windows/Makefile.in - rev.1.11 -
made my build die near the end.  Feel free to ignore :D)
Attachment #169500 - Flags: review?(cmp)
Comment on attachment 169500 [details] [diff] [review]
Use $(srcdir) for make-msi.pl

I committed a slightly different form of this patch to CVS just now in
Makefile.in's revision 1.12.  You should not encounter the 'missing
make-msi.pl' error, now.  Can you try another build with the latest trunk to
check that this is the case?
Attachment #169500 - Attachment is obsolete: true
Attachment #169500 - Flags: review?(cmp) → review-
Depends on: 229706
I can confirm that this generates a functional MSI with objdir builds now.
The Firefox trunk build machine is now generating an MSI file that installs and
runs the Firefox installer.  Silent installer functionality is not yet available
and is being tracked in bug 229706.
Flags: blocking-aviary1.1?
Trunk Windows build machines are now generating MSIs based on the installer.  I
have tested these MSIs in a personal deployment and the installation proceeds as
expected with a period of no activity occurring where the installer is invoked
silently.

Can anyone in a position to run a quick test of these MSIs in an automated
roll-out across a small number of machines via Active Directory do so and
provide any feedback?
Please try to design the MSI so that it can be customised (using an MST - a 
transform file). One particular area of customisation, however abhorrent it 
may sound, is that not all corporates will want Firefox to be the default 
browser - they may simply want to be able to install it alongside IE but keep 
IE as the default.

Thanks.
finally cleaned this up. I am uploading my RestoreIExplore.dll source project.
This should be included to restore IE when an uninstall is performed, otherwise
your system is going to be in a very bad state. Instructins in readme.txt.  For
MakeMSI scripts.
Uhm... guys, I am looking at 
http://lxr.mozilla.org/seamonkey/source/browser/installer/windows/msi/firefox.m
m

That's not the way to do an MSI!  You are going to break people's machines.  
The data files really need to sourced by the mm script, not call a setup 
program.  Also, how are uninstalls going to be performed?
Attached file managed firefox.mm (obsolete) —
I have changed my earlier script to integrate w/ the firefox build tree.  Can
someone drop this in place and see if it works?  This doesn't set default
browser by default. I also commented out any RestoreIExplore.dll, but if you
want to use that, compile my previous upload.
Attachment #166813 - Attachment is obsolete: true
(In reply to comment #77)
> This should be included to restore IE when an uninstall is performed, otherwise
> your system is going to be in a very bad state.

What if IE wasn't the default browser before Firefox was installed?  Perhaps
backups should be made of existing registry keys before changing them?  Probably
that should be another RFE, but I think it would be very easy at this point to
put that functionality into this MSI file, if not the installer.
(In reply to comment #80)
> What if IE wasn't the default browser before Firefox was installed?

What if Firefox isn't the default browser when the uninstall is performed?  The
default browser should only be changed if Firefox is the default browser at the
time of uninstallation; otherwise it is messing with the settings of other
programs.  See bug 277733.
I would like to see Bug 264889 implemented in this setup, if a new setup is
being created.
Attached file MSI using WiX toolset (obsolete) —
Here is my result of playing with the WiX toolset.  It is a wxs file that can
be candled/lighted to create a Firefox 1.0.1 MSI.  Note that it lacks a lot of
features compared to the MakeMSI scripts, but it can be done.  Note that the
source is the 1.0.1 zip install.
Flags: blocking-aviary1.1? → blocking-aviary1.1+
(In reply to comment #25)
> The problem with including the Windows Installer package within the installer
> is that the installer size would jump up from 4.7MB to somewhere in the region
> of 6-7MB. This is unacceptable as a long-term solution.

I do not not find this to be "unacceptable" : most downloadable programs out
there have install packages that are about that size (if not a lot more). When
people are downloading a software during its setup process, don't worry, they
show more patience toward its download time than when just waiting for, let's
say, a Web page : they know they are setting up a sensitive application for
their browsing experience, not just "browsing" a Web page... ;-) (and this can
be mentioned if you want to make sure that those dinosaurus, who can't make the
difference between the Web and their hard drive, really understand what they are
doing - but I think that most people are aware of this nowadays)

On the opposite, if I've noticed that this app's installer would weight less
than 2-4 megs, I would've thought "oh, another school project... let's pass",
especially for such an app, that's supposed to include many good features to be
contemporary and competitive. You know, it's about the same psyche as for many
consumers, to whom their expenses are a sign of their whealth (and thus their
vanity : being rich is considered a human value for many) : the price in itself
sets some quality standard in many people's mind, even without actually having
tried the product. We can make the same parallel between the weight of an
install package and its underlying quality (which is stupid of course, you and I
know it as we are "knowledgeable" of the tech arena, but that's not the case for
the majority of people though).
Blocks: 267888
No longer blocks: 271351
OpennOffice.org's version 2 beta uses a pretty well-designed MSI. Maybe they
could be helpful to Mozilla?
Blocks: majorbugs
No longer blocks: majorbugs
This will need to mix well with the Deer Park update mechanism.  Is this still
going to make 1.5?  Do we need a separate bug for MSI & Updates?
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk/

There is no MSI for 1.6a1

Also, the MSI package didn't prompt me to install developer tools when I chose
custom.
QA Contact: bugzilla → installer
pushing onto the nominations list, but I think we're at a point where we aren't
going to see a proper MSI for 1.5
Flags: blocking-aviary1.5+ → blocking1.8b4?
Flags: blocking1.8b4? → blocking1.8b4-
Flags: blocking-aviary2.0?
(In reply to comment #88)
> pushing onto the nominations list, but I think we're at a point where we aren't
> going to see a proper MSI for 1.5

Or of any of version at all... I am not in favor of MS formats, even though it's
Windows one is talking about, but truth be told, this would be amazingly helpful
for lots of system admins in offices and such, and I think it should be a priority.

Did anyone try to contact apache win32 devs or the OOo team regarding this?
*** Bug 314655 has been marked as a duplicate of this bug. ***
As a System Administrator, I have gotten many small shops to standardize on Firefox as the default browsers (justification in TCO due to spyware/virus/foistware in pre-WinXP-SP2 operating system).

I would like to automate installations upon log-on for ALL dekstops through
Group Policy Object facility within Active Directory environment, but I cannot
do this with Firefox, as it is packaged as an .EXE file. Unfortunately in order
to harness this level of automation, the software must be packaged as an .MSI
file.  I have been successful with deployments of Perl through ActiveState's adoption of MSI.

Non-Automation of installs/deployment (example: anti-virus) is definately a blocker for selection of technologies at large corporations.  I know of corporations like Citicorp that don't use IE for security purposes.
This is something I think we want/need sooner instead of later, but we'll need one of our build gurus, and I don't think Chase has cycles to do this type of thing at present.

Benjamin? Other volunteers?
Assignee: chase → nobody
Flags: blocking-aviary2? → blocking-aviary2+
Keywords: helpwanted
I'd like to volunteer, contingent on timeframe.  I re-wrote the MSI installer for TortoiseSVN (http://tortoisesvn.tigris.org) using the WiX toolset (http://wix.sf.net).  I would describe my WiX ability as competent but not advanced.

I would not be able to produce graphics of the quality you guys seem to do as a matter of course, but I would be able to provide detailed requirements for such.

I would prefer to work to a spec detailing what the current installer does, instead of attempt to replicate what it does based on what I as a user can see it doing - I might miss stuff.  I would also prefer to start simple and work up from there.

What's the deal - do I just jump in, provide WiX code (I see someone already has, but without a UI), and instructions on how to integrate it into the build?  I don't know whether the WiX tools work on non-Windows platforms (they require .NET; don't know if they work with Mono), and I also don't know much about your build process; guidance there would be appreciated (links to building on Windows probably sufficient).
(In reply to comment #93)

Oh; 'scuse me.  It does have a UI.  WiX has moved on since February a bit, though, and now provides a UI common library and things.
I don't think there's a formalized spec, but there's probably room for improvement in our current installer.  Graphics we can handle quite easily so its mostly a matter of making it build within the tree and implementing whatever spec we hook up.

ccing beltzner, since I think he had something like a spec around at some point.
What I would like to see is the MSI use the ZIP build as a starting point, not the installer: it can just extract the files to the system as appropriate. Then we need to recreate the logic to register the app in the Windows registry and create appropriate shortcuts, matching up with the existing code at

http://lxr.mozilla.org/mozilla/source/browser/installer/windows/browser.jst#115
and
http://lxr.mozilla.org/mozilla/source/browser/installer/windows/ab-CD.jst#10
(In reply to comment #83)
> Created an attachment (id=175803) [edit]
> MSI using WiX toolset
> 
> Here is my result of playing with the WiX toolset.  It is a wxs file that can
> be candled/lighted to create a Firefox 1.0.1 MSI.  Note that it lacks a lot of
> features compared to the MakeMSI scripts, but it can be done.  Note that the
> source is the 1.0.1 zip install.

Would the frontmotion msi file solve all of these issues? 

http://www.frontmotion.com/Firefox/fmfirefox.htm

Has anyone talked to the front motion people about the msi file. an msi file for firefox would eliminate the biggest roadblock to widespread corporate adoption of firefox...
(In reply to comment #97)
> 
> Has anyone talked to the front motion people about the msi file. an msi file
> for firefox would eliminate the biggest roadblock to widespread corporate
> adoption of firefox...

Frontmotion has also been mentioned as a possible solution to bug #267888 (Windows 2000/XP/2003 Group Policies support).  The lack of GP support is also a corporate issue.
The problem with Frontmotion Firefox is that the icon is changed, and the word "Mozilla" is replaced with the word "Frontmotion".  Both of these changes devalue the Mozilla group and brand.  Also, their page says that they may start charging for MSI's at some point in the future.  We should not rely on this company to meet our current and future needs.
> The problem with Frontmotion Firefox is that the icon is changed, and the word
> "Mozilla" is replaced with the word "Frontmotion".

You *do* realize that because Frontmotion is distributing a modified version of Firefox (called their "Community Edition", with GP support among other things), they are forced by mozilla's licensing/trademarks to change the name.
I understand what you are saying, but deploying a program called "Frontmotion" to thousands of computers on a corporate network, when most people are only just starting to learn the word "Mozilla" is not a good idea on the network I administer.  On the same note, people look for icons & colors, occasionally more than the name below the icon, so if we start distributing a different kind of icon here from what people are starting to become accustomed to, then Firefox will have fewer converts, at least at my location.

My main concern is that relying on a company to re-package an open source program into a form we can all use kinda goes against the spirit of the whole project.
I don't think it does. They do it for free, and have said that if they will charge, it will only be for the purpose of covering development costs. I wish they'd release any source used, but I'm not going to throw out the baby with the bathwater. Besides, at issue isn't getting them to do it, I didn't think, but asking them how they do it and adding that as a script to the make file a la '$ make msi'. Given that they've expressed that it is tedious to do it, they may be amenable to it. If not, then they truly aren't holding to the spirit of things, and I see no reason to continue to concern ourselves with it, but only continue the effort here.
> My main concern is that relying on a company to re-package an open source
> program into a form we can all use kinda goes against the spirit of the whole
> project.

Not to add too much noise to this discussion, but that's basically the whole point of the Mozilla Corporation, so I don't think that model is inherently flawed for us.
(In reply to comment #103)

Relying on Frontmotion is an absolute no-go.  I'll be clear and say I think what they've done is really great, I have exchanged many emails with Eric from Frontmotion and, indeed, one of the ADM templates that Frontmotion's MSI uses was written by myself.

However, there is a fundamental flaw for those who have used the Frontmotion MSI, namely - they are doing it for free, as a favour to the community and have no service commitment to this.  What I mean by that is, what if a very critical, 0-day flaw comes out and the recommended resolution is to upgrade to the latest version of Firefox.  This third-party MSI might be produced in hours, but the guys from Frontmotion may be busy, may be away on holiday and it could take weeks.

What I am saying is that producing an MSI is not just a technology issue, but is a service commitment.  There is no point unless there are set Service levels which say something along the lines of "an MSI will be produced for any patch within 24 hours".

That's not an unsurmountable task.  For any new version of Firefox, I produce an MSI for Firefox within a couple of hours.  Unfortunately, that is using Wise Package Studio and my license says I cannot redistribute the MSI outwith my organisation.  However, I could put significant input into the manifest of what should be in there, validate the ICEs, QA, etc.  I have also created an extension (ADM XPI) which could be used to allow any setting to be set via GPO.

However, I believe that it is best to concentrate on trying to produce a script for WiX or MakeMSI, some documentation on how to convert that into an MSI, and most of all, a commitment from Mozilla Corporation to eventually take up that service commitment for any updates for Firefox.
What can be learned from the Thunderbird folks who are already building MSIs?

http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-trunk/thunderbird-1.6a1.en-US.win32.installer.msi

See also Bug 274624 and Bug 229706
It could be useful to see the MakeMSI file they use to build that.  However, it is far from being anything that could be described as a useful MSI.  Looking at the MSI, it is simply the same as the MSIs that came out of Firefox early on last year - it simply wraps the setup files and then calls the setup.exe in command-line quiet mode.

This will lead to it installing correctly, but being virtually unconfigurable and will also fail in the other key areas of MSI such as self-repair and uninstall.

It would be a start though - for those of us desperate for anything. If anything, it could serve as a catalyst for someone to take the reins and add the extra MSI functionality.
(In reply to comment #107)
> It would be a start though - for those of us desperate for anything. If
> anything, it could serve as a catalyst for someone to take the reins and add
> the extra MSI functionality.

Hmm... Or it could make it so that people say 'we have an MSI that works, why bother?'

Also, for corporate installations, this sort of installer could cause more problems than it solves. If people try it and it doesn't work or uninstall properly, it could turn them off FireFox for good. How are you going to explain to them that 'we've fixed the MSI installer so it works properly now' - they'll ask why we didn't get it right the first time.

I really think this should be done properly from the word go.
Some of the makemsi files where removed from the CVS repository for licensing issues - see bug 324118.
Do we have any idea why the MSI files are so big?

Does Microsoft not do any compression at all for the MSI files?
The MSI itself just has the files as they come, AIUI, and you are advised to use cabinet files to compress the data. As far as I can see, the best way is to take all your files, make a single .cab file of them, and then tell the MSI that each file is inside that .cab file. In particular, the item "Including a Cabinet File in an Installation" on the following page seems to do well at explaining how it is done:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/msi/setup/using_cabinets_and_compressed_sources.asp
> Do we have any idea why the MSI files are so big?

What makes you say that?  IMO, the overhead of .msi files is small compared to the payload we're talking about here.

> Does Microsoft not do any compression at all for the MSI files?

Compression is optional.
(In reply to comment #113)
> > Do we have any idea why the MSI files are so big?
> 
> What makes you say that?  IMO, the overhead of .msi files is small compared to
> the payload we're talking about here.

On my initial tests, the MSI was 6 meg - that's 20% bigger - I'd love to get the MSI files to a size that Mozilla would use them as the default install package instead of going to Nullsoft - that would require them being under 5 MB.
The .msi's format simply provides more features (ease of customization/deployment/upgrades, automatic repair among a few), so one would expect it to be at least a little larger.  And, frankly, I (and most AD admins I'd wager) could care less whether the .msi version is a little (20%) bigger.
MSI files support MS .cab compression either within or external to the MSI itself. The overhead of the MSI schema itself is relatively small -- certainly under 200kB. If greater compression is desired, it's easy to wrap the MSI and a call to msiexec in a self-extracting executable using whatever compression you like. (Note: This is what Adobe does with Acrobat.)

Call this the hair that broke the camel's back... or something. Whatever.

Most (if not all) AD installations of Firefox via MSI will be through an administrative installation point, which will take up the full 15-16MB of an uncompressed installation. The difference between 5 and 6MB is irrelevant. Mozilla has refused to provide (or accept any number of offers to create) an MSI-based installer -- even as an option alongside other installation mechanisms. Firefox chooses file versioning rules that specifically break Windows Installer's healing and update capabilities. Firefox insists on using the uninstall program for functionality unrelated to uninstallation (making it impossible to eliminate the otherwise unnecessary uninstall program from an MSI if one were made). Firefox still refuses to provide a mechanism of patching Firefox installations for non-admin users that doesn't require un/re-installing or touching every affected box (vastly unrealistic given the frequency of security updates). Firefox provides no means of pushing out administrator-defined "mandatory" and "recommended default" settings. Firefox does not use the registry, so per-setting access control is impossible. (I realize this is done in the interests of interoperability.) Firefox does not uninstall cleanly. Firefox has not fixed cifs file-handling bugs that primarily affect roaming profiles in enterprise environments. Documentation about Firefox, preferences, customizability, etc. is sketchy, aside from source code. The sum of these observations can be interpreted as hostility to deployment of Firefox in enterprise environments. 

In light of this, why should an enterprise struggle to deploy Firefox and take on the huge cost of deploying continuous updates when (from an enterprise deployment perspective) vastly superior products are available? You'll want examples... Opera does better in some ways, and of course IE has the deployment and update stuff nailed.

(Oh, and don't bother answering that question, because I already know.)
(In reply to comment #116)
> Mozilla has refused to provide (or accept any number of offers to create) an
> MSI-based installer -- even as an option alongside other installation
> mechanisms.

That is the problem I am trying to solve.
> Firefox chooses file versioning rules that specifically break
> Windows Installer's healing and update capabilities.

Can you provide more information on this please?

> Firefox insists on using
> the uninstall program for functionality unrelated to uninstallation (making it
> impossible to eliminate the otherwise unnecessary uninstall program from an MSI
> if one were made).

Can you provide more information on this please?

 Firefox still refuses to provide a mechanism of patching
> Firefox installations for non-admin users that doesn't require un/re-installing
> or touching every affected box (vastly unrealistic given the frequency of
> security updates).

Isn't this solved with the new update feature?

> Firefox provides no means of pushing out
> administrator-defined "mandatory" and "recommended default" settings.

Actrually, the CCK does this (http://www.mozilla.org/projects/cck)

> Firefox
> does not use the registry, so per-setting access control is impossible. (I
> realize this is done in the interests of interoperability.)

Do you mean use the registry instead of prefs.js for user configuration?

> Firefox does not
> uninstall cleanly.

Can you be more specific on this?

> Firefox has not fixed cifs file-handling bugs that primarily
> affect roaming profiles in enterprise environments. Documentation about
> Firefox, preferences, customizability, etc. is sketchy, aside from source code.
> The sum of these observations can be interpreted as hostility to deployment of
> Firefox in enterprise environments. 

Again, a problem we are trying to help solve.

> 
> In light of this, why should an enterprise struggle to deploy Firefox and take
> on the huge cost of deploying continuous updates when (from an enterprise
> deployment perspective) vastly superior products are available? You'll want
> examples... Opera does better in some ways, and of course IE has the deployment
> and update stuff nailed.
> 
> (Oh, and don't bother answering that question, because I already know.)
> 

(In reply to comment #117)
> (In reply to comment #116)
>  Firefox still refuses to provide a mechanism of patching
> > Firefox installations for non-admin users that doesn't require un/re-installing
> > or touching every affected box (vastly unrealistic given the frequency of
> > security updates).
> 
> Isn't this solved with the new update feature?

Only if the Firefox install directory has write access for the user.
I don't think you can characterize "indifference" as "hostility" but otherwise there's some valid points there.  We're heavily focused on the consumer space, but that doesn't mean we're going to be hostile to efforts to spread the love.

That said, I don't see the real value in shipping a stock Firefox MSI right now.  What I _do_ see the value in is in building out a customization kit that would allow the creation of Firefox MSI packages for internal deployments, since its the custom packaging that really helps solve the problems (i.e. setting up remote pref locking, custom bookmarks/homepage, etc).  Obviously there are some significant redistribution licensing issues involved there, but that's the path that I think we'd ultimately benefit from the most.
Can we move this meta-discussion to a wiki page somewhere and identify a realistic set of requirements for a MSI? We're not "refusing" MSIs in general, but we're going to need somebody to contribute a reliable MSI build system which when installed matches the binary bits produced by the .exe installer.

A lot of the requirements articulated around honoring preferences and group policies from the registry is not really related to the MSI itself at all, except to the extent that apps shipped as MSIs also typically honor registry-set prefs and group policy.

Let's keep this bug about the MSI packaging and installation implementation and use other existing bugs to discuss prefloading and group policy.
I concur with Benjamin Smedberg.  I actually want to address the later issue in a seperate bug where I can look deeply into the issue and provide plausible solutions, but I haven't put together something until this one issue is resolve: MSI package.  Let's just have one battle at a time.

I've just started a contract with EDS and Department of Defense (USA), and I don't know what my influence will be for using Firefox due to security issues with some bundled product in Windows, but I hope I'll have the same success as I have had with previous clients in SF financial district.  Given the number of installs would be in the thousands and thousands (assuming I capture some attention on the topic) this well, is bug quite interesting.

I'm totally ignorant as far as implmentation of MSI (install and uninstall), but I would be willing to sacrifice my gaming time and personal project time on this, welcome e-mails or mailing list, forum, etc. on this...
this isn't really on radar, and not a blocker for Firefox 2.
Flags: blocking-firefox2+ → blocking-firefox2-
Priority: P2 → --
Target Milestone: Firefox1.5 → Future
(In reply to comment #122)
> this isn't really on radar, and not a blocker for Firefox 2.

I still don't understand why not.

When Firefox came out it got incredible positive feedback from the press. Almost all of the articles identified the biggest flaw in Firefox - the lack of features for corporations. Without the features, Firefox doesn't have a chance at unlocking IE's corporate stranglehold. 

Business don't want to muck with creating MSIs or relying on unsupported 3rd party tools. What happens when Firefox tries to update itself automatically after it was installed from an MSI? How will administrators deploy updates? Why would they be expected to put in lots of extra time when they already have a working browser (IE). IT personell need to justify their time to their boss - how are they going to justify an expensive (and in management's eyes, risky) company-wide Firefox deployment?

MSI is Windows' native installation package. Saying you don't want to support it directly is about the same as telling users that you don't fully support Windows. 

No offense, but the entire reason Firefox is popular on the Windows platform compared to other non-Microsoft browsers is the fact Windows is treated as a "first class citizen" - it was designed to look and feel like a true Windows app. No posix dependancies/libraries to install, X-Windows GUI widgets, etc.

Every time I hear of a new Firefox release, I check to see if the following things have been resolved:
* Does it support policy deployment?
* Is it MSI?
* Have they gotten rid of that horrible bookmark.htm file?

The only impression I've gotten from the discussion of this bug/feature request is the typical "It's an Evil Microsoft feature who cares about that" response I'd expect from other Open Source projects, but not Firefox. 

122 comments later, and your Windows users, politely begging for this critical feature, have been swept under the carpet.
Re: shadowchaser@transfur.com 

Hear, hear!  [clap clap clap]
I have been doing my own MSI for sometimes now, so I can give you my thoughts.

From the my users perspective, their risk is that I may not be around forever creating the MSI.  Of course I will do my best and have been for a while which is why I have such a large install base now.

The MSI's do disable the auto update feature.  This is required as they won't have write permission to the installed files.  My community edition does support group policy w/o ugly and unsecure login scripting.  But in order to support that feature, I did have to add Windows specific code.  I don't think the dev team wants to take the development in that direction.  As for bookmark.htm, you got me there. :/

(In reply to comment #123)
> working browser (IE). IT personell need to justify their time to their boss -
> how are they going to justify an expensive (and in management's eyes, risky)
> company-wide Firefox deployment?
[snip]
> Every time I hear of a new Firefox release, I check to see if the following
> things have been resolved:
> * Does it support policy deployment?
> * Is it MSI?
> * Have they gotten rid of that horrible bookmark.htm file?
We have consistently stated that our goal is to support consumers, enterprise deployment is an entirely different scenario that we're not set up to support at present.  Entering the enterprise market requires a lot more than just an MSI and some group policy support, its a matter of adopting an entirely different focus than targeting the consumer market.  Right now, it doesn't make a lot of sense for us to do that.
(In reply to comment #126)
> We have consistently stated that our goal is to support consumers, enterprise
> deployment is an entirely different scenario that we're not set up to support
> at present.  Entering the enterprise market requires a lot more than just an
> MSI and some group policy support, its a matter of adopting an entirely
> different focus than targeting the consumer market.  Right now, it doesn't make
> a lot of sense for us to do that.

Actually the enterprise market has been consistently stated as an important growth area.

"Continuing to fill needs for enterprise customers, seeing wide deployment in Fortune 500 companies, universities, as well as wherever high quality and reliability are important." Chris Hofmann, http://www.digital-web.com/articles/chris_hofmann/

"A: It fills a wide variety of needs in many different niches. We have talked to several enterprise and large organizations that have Mozilla deployments in place, or are considering them." Chris Hofmann, http://www.freeos.com/articles/4695/

I disagree with you that entering the enterprise market will take a lot more than an MSI and GP support. In most cases this is the only reason Firefox has not been deployed in this environment. 

This is a "Perfect Solution" fallacy. "The perfect solution fallacy is a logical fallacy that occurs when an argument assumes that a perfect solution exists and/or that a solution should be rejected because some part of the problem would still exist after it was implemented." http://en.wikipedia.org/wiki/Perfect_solution_fallacy

For example, we shouldn't put seat belts in cars because people will still die in traffic accidents.
They're not saying they won't make an MSI.  They're saying they won't have it in time for Firefox 2.0.  As always, your assistance in creating patches is appreciated.  
It's not clear why enterprise support is such a low priority.  What people use at work is fairly likely to trickle down to what they use at home, whereas the converse is not true.  Firefox had a golden opportunity to capture market share when the press was obsessed with IE's vulnerabilities.  But as everyone here has pointed out, without an MSI it just did not make sense to deploy.  That's the major feature IT managers want.  The other enterprise stuff is just gravy.

I worry that you're going to look back at this as a huge strategic blunder.


(In reply to comment #126)
> We have consistently stated that our goal is to support consumers, enterprise
> deployment is an entirely different scenario that we're not set up to support
> at present.  Entering the enterprise market requires a lot more than just an
> MSI and some group policy support, its a matter of adopting an entirely
> different focus than targeting the consumer market.  Right now, it doesn't make
> a lot of sense for us to do that.

People routinely use the same software at home as they are accustomed to using at work or have even been taught to use at work (sometimes at considerable expense).  Corporate usage would also be a huge acceptance for Firefox, so I don't understand why these features are viewed as being so insignificant.

I don't really understand either why MSI wasn't used in the first place.  It is The Proper Way to install software on Windows platforms and has been for a considerable time.  I can actually live without GP, but MSI and corporate updates are a must.

So when is MSI and GP going to be a blocker?  In version 3.0 or 4.0 or ...?  Is MSI and GP going arrive to Firefox long after Windows Vista and IE7 (with its new look, tabs, improved printing, RSS feeds, integrate search, improved security; http://www.microsoft.com/windows/ie/ie7/about/features/) and all corporate support?

Please remember that Microsoft won the (1st) browser war and killed Netscape.  It can win another browser war and kill Firefox, and as far as Microsoft is concerned, the war probably hasn't even started yet.  Firefox vs. IE7 might become an epic battle and corporate support is needed.
I don't see why a msi with the basics it can't be, it was part of the early 1.1  (became 1.5) specs, and in the early part of it there were daily builds with msi installs. I think they were stopped due to license issues with the particular tools used to build it but I think Mozilla has since moved to a different installer system (NSIS??) since so a basic msi should be available at almost zero effort, and while it doesn't necessarily prove all the 'bolt down setings with AD+GPO' that would be very sweet in the corporate world for most small to medium organisations to be able to do forced automated msi upgrades to ensure security and consistency across the organisation with minimal administration effort would be a good start and often more than enough (which is the case in my organisation as we aren't overly interested in bolting down the application but would like to force upgrades with minimal admin effort to enforce security).
(In reply to comment #129)
> It's not clear why enterprise support is such a low priority.  What people use
> at work is fairly likely to trickle down to what they use at home, whereas the
> converse is not true.  Firefox had a golden opportunity to capture market 

Evidence shows the contrary, actually. People enjoy choice, and what we've seen is IT departments who convince their parent corporations to deploy Firefox based on requests from users who've switched at home. That enterprise innovation leads home use is a fallacy. Countless examples show that consumer/home technology leads enterprise (ex: IM, blogs, search, wiki, etc).

> when the press was obsessed with IE's vulnerabilities.  But as everyone here
> has pointed out, without an MSI it just did not make sense to deploy.  That's
> the major feature IT managers want.  The other enterprise stuff is just gravy.

Agreed. The problem is that they don't just want MSI. In fact, most IT shops that require MSI also require the sort of functionality that's provided by the CCK: group policies, ability to pre-populate profiles, restricted IP ranges, special proxies for internet use tracking, etc, etc.

This bug hasn't been WONTFIX'd because, well, we think it's worth fixing. That said, the value isn't worth blocking a release, nor has it gotten to be a high point on anyone's radar. Perhaps the IBMers working on CCK would like to take it or something like it. Perhaps you would like to start building a list of the features that IT departments would truly require, prioritizing the buglist, and driving the development of the feature.

Own it, baby, own it. :)

> I worry that you're going to look back at this as a huge strategic blunder.

It would rank somewhere in the middle of the list, don't you worry!
Personally in terms of priority in the enterprise feature list, the MSI tops the list.

Without Frontmotion's MSI package Firefox would not be deployed on our University desktops. The minute Frontmotion drops the MSI support, or starts charging for it, is the same time when an automated removal of the Firefox from all desktops takes place.

The rest of the features like GP and profile specific user configuration things are less important when the browser can't be installed to begin with.

The browser cannot be installed as there isn't enough manpower to visit each machine to do a manual install, and visit each machine when an update comes out.

I guess the ability to do an install of Firefox will be possible as soon as Microsoft finishes the aquisition of Softricity, so at least there is light at the end of the tunnel. I expect Microsoft will announce pricing on the new rebranded product in about 1 year, with a release of the software soon afterwards.
(In reply to comment #132)

> Agreed. The problem is that they don't just want MSI. In fact, most IT shops
> that require MSI also require the sort of functionality that's provided by the
> CCK: group policies, ability to pre-populate profiles, restricted IP ranges,
> special proxies for internet use tracking, etc, etc.
>
> This bug hasn't been WONTFIX'd because, well, we think it's worth fixing. That
> said, the value isn't worth blocking a release, nor has it gotten to be a high
> point on anyone's radar. Perhaps the IBMers working on CCK would like to take
> it or something like it. Perhaps you would like to start building a list of 
> the features that IT departments would truly require, prioritizing the 
> buglist, and driving the development of the feature.
>
> Own it, baby, own it. :)

I agree with you there. As much as I would like to see it in Firefox 2.0, until there is an established plan for MSI support it wouldn't make sense to block an entire release at this stage for a feature that hasn't even entered initial development.

However, if this project gets off the ground in time - who knows? 

* What technology will be used to build the MSI, and how will it be integrated into Firefox's build process? Would the open source WiX project be suitable? How do we get the build process ready for MSI development, and ensure it doesn't block any releases? How do we get the MSI project off the ground in relation to the overall build?

* The software update portion of Firefox will likely need modifications to support patches to MSI modules. Do we redeploy 

* Is having a depdencency on the Windows Installer API acceptable for all Windows Firefox releases? How will down-level clients be handled? I imagine using an older release (v1 or v2) of MSI would mean a larger number of clients would have it installed, but what happens in the event they don't have the required Windodws components? Package the MSI redistributable with Firefox? One option would be to ship with some sort of binary bootstrapper that would handle the redistributable detection/install/update before the MSI itself is launched and download it on demand. Another option would be to have to Windows distributions - one of legacy (95,98,etc) and one for the newer OSes that ship with recent versions of Windows Installer.

* Windows Vista will include new technology for Windows Installer. One of the biggest changes is that it is possible to patch software without being logged in as administrator, provided that the authenticode certificate on the patch matches the one on the original MSI package. Windows Vista will ship with LUA (limited user accounts) by default - switching to MSI now will give Firefox a huge advantage when it comes to automated patching later. The nightmare scenario will be when Vista comes out and the existing Firefox updater is unable to modify files in the Program Files directory. Firefox's credibility will be damaged - many computers will not be automatically patched when vulnerabilities occur. Technically this is a problem right now, but most home users run with accounts that have administrator rights.

* Any policy/GPO related tasks should be done afterwards - there's no point implementing them before the MSI itself, or blocking the MSI due to lack of group policy support. One solution for group policy would be to have a standard "Firefox" key in the policy section of the registry, with a named value matching the name of a Firefox configuration option. Want to have a policy to change the homepage? Create a key called "browser.startup.homepage", matching the existing Firefox preference name. An ADM template could be developed, similar to the IE one, that include the typical settings an administrator would want to set.

Any additional policy support (ie/ the IP range blocking that was mentioned) would be done by adding new Firefox preferences. The GPO support in Firefox would consist of a module to load preferences or enforced policies from the registry and being able to "lock" preferences from being changed. 

I don't know enough about Firefox development to "own" this - but how do we get started? Get buyin from the people in charge of the build process and automatic update module? 
(In reply to comment #134)

> * What technology will be used to build the MSI, and how will it be integrated
> into Firefox's build process? Would the open source WiX project be suitable?
> How do we get the build process ready for MSI development, and ensure it
> doesn't block any releases? How do we get the MSI project off the ground in
> relation to the overall build?

WiX is the way I would suggest to go.  I have been looking at it and would like to work towards a WiX script which other IT admins can then take to compile their own installs.  Until Mozilla release an MSI themselves, this is as far as I think anyone should take it.

> * The software update portion of Firefox will likely need modifications to
> support patches to MSI modules. Do we redeploy

Minor issue.  Firefox is small enough to remove and reinstall without affecting users too much

> * Is having a depdencency on the Windows Installer API acceptable for all
> Windows Firefox releases? How will down-level clients be handled? I imagine
> using an older release (v1 or v2) of MSI would mean a larger number of
> clients would have it installed...

A non-issue.  Microsoft Installer databases tend to be backwards compatible.  Newer MSIs simply feature more tables which it gets information from.  If the MSI engine on a machine is, say, Version 2, it simply doesn't support functionality introduced in Versions 3, 3.1 and forthcoming in 4 (such as the patching technology you mentioned).

It would make sense to include the MSI engine redist until support for Firefox for Windows 98 and the like still exist.  However, if they are talking about dropping support for those older versions of Windows, this isn't necessary.

> * Any policy/GPO related tasks should be done afterwards - there's no point
> implementing them before the MSI itself, or blocking the MSI due to lack of
> group policy support. One solution for group policy would be to have a standard
> "Firefox" key in the policy section of the registry, with a named value
> matching the name of a Firefox configuration option. Want to have a policy to
> change the homepage? Create a key called "browser.startup.homepage", matching
> the existing Firefox preference name. An ADM template could be developed,
> similar to the IE one, that include the typical settings an administrator would
> want to set.

This already exists, almost exactly as you describe it, as a proof of concept extension I wrote.  Can be found here:  http://homepages.ed.ac.uk/mcs/GPOforFirefox.zip

Any value set at HKLM\Policies\ADM\Firefox will be set as default and changeable, any at HKLM\Policies\ADM\Firefox\Locked, will be set and locked, and values can be set in HKCU at the same place.  This is a very small, easy to implement bit of code I will look to implementing as a patch to Firefox.  Since Firefox 1.5 added in the windows-registry-key interface, you can enumerate the Firefox registry key in the Policies area and then convert them easily to pref and lockpref.

However, saying all this, I kind of agree with what Mike Conner said.  There has to be some fundamental changes to the way Mozilla Corp do things before Firefox can be completely suitable for the enterprise.  A simple example is End of Life for major versions of Firefox need to be sensible and projected much further in the future.  When you consider many educational establishments are only able to update major versions of their apps once a year (eg.  during summer, when lecturers are updating course material for the coming academic year), they would need every major version (such as Firefox 1.0.x) to be supported and patched for 2 years minimum (up to a year until deploy + a year in operation).  There are also some other, less palatable, areas such as Mozilla giving some voice in future direction to those who deploy Firefox on a major scale.  For example, it would be a big risk if a major international bank was to base a mission-critical application on Firefox only to find the functionality its based upon is removed or changed in a new version.

Finally, I do believe Mozilla needs to take a lead on policy here.  There are some dangerous precedents being set.  Whilst Frontmotion have proven themselves to be trustworthy, a reliance on third-party MSIs is dangerous.  It really leaves the door open for another third party to produce an MSI based on the fact that third party MSIs are OK, and then slip in some malicious software (and trust me, the irony that I am saying this whilst pointing to an extension hosted somewhere other than Mozilla's site is not lost on me!).
> Another option would be to have to Windows
> distributions - one of legacy (95,98,etc) and one for the newer OSes that ship
> with recent versions of Windows Installer.

FWIW, that becomes less of an issue on trunk, where Win9x won't be supported anyway.
Should this be blocked by:

Bugzilla Bug 289370
Firefox updates should only show up in Add/Remove programs if Show Updates is checked
(In reply to comment #135)
> However, saying all this, I kind of agree with what Mike Conner said.  There
> has to be some fundamental changes to the way Mozilla Corp do things before
> Firefox can be completely suitable for the enterprise.  A simple example is End
> of Life for major versions of Firefox need to be sensible and projected much
> further in the future.  When you consider many educational establishments are

So now we're getting into why MoCo says that they don't want to sign up for "enterprise support". The strain on QA and Release resources is already near the breaking point, and extending our lifecycle on apps will just make that worse. Classically, enterprise vendors make their money on lifecycle support, selling that support for dollars. MoCo doesn't do that, and the free-software model doesn't scale when you have customers saying "well, but we want you to support it on a lifecycle basis that fits our corporate policies/IT refresh structures."

This is kind of where I was going in my previous comment: is producing an MSI useful when the project itself can't make the sort of promises that enterprise consumers might require? Is it just the MSI that's required, or are there other strings? What are the full costs, and the full expected return?

> Finally, I do believe Mozilla needs to take a lead on policy here.  There are
> some dangerous precedents being set.  Whilst Frontmotion have proven 

Mozilla needs to weight the potential benefits against the potential losses and figure out what's best for the future survival of the project. So far, every time that analysis has been made, the costs have been too high.
(In reply to comment #138)
> This is kind of where I was going in my previous comment: is producing an MSI
> useful when the project itself can't make the sort of promises that enterprise
> consumers might require? Is it just the MSI that's required, or are there other
> strings?

Yes, an MSI is useful on it's own. We currently deploy Firefox in this manner with no additional modifications, changes or requirements using Frontmotion's MSI.

Other features are nice, but are not required to do a large scale deployment.

Using a slippery slope argument of what might be desired by some people in the future is not good reasoning.
(In reply to comment #139)
> Using a slippery slope argument of what might be desired by some people in the
> future is not good reasoning.

You misinterpreted the intent behind my questions. Comment 135 asserted that simply generating an MSI wouldn't necessarily be useful for large scale deployments. I was asking if that assertion was true. I've been consistent in saying that if it's low effort, building an MSI would be a good thing to do. But the cost of the effort needs to be balanced against the usefulness of the MSI towards its ultimate purpose of enabling large scale rollouts.
(In reply to comment #140)
> You misinterpreted the intent behind my questions. Comment 135 asserted that
> simply generating an MSI wouldn't necessarily be useful for large scale
> deployments. I was asking if that assertion was true. I've been consistent in
> saying that if it's low effort, building an MSI would be a good thing to do.
> But the cost of the effort needs to be balanced against the usefulness of the
> MSI towards its ultimate purpose of enabling large scale rollouts.

While certainly not perfect for those that want to be able to force settings such as proxies and privacy options its certainly a good start and makes enterprise deployment much easier especially for security updates where you can in 2 minutes using Group Policy ensure that everyone will be updated to the latest security update. Prior to using the 3rd party MSI out organisation had everything from v1.0 right through to 1.5.0.x and just about every version in between
(In reply to comment #140)
> You misinterpreted the intent behind my questions. Comment 135 asserted that
> simply generating an MSI wouldn't necessarily be useful for large scale
> deployments. I was asking if that assertion was true. I've been consistent in
> saying that if it's low effort, building an MSI would be a good thing to do.

There's no definitive answer to that because of the range of scenarios.  However, as a guide, it really has to do with numbers, and that bearing on the TCO of Firefox within an enterprise environment.

If a company has 10 computers, they don't need an MSI.  They just have someone go round installing it.
If a company has tens to hundreds of computers, then could well need an MSI only.  If they then need to change a preference, they can then email out to everyone and ask them to do it manually, and then pick up the stragglers through user support.
Once you get into hundreds, thousands and tens of thousands of computers, that doesn't become practical, and you need to be able to both control the version and the preferences.  At this point, you need the ability to centralise the preference control.

Obviously, the above is no rule and I'm sure there are many exceptions depending on their scenario, but that generally fits.

So, if MoCo wanted to go after the SMB market, creating just an MSI would be certainly enough.  Much of what I said in comment 135 was targetted towards the larger enterprise (although, in any market size, I stand by what I said about EOL lengths).  I would also add that it is very much more likely that a larger organisation will have resident experts in repackaging applications as MSIs.  Therefore, it seems to me that the MSI should take the precedence in terms of importance.  Leave the GPO support issue in #267888 (which this call blocks).
This is an idea whose time has come. IE7 has been released, but large enterprise IT departments cannot deploy it as a safer/better general browsing tool for their users because of numerous internal applications that are incompatible with anything but IE6.

Unlike IE7, Firefox runs happily side-by-side with IE6, giving corporate IT the best of both worlds: a stable, safe, modern web browser for general browsing and use on newer, more egalitarian systems, while retaining IE6 on the desktop for intranet applications that were poorly-developed and could not be fixed in time for IE7.

But to do this, they need deployable packages. Reduce the friction of installation and management and they may give Firefox the time of day.
They have MSI builds at http://www.frontmotion.com/Firefox/index.htm

The site will actually build an MSI package to your specifications.  

To any other commenters, this isn't a question of whether this should happen, its a question of when it will happen.  Someone just needs to step up and do it.
Attached patch a more complete msi installer (obsolete) — Splinter Review
It's integrated as far as I can get it.
This requires MakeMSI
To build
cd to objectdir/browser/installer/windows
make msiinstaller
then open a windows command prompt in objectdir/browser/installer/windows/msigen and type mm firefox.mm P
Attachment #172077 - Attachment is obsolete: true
Attachment #172078 - Attachment is obsolete: true
Attachment #175803 - Attachment is obsolete: true
Attachment #278253 - Flags: review?(robert.bugzilla)
Comment on attachment 278253 [details] [diff] [review]
a more complete msi installer

Some notes and questions.
How are these strings localized?
On the trunk we only support Win2K and above.
application/x-xpinstall;app=firefox never worked and has been removed.
gopher shell integration has been removed.
-osint has been added to the open command for shell integration.
There are additional dirs under the appdir than the ones in this patch.
Last I checked we cannot distribute flash legally.
When attaching patches don't include diffs of binary files. Instead add them as separate attachments.
Attachment #278253 - Flags: review?(robert.bugzilla) → review-
Strings can be localized via an tab delimited file.  Support is easy, but there are no translations.  The MSI can be chaned to support Win2K and above installs easly.  xpinstall regkey will be removed. ditto gopher. -osint will be added. additional dirs?  It only adds localized and nonlocalzied.  Those other lines are there for testing.  It doesn't add flash, that's commented out.  how will people know where to put those binary files?
Is it possible for this file to be generated so that it is
1) localizable
2) picking up other build options such as product name
3) picking up other build options such as built extentions
4) whatever other changes that are going into the current installer

That way we only have to make changes in one place.
MAKEMSI is based on the PPWIZARD preprocessor (http://dennisbareis.com/ppwizard/introductiontoppwizardhtmlpreprocessor.htm), it can therefore execute external commands, parse results, read files or environment variables etc.

I know nothing about where this info is so can't really say more, just it can be done...

make -C @obj-XR@/xulrunner/installer installer
with this patch will create a working 'xulrunner-$version.$lang.win32.msi' in @obj-XR@/dist and '*.msm' in @obj-XR@/xulrunner/installer/windows

I understand, that this patch is only worth r-. However, it already demonstrates certain cool features:
* it is dynamically generated;
* w/out a single UI line of code, it will ask LUA for privilege escalation. it will install in per-user mode, if no admin password provided;
* .msm module can be reused in other applications. This is fascinating: I have been able to install-remove stand-alone xulrunner, firefox-on-xulrunner (only files) and a custom XR app (with a desktop shortcut) in any order w/out breaking anything. Each of them contains xulrunner module with system-wide registration.
Attachment #299877 - Flags: review?
Attached patch WiX-based XULRunner msi/msm (obsolete) — Splinter Review
With this patch, 'make installer' produces xulrunner.msm and a cfg file for it, which can be included in wix configuration for applications.

WIX_FLAGS and MSI_EXTRA makefile variables allow to add dependencies to application's main wix file. After building msm for xulrunner-1.9b3 with this patch, I was able to create msi packages for my application with one command per package:
make -f check.mk installer
make -f check.mk installer AB_CD=ru

Application configuration is available here:
http://repo.or.cz/w/abstract.git?a=tree;f=installer/windows
http://repo.or.cz/w/abstract.git?a=blob;f=check.mk
Attachment #299877 - Attachment is obsolete: true
Attachment #306248 - Flags: review?
Attachment #299877 - Flags: review?
Flags: blocking-firefox3?
This will not block the final release of Firefox 3.
Flags: blocking-firefox3? → blocking-firefox3-
Your code looks interesting Sergey, however I am a Jr Programmer and I'm having some problems running it. Can you please elaborate on what how to run this script (ie. What requirements do I need, What install dir).

Thanks
You'll need:
* msys build environment for mozilla (mozilla-build);
* wix 2.0 from http://wix.sourceforge.net/releases/2.0.5805.0/wix2-binaries.zip;

To create msm/cfg/msi:
1. extract wix zip somewhere (/c/wix in this example)
    mkdir /c/wix
    cd /c/wix
    unzip -x /path/to/wix2-binaries.zip
2. add wix directory to your $PATH:
    export PATH="$PATH:/c/wix"
3. apply my patch to mozilla tree you want to pack (works with 1.9b4 and later):
    cd /path/to/mozilla
    patch -p1 < /path/to/patch
4. build mozilla

5. cd to objdir, if any.

6. create xulrunner msm/cfg/msi:
    make installer

This sequence will produce xulrunner.msi in dist/ and xulrunner.msm in xulrunner/installer/windows/. The MSM will have a companion CFG file, which can be used to package xulrunner with XUL applications. I use it to package my Abstract project. The Makefile is here:
http://repo.or.cz/w/abstract.git?a=blob;f=installer/windows/Makefile.in 
Attached patch WiX-based XULRunner msi/msm v3 (obsolete) — Splinter Review
This patch fixes a small bug and further improves packaging process.

With the previous patch, 'make' was triggering msi package creation. This is now fixed.

Locale-specific files are now packed into a separate MSM module, which allow to create MST transform packages later.
Attachment #306248 - Attachment is obsolete: true
Attachment #313598 - Flags: review?
Attachment #306248 - Flags: review?
My day of programming are just about at an end.  
I would really apreciate it if you could make an ADDON for all parts of the mozilla family.  Then I would be able to try it and see if it might fix my broken Thunderbird! See bug 450421 .
Version: unspecified → Trunk
Comment on attachment 313598 [details] [diff] [review]
WiX-based XULRunner msi/msm v3

The patch is quiet a bit old but can you have a look at this Robert?
Attachment #313598 - Flags: review? → review?(robert.bugzilla)
Comment on attachment 313598 [details] [diff] [review]
WiX-based XULRunner msi/msm v3

It is a good start... as stated before in comment #146... how will this be localized?
Attachment #313598 - Flags: review?(robert.bugzilla) → review-
(In reply to comment #158)
> It is a good start... as stated before in comment #146... how will this be
> localized?

Looks like there are .wxl files which group all needed strings:
http://wix.mindcapers.com/wiki/Localization

The question was in relation to our localization and our build system. This currently requires the use of .properties or .dtd files. I had to do this when implementing the NSIS installer and this would need to be done for MSI.
(In reply to comment #157)
> The patch is quiet a bit old but...

The patch worked at about fiefox3 beta5, and it contains build-system-only changes. AFAICT, it should be working with 3.0.0.x branch without major changes.

(In reply to comment #157)
> how will this be localized?

I am using the patch from attachment 313598 [details] [diff] [review] to produce MSM for non-localizable part of libxul. MSM is later used to package my XUL app for 2 locales en_US and ru. If both are installed, the libxul is shared. Anyone interested may check at http://repo.or.cz/w/abstract.git?a=tree;f=installer/windows for the build-system code on the app side.

I am really interested in getting this bug resolved.
(In reply to comment #161)
>...
> (In reply to comment #157)
> > how will this be localized?
> 
> I am using the patch from attachment 313598 [details] [diff] [review] to produce MSM for non-localizable
> part of libxul. MSM is later used to package my XUL app for 2 locales en_US and
> ru. If both are installed, the libxul is shared. Anyone interested may check at
> http://repo.or.cz/w/abstract.git?a=tree;f=installer/windows for the
> build-system code on the app side.
> 
> I am really interested in getting this bug resolved.
The localization will either need to be done with .properties or .dtd files unless Axel (l10n at mozilla dot com) agrees to a new format. To get this resolved the MSI's will need to be localized and the patch will need to include how this will be accomplished.
(In reply to comment #162)
> The localization will either need to be done with .properties or .dtd files
> unless Axel (l10n at mozilla dot com) agrees to a new format. To get this
> resolved the MSI's will need to be localized and the patch will need to include
> how this will be accomplished.

A few points:
1. .properties and .dtd files are usually jarred into $(AB_CD).{jar|manifest} pair.

2. libxul produces such a pair, and I am also using such a pair in my application. The names (en_US.*) are the same but locations differ.

3. The packaging workflow of my app looks like this:
* build libxul with AB_CD=en_US;
* package into MSM:
** localization-independent part of libxul (1);
** en_US part of libxul (2);
* build my app with AB_CD=en_US;
* package into MSM:
** localization-independent of my app (3);
** en_US part of my app (4);
* build my app with AB_CD=ru;
* package ru part of my app into MSM (5);
* assemble MSI for en_US from (1), (2), (3), (4);
* assemble MSI for ru from (1), (2), (3), (5).

I am not using any language specific symbols from libxul, so (2) is fine for the ru locale of my app. This most likely isn't the case for firefox, and this is the only part that remains to be done.
I'm rather clueless about what the UE is here, but we're not going to go with a new localizable format. If we need wxl files, we should preprocess a DTD file into the wxl file. As we're talking plain xml, AFAICT, this is really just using the preprocessor to include a DTD with the localizable strings into a generic xml. That's hoping that the xml parser that reads the wxl files groks that. Otherwise, processing a .properties is probably easier.
I might be completely wrong here, but I think you guys are talking apples and oranges:

Sergey, Robert wants the text used in the installer UI to be localized using DTD or .properties files. He is not talking about the text in the application or in the libxul runtime.

(or am I wrong?)
(In reply to comment #165)
> I might be completely wrong here, but I think you guys are talking apples and
> oranges:

Mark, thanks for the comment.

There should be no changes to the application localizable symbols, so we are talking about installer symbols here, right?

If so, the installer is the only consumer of those symbols, right?

And if so, it may make sense to put the symbols directly into installer source file (that is .wxl). Or am I missing something?
We're not going to expose wxl files to our localizers. Whatever you do has to work on either .properties or .dtd files.
... which means, that's what you get at build time. You can do build-time preprocessing to other formats allright, to be used in the actual packaged installer. See comment 164.
note: App Update will also likely need work in that either:

a: the MSI App Update mechanism will need to be implemented for all versions of Windows if possible... this is preferred over b.

b: our existing App Update will need to taught to update the registry entries and anything else required by MSI.
IT IS POSSIBLE, and pretty easy to use DTD's to translate messages the installer provides!

I just managed to easily build an MSI using the latest stable version of WiX, an old wxs file I used for another project once, and a hacked en-US file, which included a DTD file where I actually defined a few entities (for Next and Back buttons).

However, there is a little difference between labels in WiX and XUL, and that difference is that in WiX, you have to define the accesskey within the label, as in "&amp;Next". 

BTW, *.wxl files are meant for localization of WiX, so I don't really think allowing localizers to work on that would be a huge risk. However, when using a DTD, the installer would not even compile if some entity is missing. Plus, the format of the file to localize would also be familiar for the localizers. So using DTD's makes sense too.
(In reply to comment #164)
IT IS POSSIBLE, and pretty easy to use DTD's to translate messages the installer provides!

I just managed to easily build an MSI using the latest stable version of WiX, an old wxs file I used for another project once, and a hacked en-US file, which included a DTD file where I actually defined a few entities (for Next and Back buttons).

However, there is a little difference between labels in WiX and XUL, and that difference is that in WiX, you have to define the accesskey within the label, as in "&amp;Next". 

BTW, *.wxl files are meant for localization of WiX, so I don't really think allowing localizers to work on that would be a huge risk. However, when using a DTD, the installer would not even compile if some entity is missing. Plus, the format of the file to localize would also be familiar for the localizers. So using DTD's makes sense too.
Whoops, sorry for double-committing... :(
Rimas, do you have an URL which gives an example or can you even supply one? I believe that it will help a lot. So probably someone could do a similar thing for Firefox and Thunderbird installers.
Well, actually, I have deleted those files after proving to myself it was possible, because I worked it out with version 2 of WiX, but Wix 3 (currently at pre-release state) didn't like the installer source at all, and I eventually decided not to waste my time... 

Anyway, what I did was:
* create a DTD file that defines a few entities;
* create a WXL file which references a DTD file and uses entities defined in it;
* compile a MSI file using the WXL file as a localization source.


Anyway, I have already tried to reference the DTD and use its entities in the same way in the WXS file (With WiX 3), and it doesn't work. I'm not sure if it's a bug or intended behavior, but when candle.exe transforms the WXS file to WIXOBJ file, it escapes ampersands, so &installerString; becomes &amp;installerString; (meanwhile &amp; remains untouched). I'll try to play with the WXL file again, and if that works out, I'll post zipped results here.
Here's an example that works with WiX 3. As you can see, the main hackery has been done in loc.wxl.  I couldn't find a WixLocalization DTD, so I just recreated the subset that I needed of it.

To compile the installer, just edit paths in build.bat and loc.wxl, then run build.bat.

The only problem is that I referenced DTD's by full path. That's lame, and should be easy to fix. One of the most obvious ways to fix it is to define a variable containing the path required, and then use it as a base. Also, I'm not sure what exactly the -cultures switch changes in light.exe (assuming that we translate all installer strings ourselves).
Blocks: 474088
Blocks: 274624
This bug has now been around for quite some time and Rimas is very close to a solution. Question is what will happen after we're done creating build scripts?

Will we start delivering out of cycle or wait for Firefox.next? Who would responsible for installing the scripts on the server? Would we provide MSIs for nightly builds?

Personally, I think we should deliver these ASAP and along the current installer, which is still the best solution for normal consumer installations. Especially European customers are literally crying about not being able use Firefox in corporate and equally important educational environments (at least that I can confirm from personal experience).

I'll look into setting up a clean build script tonight, using the normal installer DTDs for localization (I think I should find any necessary definitions in there, so that we don't need any additional definitions).

As far as I know the update mechanism by now is able to identify locked-down environments and skip updates, so there shouldn't be any problem there. The only remaining issue would be configurability via the registry (I'll file a separate bug on that), but I think we can safely provide that lateron. Also, it might be a good idea to provide update packages as well, but we'll see about that later.
I'd love to provide these from Mozilla but there are still several questions / issues that will need to be addressed such as comment #169 before this is provided by Mozilla. Also note that providing an MSI for Firefox has been given priority for Firefox.next.

(In reply to comment #176)
> This bug has now been around for quite some time and Rimas is very close to a
> solution. Question is what will happen after we're done creating build scripts?
Besides the build tools which would added to MozillaBuild the scripts should only need to be checked in.

> Will we start delivering out of cycle or wait for Firefox.next?
It will likely be part of Firefox.next.

> Who would responsible for installing the scripts on the server?
The scripts would typically be checked in as mentioned previously.

> Would we provide MSIs for nightly builds?
Typically an MSI app has an ID which would prevent installing a nightly, beta, or official (for that matter multiple official releases) side by side on the same system. Some options are having a separate ID for nightly, beta, and official which would solve this problem for the most part but not for every case. I think this question is better answered after how updates are going to work has been figured out.
Just in case lets put it on the radar.
Flags: blocking-firefox3.6?
Thanks.

About the updates: Without any knowledge of AD, I'd say that the issue would probably mostly go away when group policy guidelines are implemented. When admins can enable/disable the Firefox internal update mechanism, they can either allow the application to update itself and be done with it, but with less control or they can disable it and push the updates themselves. Firefox would just count as a single application, no matter what version, and the MSI would completely overwrite any existing installation. That way there's no conflict when an admin allows automatic updates.
MSI's keep track of the files an app has installed and our app update mechanism does not update the files installed by an MSI which will break the MSI installation.
Dang. So we're stuck with disabling autoupdate anyway. Would we need to do anything but setting app.update.auto false on the default install for that? Anyway, we're back at the group policies.
What I am planning on doing is figuring out how to make Win32's app update use msp's.
(In reply to comment #177)
> I think this question is better answered after how updates are going to
> work has been figured out.

As many people here have pointed out, the MSI architecture doesn't work correctly if files are patched without the MSI databases being updated - in other words, MSI files need MSI or MSP updates. 

I might be wrong, but my assumption would be that administrators would use MSI to initially deploy Firefox using an MSI installer then (in most cases) rely on Firefox itself for automatic security patches. It would be similar to how other software is deployed - much of the time IT just wants to get the software out there and usable to a large number of machines, not have to manually redeploy new versions as each security update gets released.

The v6.X versions of Windows can assist a bit here - signed MSI files can have signed MSP patches applied to them by standard users unless the system administrator disables the behavior via group policy.

Dual installers can be quite a bit of a pain - I remember dealing with a large number of headaches trying to deploy faulty VPN MSI packages when the "classic" version worked. Duplicating the installer means double the work at all levels - two installation packages to maintain, two patch channels/paths, twice as much testing, etc. In my experience what tends to happen is the MSI ends up being thought of as the "secondary" installer and is a bit of an afterthought - as a result, quality tends to suffer. This may no be the case with a Firefox MSI, but in every other case I've seen "dual installers" they're nothing but trouble.

MSI is a very stable technology - it might be worth considering switching Firefox entirely onto it for the Windows platform and have the 3.5->next upgrader migrate people onto an MSI based tech.

More testing up front (careful to mind the component rules and upgrade scenarios, etc) - but in the long run I think everyone wins. One set of code to maintain, standard user updates without UAC on Vista and Windows 7, and enterprise deployments. 

At work here we used a bootstrapper deployed via our legacy update system to launch into an MSI. All updates after that point are trivial MSP patches - I imagine something similar could be developed for Firefox.
Most of the time when a sysadmin deploys MSI packages with active directory, the users won't have rights to update Firefox themselves.  

So I think, that by default the MSI package should have the auto update feature turned off system wide.  

The option to enable the auto update should still exist on the feature selection screen for those who manually run the MSI (or use a transform).  Open Office and Java have similar capabilities.
(In reply to comment #183)
That is pretty well aligned with my thoughts on MSI. My main concerns being that:
a. localization for the MSI is the same as it is for Mozilla in that it uses either dtd or properties files.
b. update process is figured out as noted previously especially in regards to trunk, branch, other repo builds for nightly and beta as well as official builds.

(In reply to comment #184)
> Most of the time when a sysadmin deploys MSI packages with active directory,
> the users won't have rights to update Firefox themselves.  
This will most likely just be an option on install and / or via group policy
No longer blocks: 241532, 474088
No longer depends on: 229706
Depends on: 386740
No longer depends on: 302046
(In reply to comment #185)
> (In reply to comment #183)
> That is pretty well aligned with my thoughts on MSI. My main concerns being
> that:
> a. localization for the MSI is the same as it is for Mozilla in that it uses
> either dtd or properties files.

Possible, see attachment 352091 [details]. I think by adding a few new environment variables at compile time, we should even be able to get rid of this line:
<!ENTITY installerCulture "en-us">
in installer.dtd

> b. update process is figured out as noted previously especially in regards to
> trunk, branch, other repo builds for nightly and beta as well as official
> builds.

I think you pretty much covered it in Comment 177: the solution is having a separate ID for nightly, beta, and official.

And those who would want to have two or more released versions of Firefox at the same time, could simply use zip tarballs that Mozilla provides even now to install older versions.

> (In reply to comment #184)
> > Most of the time when a sysadmin deploys MSI packages with active directory,
> > the users won't have rights to update Firefox themselves.  
> This will most likely just be an option on install and / or via group policy

I hope it won't be disabled by default. An option at install time would be the best. Furthermore, I think we could just look at how default options are deployed in other applications that use MSI. I'm pretty sure there is a mechanism to do that.
Blocks: 52052
(In reply to comment #186)
> (In reply to comment #185)
> > (In reply to comment #183)
> > That is pretty well aligned with my thoughts on MSI. My main concerns being
> > that:
> > a. localization for the MSI is the same as it is for Mozilla in that it uses
> > either dtd or properties files.
> 
> Possible, see attachment 352091 [details]. I think by adding a few new environment
> variables at compile time, we should even be able to get rid of this line:
> <!ENTITY installerCulture "en-us">
> in installer.dtd
Yes, I've seen it and believe it is possible. The reason I raise it as a concern is that this is a firm requirement for localization.

> > b. update process is figured out as noted previously especially in regards to
> > trunk, branch, other repo builds for nightly and beta as well as official
> > builds.
> 
> I think you pretty much covered it in Comment 177: the solution is having a
> separate ID for nightly, beta, and official.
> 
> And those who would want to have two or more released versions of Firefox at
> the same time, could simply use zip tarballs that Mozilla provides even now to
> install older versions.
I see this as the fallback position if no better way is found to be able to do this.

> > (In reply to comment #184)
> > > Most of the time when a sysadmin deploys MSI packages with active directory,
> > > the users won't have rights to update Firefox themselves.  
> > This will most likely just be an option on install and / or via group policy
> 
> I hope it won't be disabled by default. An option at install time would be the
> best. Furthermore, I think we could just look at how default options are
> deployed in other applications that use MSI. I'm pretty sure there is a
> mechanism to do that.
The burden will be placed upon the admins distributing Firefox but we will make the burden very small.
(In reply to comment #187)
> > > (In reply to comment #184)
> > > > Most of the time when a sysadmin deploys MSI packages with active directory,
> > > > the users won't have rights to update Firefox themselves.  
> > > This will most likely just be an option on install and / or via group policy
> > 
> > I hope it won't be disabled by default. An option at install time would be the
> > best. Furthermore, I think we could just look at how default options are
> > deployed in other applications that use MSI. I'm pretty sure there is a
> > mechanism to do that.
> The burden will be placed upon the admins distributing Firefox but we will make
> the burden very small.

I agree with this.
If the users are not allowed to install firefox, they should also not be able to update FF.

That way the admins can decide when/if they wish to upgrade to a new version.
And just adding the new msi to the ADS and marking it as update for the old package ist the way it is intended to work.
(In reply to comment #184)
> Most of the time when a sysadmin deploys MSI packages with active directory,
> the users won't have rights to update Firefox themselves.  
> So I think, that by default the MSI package should have the auto update feature
> turned off system wide.  

Seconded. I am a sysadmin who deploys lots of MSI files onto a school network
using Group Policy, and this is exactly the case for us. Whenever a new version
of Flash/Shockwave/Java/QuickTime comes out, we deploy the new MSI manually,
with auto-updates switched off. Means that we have complete control of what's
installed on the network, and can do proper testing etc before the updates get
deployed.

I should also say (in reply to comment #169) that trying to update the registry
entries etc created by MSI is a very bad idea - if you get it even slightly
wrong you can cause a lot of problems. Best to stick to a complete
uninstall/reinstall for the time being, and then provide an option for MSPs at
a later date.
The default will be to leave auto update on since the MSI will be used by regular users as well. The admins deploying the MSI will be able to disable app update as noted in comment #187.

I'm aware that trying to update the registry entries is a very bad idea and plan on using .msp's for patching which will update the registry. Comment #169 was to note that this needed to be done and gave a couple of options on how to do it so anyone that supplied a patch for this bug realized that this is a requirement.
Just a reminder from older discussions (comment #134 and #135) an easy way to set the homepage is really necessary. That is a deal breaker in many organizations, including the one where I work...
(In reply to comment #188)
> I agree with this.
> If the users are not allowed to install firefox, they should also not be able
> to update FF.

IIRC, our update mechanism is disabled when the user you're running as doesn't have write access to the application directory anyway, so if Firefox is installed by an admin and run by a normal user, updates will not be attempted.
(In reply to comment #192)
> IIRC, our update mechanism is disabled when the user you're running as doesn't
> have write access to the application directory anyway, so if Firefox is
> installed by an admin and run by a normal user, updates will not be attempted.
With MSI's we will most likely be using MSP's for patching and one of the goals will be when the user doesn't have write access they will still be able to update on at least some platforms per comment #183 so this isn't applicable. We will provide the ability to turn it off per comment #187.
(In reply to comment #193)
> (In reply to comment #192)
> > IIRC, our update mechanism is disabled when the user you're running as doesn't
> > have write access to the application directory anyway, so if Firefox is
> > installed by an admin and run by a normal user, updates will not be attempted.
>
> With MSI's we will most likely be using MSP's for patching and one of the goals
> will be when the user doesn't have write access they will still be able to
> update on at least some platforms per comment #183 so this isn't applicable. We
> will provide the ability to turn it off per comment #187.

Most other apps distributed in MSI allow for altering the default behavior in cases like this (for example controlling the app's auto updates functionality) by changing the values of certain predefined properties in the property table of the MSI. The properties in the property table of the MSI can easily be altered by the admin by simply setting a different value on the msiexec command line when installing the package, or using a simple MSI editor to edit the MSI, or even more elegantly create a transform (MST) with these specific different settings set.

So if the Firefox MSI could have some AUTOMATIC_UPDATES=1 property defined in the property table of the MSI, coupled with having this property control some custom action, most admins who want to disable this functionality could do this by just changing the property value to 0.
(In reply to comment #194)
> So if the Firefox MSI could have some AUTOMATIC_UPDATES=1 property defined in
> the property table of the MSI, coupled with having this property control some
> custom action, most admins who want to disable this functionality could do this
> by just changing the property value to 0.

Or better yet, use app.update.auto=true like the setting in about:config.
(In reply to comment #195)
> (In reply to comment #194)
> > So if the Firefox MSI could have some AUTOMATIC_UPDATES=1 property defined in
> > the property table of the MSI, coupled with having this property control some
> > custom action, most admins who want to disable this functionality could do this
> > by just changing the property value to 0.
> 
> Or better yet, use app.update.auto=true like the setting in about:config.

Are you sure periods are allowed in MSI property names? I cannot find anything that would suggest that they are allowed. Furthermore, all standard MSI properties use uppercase and underscore, and so do the custom properties of all applications I have encountered. See for example http://wiki.services.openoffice.org/wiki/Documentation/How_Tos/Automatic_Installation_on_Windows#MSI_Properties.
(In reply to comment #196)
> Are you sure periods are allowed in MSI property names? I cannot find anything
> that would suggest that they are allowed. Furthermore, all standard MSI
> properties use uppercase and underscore, and so do the custom properties of all
> applications I have encountered. See for example
> http://wiki.services.openoffice.org/wiki/Documentation/How_Tos/Automatic_Installation_on_Windows#MSI_Properties.

You are correct about lowercase letters in public properties, but periods are allowed in all properties. More info:
http://msdn.microsoft.com/en-us/library/aa371245%28VS.85%29.aspx

How about:
APP.UPDATE.AUTO=1
(In reply to comment #197)
> You are correct about lowercase letters in public properties, but periods are
> allowed in all properties. More info:
> http://msdn.microsoft.com/en-us/library/aa371245%28VS.85%29.aspx
> How about:
> APP.UPDATE.AUTO=1

Given that each property would need to be manually programmed into the MSI, it might be better to go with a more standard MSI convention for property naming. 

I don't think usability will suffer - it should be fairly obvious to most admin's looking at the MSI property table. It might even reduce confusion, given that an MSI Properties != Firefox settings (in other words, I doubt it makes sense to program the MSI to support the arbitrary configuration of any Firefox setting at install time)
The last several comments are all bikeshedding[0], and I'm sure all their contributions would easily and trivially have been figured out in the work of writing a patch (or can be made after a patch manifesting the problem is actually provided).  Please don't comment unless you have a meaningful technical contribution -- anybody can argue naming, and naming that's forbidden is an especially easy problem to understand and fix.

0. http://www.bikeshed.com/
Rob: how far away are we on this? Not gonna hit 1.9.2, and I'm not sure it makes sense to aim for 1.9.3 but wanted a rough sense of feasibility.
blocking2.0: --- → ?
Flags: blocking-firefox3.6? → blocking-firefox3.6-
With the short cycles for 1.9.2 and 1.9.3 I highly suspect this won't make it until after 1.9.3 especially since this will affect app update as well.
Is there any way we could help to speed up this bug?

For corporate deployment is really needed, and I'm sure lots of sysadmins would install Firefox today in their companies if they just have to drag and drop the msi package ;)
I've been working on this a bit lately.  I've taken Sergey's patch and updated it to run on the Wix 3.0 toolchain and for Firefox instead of Xulrunner.  I'm also confident that we can solve the l10n concerns.  I haven't played with the app update stuff too much though.  I would post my work-in-progress, but the laptop I develop on is currently in another time zone for repairs :-(
Attached patch MSI with WiX WIP 1 (obsolete) — Splinter Review
This diff creates an MSI using WiX for Firefox.  It obviously has a long way to go, but it provides equivalent functionality to the packaged zip distribution (i.e., it runs).

Notes:

1. It's obviously missing stuff like Add/Remove Programs support, a real GUI, etc.
2. The component GUIDS that are generated are stable, so this sort of MSI can be used to build patches
3. I used heat (the WiX harvester utility that does the heavy lifting here) on installer-stage/nonlocalized and installer-stage/localized to build two separate include files.  This is fragile (you need to specify the directory structure manually, and use XSLT to rework the generated WiX files) and unnecessary (no real benefit of having separate ComponentGroups as far as I can tell) so I'll probably switch to using the result of make package in the next draft.
Assignee: nobody → me
Attached patch MSI with WiX WIP 2 (obsolete) — Splinter Review
Some cleanup over WIP 1, includes Add/Remove Programs Support that is branding aware.
Attachment #418915 - Attachment is obsolete: true
I've been working on some UI stuff for the installer and will have a much more thorough patch in a week or two.  In the meantime I did some tests on size.  I wrapped the MSI in a 7z self extracting executable as the regular installer is.  Obviously the MSI isn't quite fully functional, but the size shouldn't increase dramatically as features are added (especially since the MSI database is dwarfed by the actual files) though some size may be contributed by the branding images.

This is for a debug trunk build.

Type                                Size
ZIP                                 12.8 MB

NSIS Installer                      8.71 MB

MSI (Cabinet not embedded in MSI)   9.72 MB

I believe the difference is primarily due to the compression used (I didn't run the compression the scripts run, just used the 7zip desktop program with the defaults) because the NSIS installer executable is 472 KB and the MSI database is ~300 KB so far so the overhead is not bad yet (keep in mind this is sans UI in the MSI).
Hi Kyle, I appreciate you taking this on.

The main reasons that the NSIS installer is wrapped in a 7zip self extracting archive is to lower the file size and to make repackaging of localized builds simple. It isn't a requirement as long as these two requirements are met.

How are app updates going to be handled with MSI installations?

How are people going to be able to install multiple builds side by side?

Thanks
Robert,

Speaking as a network admin, I don't care about multiple builds side-by-side in 99% of my installs.  I also don't care about auto-updates.

The whole point of getting the MSI out there is so admins like me who manage 700 desktops can easily push out an identical copy of Firefox to every machine.

If someone needs some weird one-off setup with multiple builds, I can manage their machine separately.

I also don't want FF automatically downloading updates on hundreds of computers at the same time.  *I* will download the updated MSI from the Firefox website, test the update with a select group of people, and push it out on my schedule.

To be blunt, the main reason I am running Internet Explorer on 800 desktops instead of Firefox is because I can't easily deploy Firefox.  I can easily choose via Microsoft Software Update Services when to install updates to IE.

The second (and much lesser) reason is that I can't easily control it via Group Policy to do things like automatically set the home page, lock down plugins, etc...

I don't mean this to sound as harsh as it's probably coming across--but I'd rather have the MSI now even if it doesn't do everything the standard installer does than have to wait another 6 years.  (This bug will have it's sixth birthday in a few days.)

Thank you Kyle for tackling this.  When the MSI comes out, you will have a *lot* of admins that want to buy you a beer... ;)
Aaron, there have been MSI's available for Firefox for years now ( http://www.frontmotion.com/Firefox/ for example ) though they aren't made available by Mozilla due to the app update issue. If Mozilla publishes an MSI without the ability to update and regular users install it then many of them will end up stranded and I highly doubt that will be acceptable. To be blunt I would love it if we had an MSI distribution.
(In reply to comment #207)
> How are app updates going to be handled with MSI installations?
> 
> How are people going to be able to install multiple builds side by side?
> 
> Thanks

To be straight, I have no idea yet.  It's obviously a big project to tackle.  I haven't even taken a look at the app updater code.  My conceptual idea for that is simply to wrap the MSP files in the MAR format and then replace the updater helper executable's guts with "msiexec /p patch.msp".

As far as running multiple builds side by side, what exactly does that mean?  Are we talking about running say 3.5 and Minefield side by side, or running the en-US and AB-CD locales side by side on the same version, or running 3.5.1 and 3.5.7 side by side?  I have no idea if we support the last one, but the first two should not be particularly difficult as long as we keep the upgrade GUIDs managed well (as they are the equivalent of the "channels" that currently exist).  Achieving the third one with working updates could get tricky though, but I'm not sure if we support it and don't really see a use case for keeping an older minor version around.
Application updates can be turned off with a preference. A customized corporate build would contain this preference.

I doubt folks would use the off the shelf MSI.

And most people are aware of the FrontMotion MSIs, but they want official Mozilla MSIs.
What are the characteristics that they want in their MSIs that they would get from "official Mozilla MSIs" but don't from the FrontMotion ones?  Many of our users get their Firefox from contributed builds for OS/2 or Solaris or AIX, which are no more "official Mozilla" than the FrontMotion ones in any useful sense.  It would be helpful to talk about specific characteristics, rather than more general labels, since otherwise -- or even so! -- it may well be that MSIs we host for download won't provide what people are looking for.

And if people aren't going to use the off-the-shelf MSI, what does that mean about them being official?  If it's a matter of support, we provide that through SUMO for even explicitly non-official Firefox setups, like Iceweasel and such, if the problem seems to be a core issue and the user or their agent can help us determine that.  I don't see how we would provide any more support to someone who tweaked their MSI and got a failure that didn't show up with ours (or FrontMotion's, for that matter).

We do support running any number of builds side by side, be they the same major stream or not.  All of Firefox's application state is and should remain local to the installation area, so if you can install an MSI to an alternate path, then it should be fine.  If you can't, then people using MSIs for anything are inherently boned on this point for any application.

Whatever we do with GUIDs, I think it's important that users with MSI installations be able to update from 3.5.x to 3.6 via an MSI-friendly path, just as they would be able to from 3.5.x to 3.5.x+1.
Kyle, using different guids and managing the guid's for trunk, branch, release, etc. is along the lines of what I was thinking as well. It won't support everything that can be done today but I think it is close enough and nightly testers trying to track down a regression range could still use the zip builds. For updates we'll want to use msp's since updates can be applied by non-admin users as long as the cert for the install matches the cert for the update but that problem can be fixed separately after there is the ability to create MSI's but likely before we make MSI's available officially.
"What are the characteristics that they want in their MSIs that they would get
from "official Mozilla MSIs" but don't from the FrontMotion ones?" With the FrontMotion ones you have to trust two entities, with "official" ones only one. By the way: Would it even be possible to sign the MSIs?
(In reply to comment #214)
> By the way: Would it even be possible to sign the MSIs?

That's a prerequisite for non-admin patching on newer Windows versions which is one of the end goals for using an MSI installer, so that is possible and will be done.
Alias: MSI
As far as GUID management goes, WiX has evolved to the point where it can do 99% of the work automatically.  Pretty much everything in an MSI needs a GUID.  Component GUIDs (99% of the work) can be handled automatically by WiX.  They are generated by the SHA-1 hash GUID algorithm from the component path (version 5?) so they are unique and stable so that patching can work.

Really the only GUIDs we need to handle are the upgrade codes.  An upgrade code is roughly analogous to an "update channel" in Mozilla's system.  In my mind, the nightly builds will have one constant upgrade GUID and each major stable release will have its own upgrade GUID and each branch's tip will have its own upgrade GUID.  Of course these can be tweaked as necessary to fit our needs.

I think we can easily support everything we do today except run the same build side by side, which is still possible, just takes some more work and a dirty hack but it works.  That's difficult because the product code (which WiX will generate at packaging time on our end) needs to be unique on the user's machine (it's a bit more complex than that, but that's more or less how it works).  The canonical way to get around that is to use a bootstrapper that can generate a new product code on the fly if needed.

Shorter: the corporate use case is actually a lot simpler than the end-user use case.
This is the wish list from my employer, in order of importance:

1. Auto updates turned off. (A must since regular users lack privileges to do the update anyway.)

2. Ability to set default start page.

3. Customization of no.2, 4 and 5 must be painless and quick.

4. Setting some default bookmarks.

5. Setting some default extensions.
(In reply to comment #208)

> 99% of my installs.  I also don't care about auto-updates.
> 
> The whole point of getting the MSI out there is so admins like me who manage
> 700 desktops can easily push out an identical copy of Firefox to every machine.
> 
> If someone needs some weird one-off setup with multiple builds, I can manage
> their machine separately.
> 
> I also don't want FF automatically downloading updates on hundreds of computers
> at the same time.  *I* will download the updated MSI from the Firefox website,
> test the update with a select group of people, and push it out on my schedule.

Firefox has a very good trackrecord concerning updates if you ask me. This is because of those automatic updates.

So if you do want to do updates on your timeframe in your organisation.

How do you get notified about updates ?
I'd also like to be able to disable auto-updates in the MSI.

(In reply to comment #218)
> Firefox has a very good trackrecord concerning updates if you ask me. This is
> because of those automatic updates.

I'm not concerned with Mozilla's track record, I'm concerned with when the updates are run.  I have several school labs with machines that are "frozen" so they boot in the same state every time.  If Firefox were doing auto-updates, it would download and install the update every time someone started it.  With MSI updates, I could have the updates installed in the nightly maintenance window.

> How do you get notified about updates ?

I get notified about updates because the copy of Firefox on my machine doesn't have updates disabled.
Attached patch MSI with WiX WIP 3 (obsolete) — Splinter Review
Proof of concept for l10n of the installer.  We can run the simple perl script l10nconvert.pl on a .properties file to get the XML based format WiX needs for l10n of the installer UI.  Obviously the UI is about 1/10th baked but if you set the UI level to none or basic this should install fine.

I'm going to continue working on the UI (I'm cloning the NSIS UI, though I don't know if we have any particular attachment to that.  None of the built in WiX UI choices seem to fit our needs well, they all have lots of overkill in license agreement screens, feature trees, etc.) over the next few days and the weekend.
Attachment #419518 - Attachment is obsolete: true
(In reply to comment #220)
>...
> Proof of concept for l10n of the installer.  We can run the simple perl script
> l10nconvert.pl on a .properties file to get the XML based format WiX needs for
> l10n of the installer UI.  Obviously the UI is about 1/10th baked but if you
> set the UI level to none or basic this should install fine.
Is it possible to use dtd's instead? If not, new scripts should be written in python.

> I'm going to continue working on the UI (I'm cloning the NSIS UI, though I
> don't know if we have any particular attachment to that.  None of the built in
> WiX UI choices seem to fit our needs well, they all have lots of overkill in
> license agreement screens, feature trees, etc.) over the next few days and the
> weekend.
There is no attachment to the current UI but there is a desire to simplify the installer / lessen the number of pages where possible. For example, the first page the next button would read install which would start the install and there would be a customize button in the body of the page to allow customization.
(In reply to comment #221)
> (In reply to comment #220)
> >...
> > Proof of concept for l10n of the installer.  We can run the simple perl script
> > l10nconvert.pl on a .properties file to get the XML based format WiX needs for
> > l10n of the installer UI.  Obviously the UI is about 1/10th baked but if you
> > set the UI level to none or basic this should install fine.
> Is it possible to use dtd's instead?
use dtd's without preprocessing that is.
(In reply to comment #222)
> (In reply to comment #221)
> > (In reply to comment #220)
> > >...
> > > Proof of concept for l10n of the installer.  We can run the simple perl script
> > > l10nconvert.pl on a .properties file to get the XML based format WiX needs for
> > > l10n of the installer UI.  Obviously the UI is about 1/10th baked but if you
> > > set the UI level to none or basic this should install fine.
> > Is it possible to use dtd's instead?
> use dtd's without preprocessing that is.

I believe so.

I take it python is preferred to perl for the other scripts as well?
(In reply to comment #223)
> python is preferred to perl for the other scripts as well?

YES!
(In reply to comment #223)
>...
> I take it python is preferred to perl for the other scripts as well?
Yes but we typically don't convert them from perl unless the perl script requires significant changes.
Status: NEW → ASSIGNED
OK, understood.  I'll go back and convert the perl to python at some point.  I'm going to keep hacking away at this this weekend and see how far I get.  Using DTDs for l10n is definitely possible (with nothing funky on the DTD side) using Rimas's example.  I going to try to hack together at least a proof of concept for updating with MSPs or doing side by side installations (probably the former).  I'll post an updated WIP sometime next week with the results.
I've also been thinking a bit more about how to handle l10n repacks.  From what I read in the build scripts (please correct me if I'm wrong) the process goes something like:

1. Build en-US build appropriately
2. Package the en-US build into a .exe installer
3. For non-en-US locale ab-CD
   a. Open up the installer
   b. Rip out all the en-US components
   c. Insert all the ab-CD components (including the new setup.exe)
   d. Sew the installer back together into a single exe
4. Enjoy your new localized installer

The problem with doing it the current way is that there is no real way to accomplish step 3b.  This is because

1) Files in an MSI are stored in a cabinet file, which makes them difficult to remove and replace.
   a) It has no directory structure in an MSI (i.e. the directory structure is stored only in the MSI database and not in the cabinet file)
   b) It is inaccessible to the tools in our build infrastructure (7zip can read them but not write them, don't see any other tools that can either).
2) The final MSI database itself will contain localized information which is not easily changed, and regenerating the entire MSI from scratch each time would defeat the purpose of "repacking". 

The way that seems most natural to do this with an MSI is.

1'. Build en-US build appropriately
2'. Package the non-localized components into an MSI
3'. For a locale ab-CD (including en-US)
    a. Package the localized components into an Merge Module
    b. Merge the MSI produced in step 2 with the Merge Module
4'. Enjoy your new localized installer

We wouldn't be "repacking" per se because no unpacking would ever occur.  We would still avoid building the entire app from source for each locale though, which from my understanding is the goal.

Thoughts from Robert, Axel, et al.?
That sounds ideal... there is no requirement to unpack / repack and the simplest method that meets the same end results is best.
Attached file Python version of l10nconvert.pl (obsolete) —
I converted the l10nconvert.pl to Python for Kyle.
Attached file Python version of brandingconvert.pl (obsolete) —
I converted brandingconvert.pl to Python.

By the way, defines without values are not treated differently, i.e.
!define FLAG_WITHOUT_VALUE
would become
<?define FLAG_WITHOUT_VALUE= ?>
(In reply to comment #230)
> Created an attachment (id=420740) [details]
> Python version of brandingconvert.pl
> 
> I converted brandingconvert.pl to Python.
> 
> By the way, defines without values are not treated differently, i.e.
> !define FLAG_WITHOUT_VALUE
> would become
> <?define FLAG_WITHOUT_VALUE= ?>

Thanks Rene!  This was really useful.  I don't think we're doing to have defines without values involved here.  The script worked perfectly with a little makefile hackery to get around a bug in our Python interpreter (see http://mail.python.org/pipermail/python-bugs-list/2007-August/039259.html on why we have to cat the input)
(In reply to comment #227)
> 1'. Build en-US build appropriately
> 2'. Package the non-localized components into an MSI
> 3'. For a locale ab-CD (including en-US)
>     a. Package the localized components into an Merge Module
>     b. Merge the MSI produced in step 2 with the Merge Module
> 4'. Enjoy your new localized installer

Kyle, are you sure it is possible to merge MSI with a Merge Module (aka MSM)? When I last checked, it was required to have all components in MSM (wix 2.0).
Regarding approach to MSI design and i18n:

The way I see many third-party packages doing i18n lately is with a master MSI and then some number of MSTs (transforms).  The MSI contains all the language-independent stuff, and then the MST modifies that to include locale-specific stuff.  Sometimes English will be included in the MSI; sometimes English will be just another language (so you always need an MST).  Some front-end to the MSI process (or perhaps custom actions in the MSI itsekf; I haven't delved too deeply) chooses the MSI based on OS locale and/or user input.

If or how this could be integrated into the Firefox build process, I have no idea.  I'm mostly an IT admin; I just dabble in software design.  I've never used WiX.  :(

Regarding updates and maintenance:

The typical application for an MSI (assuming it isn't the only package format used) is a strongly managed environment.  Users don't have administrator rights on their computers.  Software is updated by the IT department, not by end-users.  This is akin to the user-vs-root distinction from *nix land.  Some tool manages installation and updates to deployed MSI.

This can be done with native Windows tools (i.e., you don't need a third-party product).  Active Directory lets you "Assign" an MSI via Group  Policy.  So the admin creates a GPO (Group Policy Object), links it to an OU (Organizational Unit) containing the computers, and then Assigns the MSI to the GPO.  The next time the target computers reboot, they see the GPO and install the MSI, before the user even logs on.  Updates are handled via the same user interface for GPO.  Windows recognizes that a newer MSI for the same product upgrades an older MSI, and handles it appropriately.

This is a *very* common scenario in a corporate environment.  I would dearly love to see an official Mozilla MSI build.  I give many thanks to FrontMotion for their work, but being a third-party thing, they'll unavoidably lag behind the official releases.
(In reply to comment #232)
> (In reply to comment #227)
> > 1'. Build en-US build appropriately
> > 2'. Package the non-localized components into an MSI
> > 3'. For a locale ab-CD (including en-US)
> >     a. Package the localized components into an Merge Module
> >     b. Merge the MSI produced in step 2 with the Merge Module
> > 4'. Enjoy your new localized installer
> 
> Kyle, are you sure it is possible to merge MSI with a Merge Module (aka MSM)?
> When I last checked, it was required to have all components in MSM (wix 2.0).

Sergey, WiX 3.0 provides an MSM equivalent called a wixlib which works around limitations like that at the cost of not being usable outside the WiX toolchain (which isn't a problem here since we consume the wixlibs at merge time)

(In reply to comment #233)
> Regarding approach to MSI design and i18n:
> 
> The way I see many third-party packages doing i18n lately is with a master MSI
> and then some number of MSTs (transforms).  The MSI contains all the
> language-independent stuff, and then the MST modifies that to include
> locale-specific stuff.  Sometimes English will be included in the MSI;
> sometimes English will be just another language (so you always need an MST). 
> Some front-end to the MSI process (or perhaps custom actions in the MSI itsekf;
> I haven't delved too deeply) chooses the MSI based on OS locale and/or user
> input.

Benjamin, this is a common design pattern but it produces a couple issues for us.  From my understanding, transforms can only be applied at installation time so this would entail shipping both en-US and ab-CD files/UI data for each ab-CD locale.  Also, the need to automatically choose the language based on OS locale (while a neat trick for say a DVD deployed application) isn't necessary for us.  Finally, the design of the WiX toolchain makes it difficult to construct an MST without first building the locale package anyways.

As for updates on my part, getting patching working took a lot more effort than I expected (mostly because WiX subscribes to the "code is the documentation" philosophy when it comes to patching).  I've been able to successfully generate a MSP patch (though not a binary patch, that crashed the toolchain but I'm not sure if that was an input problem or a bug in WiX yet).  I haven't waded into the updater yet though.

Also worth noting is that Windows Installer's UI capabilities leave quite a bit to be desired (one example quirk, Windows Installer recreates each dialog causing the taskbar entry to disappear and reappear).  I will post a sample MSI with the UI I've implemented later this week.
(In reply to comment #234)
> Also, the need to automatically choose the language based on OS locale
> (while a neat trick for say a DVD deployed application) isn't necessary for us.

Please no! I, and many other world travelers, may be "located" in a country, but our "language" preferences are still that of our country of origin (e.g., I am an American living in Germany. I prefer English software). Therefore, please don't determine the *language* from the *locale*;...

Determine the language from the OS's language!!!

Apples and apples, if you will. A lot of software gets this wrong. I constantly have to set Windows to location=Germany but Language=English. It's a PITA.

IOW: Don't confuse location with language.
Peter, read up on http://en.wikipedia.org/wiki/Locale, it's not location.

Not that I think that multi-locale is imminent as a installer feature. The one build we do right now installs the set of locales it's shipping with and selects at runtime.
I've hooked up the MSI patching system to the app updater.  If you want to try it out, I've made a build off of today's trunk at http://www.kylehuey.com/moz/msi/msiupdatetest/firefox-3.7a1pre.en-US.win32.msi (The installation UI is only about half done, so among other things, if you install it from the UI it'll go in [ProgramFilesFolder]/Minefield)

You'll need to change the update.xml location (app.update.url.override) to http://www.kylehuey.com/moz/msi/msiupdatetest/update.xml

Once you do that, if you check for updates you should automatically download and install a patch.  If you go look at your add/remove programs you should see the patch installed.
(In reply to comment #234)
> As for updates on my part, getting patching working took a lot more effort than
> I expected (mostly because WiX subscribes to the "code is the documentation"
> philosophy when it comes to patching).  I've been able to successfully generate
> a MSP patch (though not a binary patch, that crashed the toolchain but I'm not
> sure if that was an input problem or a bug in WiX yet).

This appears to be a bug in WiX.  The transform generating tool (torch) recognizes DLLs that have the same version but different timestamps as different files (though it can tell that two EXEs with different timestamps are the same).  The patching utility (pyro) then chokes because there is no actual difference between the files.
I've filed bugs to move some tangential work out of here, stuff relating to the UI and end-users mostly to focus on producing a working MSI with updates in this bug.
This bug is waiting on:

1) Implementing shortcuts
2) Implementing the necessary registry keys for shell integration
3) Writing tools/update-packaging scripts for MSIs.
4) Write patch template for Firefox. (The wxs file)

After reading MSDN some more we really don't need to worry about managing upgrade codes.  I think we can give every Firefox (Thunderbird, Seamonkey, etc) the same upgrade code because we manage the update side ourselves.  Upgrade codes also don't have to be unique per locale.

As for product codes, we want a differing product code for everything we want to be easily installable side by side (eventually we'll want everything to be installable both side by side and multiple times, but that's for Bug 543392)
That means we need product codes for:
- Trunk Nightlies (1)
- Branch Nightlies (1 per branch)
- Branch Milestones (1 per branch)
- Branch Stable Releases (1 per branch)
These also all need to be unique across locales to allow installing say the trunk en-US and trunk zn-TW side by side.

I think the best way to do this is with a python script that generates a UUID based on a string like "@MOZ_APP_NAME@/@MOZILLA_VERSION@/@AB_CD@".  That guarantees us uniqueness across all the things I listed above and doesn't require us to manually track (1+3*n supported branches)*m supported locales GUIDs.
(In reply to comment #240)
> These also all need to be unique across locales to allow installing say the
> trunk en-US and trunk zn-TW side by side.

IMO that's an overkill.
(In reply to comment #241)
> (In reply to comment #240)
> > These also all need to be unique across locales to allow installing say the
> > trunk en-US and trunk zn-TW side by side.
> 
> IMO that's an overkill.

Product codes are required to vary for other reasons too (http://msdn.microsoft.com/en-us/library/aa370854%28VS.85%29.aspx)

They also need to vary based on the target architecture x86 and x64 must have differing product codes.
(In reply to comment #240)
> This bug is waiting on:
> 
> 1) Implementing shortcuts
> 2) Implementing the necessary registry keys for shell integration
These two are being handled in the patch in Bug 547225.  Once that bug is fixed the necessary code on the MSI side is 3 or 4 lines of XML.

> 3) Writing tools/update-packaging scripts for MSIs.
Still working on.  This is tricky because of the various idiosyncrasies of MSI patching.

> 4) Write patch template for Firefox. (The wxs file)
This is done.

I think this will be ready for initial review by the end of next week.
(In reply to comment #243)
> I think this will be ready for initial review by the end of next week.

That's not going to happen, Bug 547225 turned out to be more involved than I thought.

(To myself so I remember this) Also, after thinking about this some more, I'm not sure that using the Heat tool from the WiX toolset is the best idea.  The main reason is the hackery that I currently have implemented to get the directory tree to work properly (which could break if two directories have the same name and one contains no l10n files and the other contains only l10n files), though there are some other issues related to features and patching.  It should be fairly straightforward to rewrite the guts of Heat in Python and have it process the package-manifest and removed-files files to generate the proper WiX XML for the compiler.  This has the upside of removing files through patching much easier.

That said, I will still probably have to patch WiX's patch utility since it crashes whenever you feed it two DLLs that are identical except for version (e.g. nightlies)
Nightlies will always generate different dll's even if the code doesn't change.
Current up to date summary of the planning for this is at https://wiki.mozilla.org/MSI
Gerv removed most of these files in Bug 324118, but there is some cruft remaining.  Let's clean that out before we get started.
Attachment #278253 - Attachment is obsolete: true
Attachment #313598 - Attachment is obsolete: true
Attachment #352091 - Attachment is obsolete: true
Attachment #420438 - Attachment is obsolete: true
Attachment #420705 - Attachment is obsolete: true
Attachment #420740 - Attachment is obsolete: true
Attachment #440037 - Flags: review?(jmathies)
We need to provide the branding information in a format that WiX can accept.  Depending on our distaste for duplication, we could store the branding info in the makefiles and preprocess template .nsi and .wxi files to avoid duplicating the info.
Attachment #440041 - Flags: review?(robert.bugzilla)
Attachment #440041 - Attachment is obsolete: true
Attachment #440041 - Flags: review?(robert.bugzilla)
We need a way to capture the self registration information from a DLL.  We can't hardcode this information in the WiX files because the self registration registry entries need to be part of the same component as the DLL they belong to, and the WiX markup for the actual files being installed is going to be generated at build time.  This small tool lets us capture the self registration information and examine it with a script to create the WiX markup later.  This is necessary for proper installation of AccessibleMarshall.dll
Attachment #440050 - Flags: review?(jmathies)
Attachment #440045 - Attachment is obsolete: true
Attachment #440045 - Flags: review?(robert.bugzilla)
Attachment #440037 - Flags: review?(jmathies) → review+
nits mostly - 

> +  unsigned long l;

Lets use something like dwRes here.

> +  SHDeleteKey(HKEY_CURRENT_USER, L"Software\\SelfRegCap");

Maybe "Software\\Mozilla\\SelfRegCap" so we get the blame if it's not cleaned
up? 

> +  RegOverridePredefKey(HKEY_CLASSES_ROOT, diversionKey);

Shouldn;t we check for failure here, otherwise we might register the dll proper
farther down.

> +    (*p)();

Do we care about the result?
(In reply to comment #251)
> nits mostly - 
> 
> > +  unsigned long l;
> 
> Lets use something like dwRes here.
> 
> > +  SHDeleteKey(HKEY_CURRENT_USER, L"Software\\SelfRegCap");
> 
> Maybe "Software\\Mozilla\\SelfRegCap" so we get the blame if it's not cleaned
> up? 
> 
> > +  RegOverridePredefKey(HKEY_CLASSES_ROOT, diversionKey);
> 
> Shouldn;t we check for failure here, otherwise we might register the dll proper
> farther down.
> 
> > +    (*p)();
> 
> Do we care about the result?

All good points, will fix.
(In reply to comment #252)
> > > +    (*p)();
> > 
> > Do we care about the result?
> 
> All good points, will fix.

I was thinking about this while grabbing some lunch. Seems like if the test reg fails, the install should fail because the dll likely isn't going to register. Sort of an early dependency test before we actually do the main registration.
(In reply to comment #253)
> (In reply to comment #252)
> > > > +    (*p)();
> > > 
> > > Do we care about the result?
> > 
> > All good points, will fix.
> 
> I was thinking about this while grabbing some lunch. Seems like if the test reg
> fails, the install should fail because the dll likely isn't going to register.
> Sort of an early dependency test before we actually do the main registration.

I agree, just want to point out that this is happening at build time, not at install time.  If the DLL fails to self-register we should just abort building the installer.  During install we're never going to actually self-register the DLL, we're going to let the MSI engine handle the registry entries that we captured (this is required for repair functionality and making the MSI component model work).
Attachment #440050 - Attachment is obsolete: true
Attachment #440050 - Flags: review?(jmathies)
(In reply to comment #254)
> I agree, just want to point out that this is happening at build time, not at
> install time.  If the DLL fails to self-register we should just abort building
> the installer.  

So over time we could end up with a very large number of dll's registered on the build machines ? If so, could we take some steps to unregister each dll after a successful registration in order to keep the registry trimmed down ?
(In reply to comment #255)
> (In reply to comment #254)
> > I agree, just want to point out that this is happening at build time, not at
> > install time.  If the DLL fails to self-register we should just abort building
> > the installer.  
> 
> So over time we could end up with a very large number of dll's registered on
> the build machines ? If so, could we take some steps to unregister each dll
> after a successful registration in order to keep the registry trimmed down ?

Yes.  The general plan is to do:

"selfregcap.exe foo.dll"
examine the diverted registry key and generate the appropriate WiX markup if any
"selfregcap.exe" (which will delete the diverted registry key and clear out all of the information)

Additionally I'm marking this registry key as volatile so it should not persist across reboots.
Third time is the charm.  The main point of interest here is whether we want to duplicate the branding information or if we should preprocess .nsi and .wxi templates with branding information stored in the makefile or something similar.
Attachment #440465 - Flags: review?(robert.bugzilla)
Trivial fix to comments in branding information which have an incorrect location.
Attachment #440469 - Flags: review?(robert.bugzilla)
Comment on attachment 440466 [details] [diff] [review]
Part 3: Capture the self-registration information from DLLs V1.1

if (S_OK != dwRes) {

I believe there are other return values we can get here that indicate success. MSDN says S_OK is the expected return value but if we have access to it, we might want to use the SUCCEEDED macro.
Attachment #440466 - Flags: review?(jmathies) → review+
(In reply to comment #260)
> (From update of attachment 440466 [details] [diff] [review])
> if (S_OK != dwRes) {
> 
> I believe there are other return values we can get here that indicate success.
> MSDN says S_OK is the expected return value but if we have access to it, we
> might want to use the SUCCEEDED macro.

Good point.  I've fixed this in my queue.
Comment on attachment 440466 [details] [diff] [review]
Part 3: Capture the self-registration information from DLLs V1.1

Not sure if the localization process will require localizers to also run this but if they do need to then the binary for this will likely need to be added to mozilla-build.
There's another drastic packaging change, omnijar in bug 552121. Not sure how these two interact.
(In reply to comment #263)
> There's another drastic packaging change, omnijar in bug 552121. Not sure how
> these two interact.

Can't really tell if they interact or not based on what's been written in the bug, but I will watch it.  Thanks for the heads up!

(In reply to comment #262)
> (From update of attachment 440466 [details] [diff] [review])
> Not sure if the localization process will require localizers to also run this
> but if they do need to then the binary for this will likely need to be added to
> mozilla-build.

This is being run only for the build of the language-neutral portion, so that won't be necessary.
Now things start to get interesting.  Here we process the package-manifest file into WiX markup.  Each [foo] directive becomes a ComponentGroup element and each file becomes a File element.  We keep the l10n files separate so that we can handle localization of the installer later.  I'm assuming here that the only localized group is going to be the AB_CD group.  That appears to be true for all of the toolkit apps.  We also may need to modify this eventually to handle optional groups for the other apps.

The output from this is two separate WiX files, "pyheat <arguments> base" produces one containing a fragment which contains all of the markup to handle the non-localized components (one component for each file) and "pyheat <arguments> l10n" produces the other containing all of the markup to handle localized components (again, one component for each file).
Attachment #440729 - Flags: review?(jmathies)
Attached patch Part 5: Connect Parts 3 and 4 (obsolete) — Splinter Review
This should be pretty straightforward, we call the helper executable we built in part 3, figure out what to do with the output, and then call the helper again to clean up the registry entries.
Attachment #440730 - Flags: review?(jmathies)
This patch does two things.  First it modifies pyheat to create a WiX file with the directory layout, which we'll need when we build the MSI.  This is done by calling "pyheat <arguments> dir".  We could fold this into the general non-localized components file if we wanted to; I kept them separate primarily for ease of debugging.

The second is more interesting.  One of the many quirks of the MSI engine is that minor upgrades don't allow you to remove files easily.  For upgrade scenarios you get to choose between:

1) A major upgrade which removed all installed files and recopies all newer versions.  This requires shipping a complete MSI.
2) A minor upgrade which changes only the files it needs to.  This allows us to ship an MSP that only contains the binary diffs of what we need.

We'll want to do 2, obviously.  Since for some brain-dead reason the component structure must be only grow across minor upgrades (that is, we cannot remove components, you actually can add components just fine) this patch causes "pyheat <arguments> rm" to generate a WiX file that creates dummy component entries with a false condition (so that they will be removed in the patch).  This is hackish but it's the canonical way to handle this[0].  This requires us to maintain a separate list of files that we've removed from an MSI in a point release.  This "msi-removed-files" (not part of this patch since it'll live in /<application>(/installer/windows/msi), not /toolkit) will be empty on trunk and then if we find we need to remove a file after shipping off that branch we can add the files there. (we could also find a way to annotate removed-files.in with what got removed in what release, but that seems harder).

I'm not sure that we'll need to remove files on point releases but it seems better to design for the possibility rather than needing to figure out what to do when we decide that we want to do that (especially because not handling this now requires EVERYONE to be aware that removing files in point releases is not safe).

Brain-dead design ftw.

[0] http://blogs.msdn.com/pmarcu/archive/2009/05/19/wix-removing-files-with-patches.aspx
Attachment #440733 - Flags: review?(jmathies)
Tiny python script to generate a product code from its argument.  In the makefile we're going to feed this script the string

"www.mozilla.org/msi-namespace/$(MOZ_APP_NAME)/$(MOZILLA_VERSION)/$(TARGET_CPU)/$(AB_CD)"

So product codes will vary by app, version, architecture, and locale.  Variation by architecture and locale is required, variation by app is good too (don't want Thunderbird overwriting Firefox) and variation by version is a prerequisite to installing multiple versions side by side.
Attachment #440734 - Flags: review?(robert.bugzilla)
These are the build system variables we'll need in the WiX.
Attachment #440735 - Flags: review?(jmathies)
That last one will make a bit more sense after I post the makefile patch, but I'm out of patch splitting time this morning so that will happen tonight.
Attachment #440735 - Attachment is obsolete: true
Attachment #440735 - Flags: review?(jmathies)
This is the toolkit portion of the MSI makefile.  The app specific makefile will include this and define the variables MOZ_MSI_L10N_FILES and MOZ_MSI_NOL10N_FILES for the files that should be included in the localized and non-localized parts of the build process.

The $(MOZ_APP_NAME)_base.wixlib file the is the language neutral intermediate file.  All of the language neutral files will be linked into this library.  Localized builds will then be built by combining this file with the localized files.  Building the wixlib is ifdefed so that we don't try to build it again during the l10n packaging stage.
Attachment #441141 - Flags: review?(jmathies)
This patch hooks up Firefox to the Toolkit foundations we've laid so far and produces a working MSI.  What we have at this point is hardcoded en-US and is essentially a glorified ZIP installer :-) (no shell/registry integration, no shortcuts, etc.)
Attachment #441220 - Flags: review?(jmathies)
I really wish there were something that stops me from attaching things if I forget to qrefresh.
Attachment #441221 - Flags: review?(jmathies)
Attachment #441220 - Attachment is obsolete: true
Attachment #441220 - Flags: review?(jmathies)
Comment on attachment 440465 [details] [diff] [review]
Part 2: Provide branding information to WiX

I don't think we want to bother with implementing SurveyURL for uninstall presently unless there is a way to open an url unelevated from the uninstaller when it is launched from Programs and Features.

Also, I have no problem with changing any of the names from what was used in NSIS if the new names are clearer.

I'm fine with this and I'm mainly minusing to get answers to the above
Attachment #440465 - Flags: review?(robert.bugzilla) → review-
(In reply to comment #275)
> (From update of attachment 440465 [details] [diff] [review])
> I don't think we want to bother with implementing SurveyURL for uninstall
> presently unless there is a way to open an url unelevated from the uninstaller
> when it is launched from Programs and Features.

We can do that.  We'll condition a custom action to run on uninstall that ShellExecutes the URL (custom actions can run at low privilege).

When we implement this though we'll have to make it configurable via a property or something so that it doesn't run in an institutional scenario.

> Also, I have no problem with changing any of the names from what was used in
> NSIS if the new names are clearer.

I think these names are fine.  IIRC the relevant names correspond exactly to the ARP registry keys and I don't see anything wrong with BrandFullNameInternal and SurveyURL.  If you have suggestions though, I can change them.
(In reply to comment #276)
> (In reply to comment #275)
> > (From update of attachment 440465 [details] [diff] [review] [details])
> > I don't think we want to bother with implementing SurveyURL for uninstall
> > presently unless there is a way to open an url unelevated from the uninstaller
> > when it is launched from Programs and Features.
> 
> We can do that.  We'll condition a custom action to run on uninstall that
> ShellExecutes the URL (custom actions can run at low privilege).
> 
> When we implement this though we'll have to make it configurable via a property
> or something so that it doesn't run in an institutional scenario.
It would be optional on the last page of the uninstall / not available for silent uninstalls and off by default so I don't think we need to do anything for admin scenarios / corp environments. Keep in mind that shellexecute against a url won't work if we were the default browser unless we set another browser as default.

> 
> > Also, I have no problem with changing any of the names from what was used in
> > NSIS if the new names are clearer.
> 
> I think these names are fine.  IIRC the relevant names correspond exactly to
> the ARP registry keys and I don't see anything wrong with BrandFullNameInternal
> and SurveyURL.  If you have suggestions though, I can change them.
Sounds good.
Attachment #440465 - Flags: review- → review+
Attachment #440469 - Flags: review?(robert.bugzilla) → review+
Comment on attachment 441140 [details] [diff] [review]
Part 8: Reflect build system variables into WiX

Can a newline be added to this file? If not that's fine.

btw: I don't know if jimm is fluent in python and I'm not all that knowledgeable in python... might want to ask around on irc for someone to review the python patches.
Comment on attachment 440734 [details] [diff] [review]
Part 7: Generate product codes at build time

I'm a little concerned with creating a uuid each time... we have talked about it before but can you prove some info that might help assuage my concerns?
(In reply to comment #278)
> (From update of attachment 441140 [details] [diff] [review])
> Can a newline be added to this file? If not that's fine.

Yes, it could be.

> btw: I don't know if jimm is fluent in python and I'm not all that
> knowledgeable in python... might want to ask around on irc for someone to
> review the python patches.

OK.

(In reply to comment #279)
> (From update of attachment 440734 [details] [diff] [review])
> I'm a little concerned with creating a uuid each time... we have talked about
> it before but can you prove some info that might help assuage my concerns?

Sure. We're creating a UUID based on "www.mozilla.org/msi-namespace/$(MOZ_APP_NAME)/$(MOZILLA_VERSION)/$(TARGET_CPU)/$(AB_CD)".  Version 5 UUIDs ( http://en.wikipedia.org/wiki/Universally_Unique_Identifier#Version_5_.28SHA-1_hash.29 ) are formed by taking a SHA-1 hash of the input string and combining that with some other stuff specified in the RFC.  As long as the input string remains the same the UUID produced will remain the same (on all computers, everywhere.  Only version 1 UUIDs depend on MAC addresses).

Because of the string chosen our product codes will vary based on the appname, app version, CPU arch, and locale.  CPU arch (really whether it's 32 or 64 bit, but this looks like how we get at it elsewhere) and locale are required by Windows Installer.  The product code must also be different for any versions that should be installable side by side (there's more required elsewhere to make that possible though). http://msdn.microsoft.com/en-us/library/aa370854%28VS.85%29.aspx
Feel free to throw the Python bits into my review queue.
Attachment #440729 - Flags: review?(jmathies) → review?(ted.mielczarek)
Attachment #440730 - Flags: review?(jmathies) → review?(ted.mielczarek)
Attachment #440733 - Flags: review?(jmathies) → review?(ted.mielczarek)
What is the end-game here?  Is it that once this bug is fixed, Firefox's installer will start being MSI?  Or is it just that Mozilla.org will have the ability to deploy as MSI? 

Is this bug a meta-bug and a code enhancement at the same time, meaning that even once the code lands on trunk this bug will remain open until the others are fixed?

Don't get me wrong, I'm cheering to see it this close already.  I just want a bit of clarity.
My goal is that when this bug is closed the build system will be able to produce an MSI packaged Firefox that is localized properly, capable of receiving MSP updates, and capable of handling all applicable shell/OS integration.  The package produced will be suitable for use in a corporate/institutional deployment scenario, though not necessarily for end-users.

In particular, the explicit non-goals for this bug are to:
1) Provide a working user interface in the installer.  All of the settings will be customized on the command line for the time being.  Bug 543390 will cover this.
2) Use 7 zip compression.  Bug 543393 will cover this.
3) Be able to install multiple instances side by side.  This is hard.  Bug 543392 will cover this.

Replacing the NSIS installer with an MSI solution will still be relatively far away when this bug is closed.
Which of the 10 patches does the l10n piece right now? Might make sense for me to have a look at that one.
Attachment #441141 - Attachment is obsolete: true
Attachment #441141 - Flags: review?(jmathies)
Attached patch Part 11: Support l10n packaging (obsolete) — Splinter Review
This patch does the necessary magic to support a non-en-US MSI package.  The patch in Bug 563186 needs to be applied between Part 10 and Part 11 to successfully build an l10nized MSI.

Notes:
I'm setting the MSI's language to 0 (language neutral).  The language property is important only for selecting UI strings for strings not stored in the MSI database.  For the moment that means that the progress dialog presented when installing the MSI will be in English regardless of the locale.

I'm stealing the NSIS helper.exe for the moment.  That will change in a future Part.

Axel: this is the piece you are looking for.
Attachment #442946 - Flags: review?(jmathies)
Attachment #442946 - Flags: feedback?(l10n)
I think it's reasonable to have what I described in Comment 283 for Firefox 3.7.
Target Milestone: Future → Firefox 3.7
If the old installer is NSIS and the new one is MSI, do we need a bug for swapping the installers during an upgrade?
That is quite a few steps away from where we are now and I'd prefer to wait until we are farther along before filing more bugs unless the person doing the work (e.g. Kyle) sees a need to file / work on the additional bug(s) at this time.
We'll want to store the product code on disk so that we can use it later (e.g. so that the app updater, shell integration, etc can make sure they're dealing with the right installation)
Attachment #442956 - Flags: review?(jmathies)
Set the standard MSI ARP properties.  The MSI engine will create the right registry keys from these properties.  See http://msdn.microsoft.com/en-us/library/aa368032%28VS.85%29.aspx
Attachment #442957 - Flags: review?(jmathies)
We're going to interact properly and we don't want Windows guessing what to do on our behalf.

http://msdn.microsoft.com/en-us/library/bb736313%28VS.85%29.aspx
Attachment #442958 - Flags: review?(jmathies)
This patch adds support to the Firefox MSI for the two standard shortcuts and adds both Firefox and backend toolkit support for having localized strings within the installer metadata.  The only file localizers should have to touch is the browser/locales/ab-CD/msi.dtd file.
Attachment #442963 - Flags: review?(jmathies)
Attachment #442963 - Flags: feedback?(l10n)
Attachment #442963 - Attachment is patch: true
Attachment #442963 - Attachment mime type: application/octet-stream → text/plain
Comment on attachment 442963 [details] [diff] [review]
Part 15: Handle Shortcuts (and l10n within the installer data itself)

The DTD file itself should remain to be a valid DTD file.

I'm not sure why you're relying on MOZ_APP_DISPLAYNAME here. If you absolutely want to avoid using just plain "Firefox", you should include brand.dtd and use &brandShortName; there.

The other thing, in the current installer, I think only the 7z expansion dialog is non-localizable, I'm not exactly sure if there's more UI you're showing yet? If so, those should be localizable as well, and we shouldn't rely on any l10n inside wix itself, IMHO.
Attachment #442963 - Flags: feedback?(l10n) → feedback-
(In reply to comment #294)
> (From update of attachment 442963 [details] [diff] [review])
> The DTD file itself should remain to be a valid DTD file.

I'm not sure what you mean.

> I'm not sure why you're relying on MOZ_APP_DISPLAYNAME here. If you absolutely
> want to avoid using just plain "Firefox", you should include brand.dtd and use
> &brandShortName; there.

OK.

> The other thing, in the current installer, I think only the 7z expansion dialog
> is non-localizable, I'm not exactly sure if there's more UI you're showing yet?
> If so, those should be localizable as well, and we shouldn't rely on any l10n
> inside wix itself, IMHO.

Right now, because the installer we're building does not include any UI of its own, the only thing that gets displayed is the basic progress UI and error dialogs.  These come with Windows and I believe are in the language that Windows is using.  We're relying on that for the moment, but I fully intend for that to go away.

If you're on Windows and want to see what I'm talking about, try www.kylehuey.com/moz/msi/firefox-3.7a5pre.en-US.win32.msi.  Be warned that it'll overwrite whatever is in your Program Files/Minefield because there's no UI to choose another location.
Comment on attachment 442946 [details] [diff] [review]
Part 11: Support l10n packaging

tentative feedback-, as I don't understand enough of what's where.

But, this should work just based on getting the binary files from ftp, aka, you'll need to at least touch the wget-en-US target in toolkit/locales/l10n.mk, probably conditionally. And then you'll need magic to get from that file to your $(MOZ_APP_NAME)_base.wixlib. I would expect that that changes a few of your paths, too.

I'd love to be able to share the nsis and the msi target, as they're having a bunch in common, but I'm not sure if that's feasible.

In 
@$(MAKE) package-msi AB_CD=$* MSI_INSTALLER_IN=$(MSI_INSTALLER_IN)
you forward the MSI_INSTALLER_IN variable, is that necessary?

Are we going to have special names for the msi for official releases? Those go through funky code paths that I never quite understand, so you should check those, too.
Attachment #442946 - Flags: feedback?(l10n) → feedback-
(In reply to comment #295)
> (In reply to comment #294)
> > (From update of attachment 442963 [details] [diff] [review] [details])
> > The DTD file itself should remain to be a valid DTD file.
> 
> I'm not sure what you mean.

#filter substitution

is not valid XML, and will hurt anyone using standard xml-oriented editors, and compare-locales.
(In reply to comment #297)
> (In reply to comment #295)
> > (In reply to comment #294)
> > > (From update of attachment 442963 [details] [diff] [review] [details] [details])
> > > The DTD file itself should remain to be a valid DTD file.
> > 
> > I'm not sure what you mean.
> 
> #filter substitution
> 
> is not valid XML, and will hurt anyone using standard xml-oriented editors, and
> compare-locales.

Ah ok, that's going to go away once I switch to brand.dtd
This is a better solution.  We already put the brand name stuff in defines.wxi.in; with a slight bit of magic we can make that preprocessor variable persist until we link in the localization stuff and have a much nicer DTD (we also don't have to go chasing down brand.dtd which isn't easy to get at at build time).
Attachment #442963 - Attachment is obsolete: true
Attachment #442993 - Flags: review?(jmathies)
Attachment #442993 - Flags: feedback?(l10n)
Attachment #442963 - Flags: review?(jmathies)
Comment on attachment 442993 [details] [diff] [review]
Part 15: Handle shortcuts and l10n strings V1.1

Actually, this is not ideal either.  Some locales (e.g. hi-IN) put the brand names into the native alphabet which this certainly won't accomplish.
Attachment #442993 - Attachment is obsolete: true
Attachment #442993 - Flags: review?(jmathies)
Attachment #442993 - Flags: feedback?(l10n)
It doesn't look like the NSIS installer gives localizers any choice for how "Mozilla Firefox" appears in the shortcut.  Rob, Axel, is this something we care about?
Then I don't have to give you that comment ;-).

Another thing to note is that you should support l10n-merge for the DTD file, that is, if LOCALE_MERGEDIR is given, the files are picked up from the following dirs, in this order:

$(LOCALE_MERGEDIR)/browser/.. # browser/locales being relativesrcdir
$(LOCALE_SRCDIR) # includes browser, aka relativesrcdir base
@srcdir@/locales/en-US

i.e., it checks for in the LOCALE_MERGEDIR, which is a sparse directory in the same structure as l10n, then l10n, then en-US.
Comment on attachment 441140 [details] [diff] [review]
Part 8: Reflect build system variables into WiX

Note, we'll be making some modest changes here in the near term for win7 integrations work. See bug 521141, bug 526663, and bug 562753.
Attachment #441140 - Flags: review?(jmathies) → review+
Comment on attachment 441221 [details] [diff] [review]
Part 10: Build a (very) basic en-US Firefox MSI

+           UpgradeCode="2c7a4972-b19b-459f-ac96-2d75692d95b9">


Doesn't UpgradeCode change with each rev of the app? If I remember correctly, you up this for major upgrades which would cause msi to uninstall previous versions. We have to do this every time we remove or add a file to the install. I'm curious how we plan to manage minor/major upgrades, can you fill me in on your plans?
 (In reply to comment #304)
> (From update of attachment 441221 [details] [diff] [review])
> +           UpgradeCode="2c7a4972-b19b-459f-ac96-2d75692d95b9">
> 
> 
> Doesn't UpgradeCode change with each rev of the app? If I remember correctly,
> you up this for major upgrades which would cause msi to uninstall previous
> versions. We have to do this every time we remove or add a file to the install.
> I'm curious how we plan to manage minor/major upgrades, can you fill me in on
> your plans?

That's one way to do it.  Another way to do it (and the way I think we should do it, because it doesn't involve bookkeeping w/UpgradeCodes) is to give every package we ship the same UpgradeCode.  There are two different paths to an upgrade:

1) The user receives the update through the application updater.  In this case updater.exe will be aware of the ProductCode that should be installed over (Part 12) and we'll write some logic into the installer package to shortcircuit the normal process of scanning the system for all products with the appropriate UpgradeCodes and instead install over the provided ProductCode.

2) The user downloads the MSI package and installs it manually.  Then when the MSI engine scans for the relevant UpgradeCodes we will get back a list of every application installation on the machine.  We can then present UI allowing the user to choose which installation to upgrade or to install another installation entirely.  (Basically between FindRelatedProducts[0] and RemoveExistingProducts[1] we can present UI to narrow the selection down).

We don't have to do a major upgrade every time we add or remove a file though.  You can add files in minor upgrades without any trouble, and the code in Part 6 allows us to remove files in minor upgrades as well (it creates placeholder component entries to keep the MSI engine happy, but sets their condition to false so they aren't installer).  Minor upgrades are preferable when possible because we don't have to ship everything (we can ship a binary delta) and the MSI engine does not uninstall and install everything (so the upgrade time is less)

As for when we use major/minor upgrades:
- Any upgrade that crosses branches (e.g. 3.5.9 -> 3.6.3) is a major upgrade (this is a MSI requirement because of the version # change)[2]
- Any upgrade on a branch that would be shipped as a partial MAR will be a minor upgrade. (e.g. 3.6.3 -> 3.6.4)
- Any other upgrade is a major upgrade.

Essentially partial MAR == minor upgrade, complete MAR == major upgrade, except that if we cross branches it has to be a major upgrade (I'm not sure whether we ship partials for an upgrade across branches).

[0] http://msdn.microsoft.com/en-us/library/aa368600%28VS.85%29.aspx
[1] http://msdn.microsoft.com/en-us/library/aa371197%28VS.85%29.aspx
[2] http://msdn.microsoft.com/en-us/library/aa369786%28VS.85%29.aspx
Comment on attachment 442920 [details] [diff] [review]
Part 9: Provide app agnostic MSI building capability V1.1

Writing that last comment made me realize we should be generating the ProductCode based on MOZ_APP_VERSION, not the platform version.
Attachment #442920 - Attachment is obsolete: true
Attachment #442920 - Flags: review?(jmathies)
Keywords: helpwanted
We're going to need to get brand.dtd somewhere we can find it and this should do the trick.  I tested this with hi-IN to make sure that the right files end up in $(DIST)/branding.

@Axel: Is the LOCALE_MERGEDIR stuff correct here?
Attachment #443024 - Flags: review?(ted.mielczarek)
Attachment #443024 - Flags: feedback?(l10n)
Actually, I think this is what we want from an l10n perspective.
Attachment #443025 - Flags: review?(ted.mielczarek)
Attachment #443025 - Flags: feedback?(l10n)
Attachment #443024 - Attachment is obsolete: true
Attachment #443024 - Flags: review?(ted.mielczarek)
Attachment #443024 - Flags: feedback?(l10n)
Attachment #441221 - Flags: review?(jmathies) → review+
(In reply to comment #305)
> 1) The user receives the update through the application updater.  In this case
> updater.exe will be aware of the ProductCode that should be installed over
> (Part 12) and we'll write some logic into the installer package to shortcircuit
> the normal process of scanning the system for all products with the appropriate
> UpgradeCodes and instead install over the provided ProductCode.
> 
> 2) The user downloads the MSI package and installs it manually.  Then when the
> MSI engine scans for the relevant UpgradeCodes we will get back a list of every
> application installation on the machine.  We can then present UI allowing the
> user to choose which installation to upgrade or to install another installation
> entirely.  (Basically between FindRelatedProducts[0] and
> RemoveExistingProducts[1] we can present UI to narrow the selection down).


That second option seems risky, what if they chose the wrong install?

Since this is going to to be used by corp enivrons, how does centralized update management fit into this? Would admins be able to chose option two and specify which version (likely only one would be installed) to upgrade silently?
Attachment #442956 - Flags: review?(jmathies) → review+
Attachment #442957 - Flags: review?(jmathies) → review+
<Property Id="MSIDEPLOYMENTCOMPLIANT" Value="1" />

If someone is installer with an older version of the msi, how would we handle that? Will we enforce minimum windows installer version requirements?
(In reply to comment #307)
> Created an attachment (id=443023) [details]
> Part 9: Provide app-agnostic MSI building capability V1.2

These make file changes should be reviewed by someone else.
(In reply to comment #310)
> That second option seems risky, what if they chose the wrong install?

How is that substantively different 

> Since this is going to to be used by corp enivrons, how does centralized update
> management fit into this? Would admins be able to chose option two and specify
> which version (likely only one would be installed) to upgrade silently?

I assume they would only have one version installed.  If they have multiple version installed we probably could expose what we'll do for the application updater to them in a simple way.  Not sure whether this is a real concern.

(In reply to comment #311)
> <Property Id="MSIDEPLOYMENTCOMPLIANT" Value="1" />
> 
> If someone is installer with an older version of the msi, how would we handle
> that? Will we enforce minimum windows installer version requirements?

This property is ignored on pre-Vista versions of Windows Installer.  I've set the minimum version required to Version 3, which is available (though not shipped by default) on all supported platforms. http://en.wikipedia.org/wiki/Windows_Installer#Versions  Version 3 is the first version that does patching in a decent way.

(In reply to comment #312)
> (In reply to comment #307)
> > Created an attachment (id=443023) [details] [details]
> > Part 9: Provide app-agnostic MSI building capability V1.2
> 
> These make file changes should be reviewed by someone else.

OK.
Attachment #443023 - Flags: review?(jmathies) → review?(ted.mielczarek)
Comment on attachment 442958 [details] [diff] [review]
Part 14: Tell Windows we know what we're doing with UAC.

Are we at a point yet where I can apply a set of patches and take this for a spin?
Attachment #442958 - Flags: review?(jmathies) → review+
(In reply to comment #313)
> (In reply to comment #310)
> > That second option seems risky, what if they chose the wrong install?
> 
> How is that substantively different 
> 

Maybe I misinterpeted, this would happen when the user downloaded a new, complete msi and tried to install it w/existing installs in place? I was thinking you were referring to updates.

Speaking which, users often install multiple version of the same product. In relation to product codes, will having three different major releases install via msi still be possible?
(In reply to comment #314)
> (From update of attachment 442958 [details] [diff] [review])
> Are we at a point yet where I can apply a set of patches and take this for a
> spin?

If you apply Parts 1 - 10 and the patch from Bug 563186, and install WiX somewhere in your path you should be able to build the MSI (you might have to comment out some stuff in your package-manifest if you're not doing a jemalloc build).

(In reply to comment #315)
> (In reply to comment #313)
> > (In reply to comment #310)
> > > That second option seems risky, what if they chose the wrong install?
> > 
> > How is that substantively different 
> > 
> 
> Maybe I misinterpeted, this would happen when the user downloaded a new,
> complete msi and tried to install it w/existing installs in place? I was
> thinking you were referring to updates.

Right.

> Speaking which, users often install multiple version of the same product. In
> relation to product codes, will having three different major releases install
> via msi still be possible?

I haven't tested it yet but that is a design goal for this bug.
Comment on attachment 440734 [details] [diff] [review]
Part 7: Generate product codes at build time

Thanks for the explanation!
Attachment #440734 - Flags: review?(robert.bugzilla) → review+
(In reply to comment #313)
> > If someone is installer with an older version of the msi, how would we handle
> > that? Will we enforce minimum windows installer version requirements?
> This property is ignored on pre-Vista versions of Windows Installer.  I've set
> the minimum version required to Version 3, which is available (though not
> shipped by default) on all supported platforms.
> http://en.wikipedia.org/wiki/Windows_Installer#Versions  Version 3 is the first
> version that does patching in a decent way.

WRT this, it is worth noting that Windows Installer 3.1 is a 'prerequisite' install for Windows/Microsoft Update, so anyone whose computer is at all up to date should have it already anyway.
Cleaner
Attachment #443025 - Attachment is obsolete: true
Attachment #443983 - Flags: review?(ted.mielczarek)
Attachment #443983 - Flags: feedback?(l10n)
Attachment #443025 - Flags: review?(ted.mielczarek)
Attachment #443025 - Flags: feedback?(l10n)
Comment on attachment 443983 [details] [diff] [review]
Part 15a: Export brand.dtd and brand.properties to $(DIST)/branding

I guess this should work for me.
Attachment #443983 - Flags: feedback?(l10n) → feedback+
Blocks: 564875
Comment on attachment 440466 [details] [diff] [review]
Part 3: Capture the self-registration information from DLLs V1.1

>diff --git a/toolkit/mozapps/installer/windows/msi/Makefile.in b/toolkit/mozapps/installer/windows/msi/Makefile.in
>new file mode 100644
>--- /dev/null
>+++ b/toolkit/mozapps/installer/windows/msi/Makefile.in
>+
>+DEPTH     = ../../../../..
>+topsrcdir = @top_srcdir@
>+srcdir    = @srcdir@
>+VPATH     = @srcdir@
>+
>+include $(DEPTH)/config/autoconf.mk
>+
>+DIRS = \
>+  selfregcap \
>+  $(NULL)
>+
>+include $(topsrcdir)/config/rules.mk

Drive-by comment: please don't add new makefiles to the tree that contain nothing but DIRS. Just add the subdir directly to DIRS in the ancestor makefile instead. Every Makefile is one extra invocation of make, and they add up.
(In reply to comment #321)
> Drive-by comment: please don't add new makefiles to the tree that contain
> nothing but DIRS. Just add the subdir directly to DIRS in the ancestor makefile
> instead. Every Makefile is one extra invocation of make, and they add up.

OK.  Any idea when you could get around to reviewing the scripts?
I'll get to them this week, I'm plowing through my review queue (hence why I was looking at patches on this bug).
(In reply to comment #323)
> I'll get to them this week, I'm plowing through my review queue (hence why I
> was looking at patches on this bug).

Awesome.  Thanks!
Comment on attachment 440729 [details] [diff] [review]
Part 4: Process package-manifest into WiX markup.

This would make a nice start towards replacing Packager.pm in Python. :)

As a general comment, for Python scripts that are part of the build system, we like to write them such that they'd be usable as Python modules as well. Generally this just means:
1) Don't put any logic at the top-level of the script, but keep it all in functions/classes
2) Do something like:
if __name__ == '__main__':
  main()

>diff --git a/toolkit/mozapps/installer/windows/msi/pyheat.py b/toolkit/mozapps/installer/windows/msi/pyheat.py
>new file mode 100644
>--- /dev/null
>+++ b/toolkit/mozapps/installer/windows/msi/pyheat.py
>+# PyHeat
>+#
>+# This is a python version of the Heat tool from the WiX toolset.  The
>+# motivation for not using Heat is twofold:
>+#
>+# 1) Heat doesn't understand the difference between l10n and non-l10n files
>+#    It's important for us to keep them separate so that we can produce all
>+#    of the l10n builds by only building the binary components once.
>+#
>+# 2) Heat cannot handle files that are removed in minor versions, which require
>+#    some special handling on our part to avoid breaking patches.


Can you move the "usage" comment block about the options the script takes from down below to up here? Makes it easier to open up the file and see what it expects.

>+# package-manifest allows a wildcard operator.  Here we'll fix it up
>+def FixUpFiles(files):

I'd prefer if you put function-scoped comments as docstrings instead, so like:
def FixUpFiles(files):
"""Fix up wildcard operators in filenames in a package-manifest"""

Also, this function could use a clearer name. Perhaps "ExpandWildcards"?

>+  fixedfiles = files
>+  def expandWildcard(file, group):
>+    (root, sep, tail) = file.rpartition("/")
>+    if tail == "*":
>+      foundfiles = []
>+      # Lets have some fun
>+      fixedfiles.remove((file, group))
>+      for (dirpath, dirnames, filenames) in os.walk(root + sep):
>+        for filename in filenames:
>+          foundfiles.append((os.path.join(dirpath, filename).replace("\\", "/"), group))
>+
>+      if foundfiles == []:
>+        # There is a wildcard but nothing to find.  Let's die so somebody can fix
>+        # the package manifest
>+        sys.exit("Wildcard operator at " + file + " but nothing found! Fix package-manifest?")

I don't know if this is really necessary. I don't think our existing packaging scripts will fail on this case.

>+      fixedfiles.extend(foundfiles)
>+  [expandWildcard(file, group) for (file, group) in filter(lambda (f,g): f[-1:] == "*", files)]

I'm not wild about using the list comprehension for side effects here. I'd prefer you write that out as a straight for loop if you don't need the results.

>+# Turn a directory tree structure into WiX <Directory> elements
>+def XMLifyDirectoryTree(name, basename, tree, document):

While you're adding docstrings to these functions, can you describe what the parameters are? It's not readily apparent here.

>+  workingnodes = []
>+  # Does the tree have child nodes (i.e. keys that are not "")
>+  if not tree.keys() == [""]:

if tree.keys() != [""]:

>+  # Now process just this node unless it's the base node
>+  if name != "":
>+    direlement = document.createElement("Directory")
>+    sanedir = re.sub("[-{}\[\]@]", "_", basename)

A regex felt like overkill for this, but the only other useful solution I could find would be:
from string import maketrans
sanedir = basename.translate(maketrans("-{}[]@","______")

which actually seems worse to me.


>+# Since Python doesn't have native tree types and we don't really need a full
>+# tree type, this uses dictionaries to simulate a tree.  The directory layout
>+#
>+# README
>+# LICENSE
>+# foo/
>+#     foo.exe
>+#     bar.dll
>+# monty/
>+#      python.dll
>+#
>+# is represented by the dictionary
>+#
>+# {"": ["README", "LICENSE"], "foo": {"": ["foo.exe", "bar.dll"]}, "monty": {"": ["python.dll"]}}
>+#
>+# so an empty key gives a list of files in the current directory, otherwise
>+# the key is the directory name and the value is a dictionary representing that
>+# subdirectory.  Hacky, but it gets the job done.

This is a very explanatory comment, thanks for writing it.

>+def BuildDirectoryTree(files):
>+  tree = dict()
>+  for file in files:
>+    workingnode = tree
>+    (root, sep, tail) = file.partition("/")
>+    while sep == "/":
>+      if root not in workingnode:
>+        workingnode[root] = dict()
>+      workingnode = workingnode[root]

I was going to suggest a defaultdict here, but I guess you need an arbitrarily nested dict, which would mean subclassing it (shown at the bottom of this post: http://personalpages.tds.net/~kent37/kk/00013.html )


>+# Turn a directory tree structure into WiX Components and Files
>+def XMLifyFileTree(dirname, path, tree, callback, document):
>+  global basedir

I'm not a fan of "global". I'd prefer you either pass everything around explicitly, or use a class.

>+  workingnodes = []
>+  for childnode in tree.keys():
>+    if childnode == "":

# Build list of files in this directory

>+          # Sanitize away characters not allowed to be in WiX identifiers
>+          sanedir = re.sub("[-{}\[\]@]", "_", path).replace("/", "_")
>+          if sanedir[-1:] == "_":

You don't need the :, sanedir[-1] works just fine. You can also write sanedir.endswith("_"), which I find clearer.

>+    else:

# Build tree recursively

>+# Create a true condition element on each file

This comment could stand to be clarified. What does a true condition element do?

>+def FileCallback(document, element):

>+# The arguments accepted are
>+# -m <file> : Process <file> as a package-manifest (REQUIRED)
>+# -o <directory> : Directory to write output file in (REQUIRED)
>+# -n <name> : Name of the application used to write output file (REQUIRED)
>+# -l <componentgroup> : Component Group that is the l10n group (REQUIRED)
>+# And then one of
>+# "base" : Output the WiX markup for the non-localized files and components
>+# "l10n" :               ...             localized     ...
>+try:
>+  opts, args = getopt.getopt(sys.argv[1:], "m:o:n:l:")
>+except getopt.GetoptError, err:
>+  print str(err)
>+  sys.exit(2)

I'd much prefer if you used optparse.OptionParser instead. It's a little more verbose to setup, but you get --help for free. There are copious examples in the tree:
http://mxr.mozilla.org/mozilla-central/search?string=OptionParser

>+
>+for opt, arg in opts:
>+  if opt == "-m":
>+    package_manifest = arg
>+  elif opt == "-o":
>+    output = arg
>+  elif opt == "-n":
>+    appname = arg
>+  elif opt == "-l":
>+    l10n = arg
>+
>+for line in fileinput.input([package_manifest]):

If you're just reading the one file, you can just use
for line in open(package_manifest, 'r'):

>+  sline = line.strip()
>+  if "" == sline or ';' == sline[0]:
>+    # Comments or empty lines
>+    continue
>+  elif '[' == sline[0]:
>+    # This names a ComponentGroup.  We'll create one ComponentGroup for each
>+    # [THING] directive
>+    name = sline[1:-1]
>+    groups.append(name)
>+    continue
>+
>+  # Otherwise we have a file (slice away the bin/ stuff)
>+  files.append((sline[4:], groups[-1]))

It'd probably be good to assert that the line starts with bin/ here to avoid parsing junk.

Also, you could just put this in an else and drop the continue statements in the other if blocks.


>+
>+# Take care of wildcards
>+files = FixUpFiles(files)

Seems like instead of iterating over the whole list of files again, you could just put this in the loop above. Like:
if sline.endswith('*'):
  files.extend(ExpandWildcards(sline))
(assuming you changed ExpandWildcards to take a single line from the package manifest and return a list of files)


>+if "l10n" in args:
>+  if l10n in groups:

Just combine these into a single if:
if "l10n" in args and l10n in groups:

>+    wixDocl10n = GenerateWixDocument()
>+    ProcessFileTree(l10n, BuildDirectoryTree(map(lambda (f, g): f, filter(lambda (f, g): g == l10n, files))),

This nested map/filter is really hard to grok. You want to get just the filenames out of your list of tuples? I would write that as a list comprehension:
[x[0] for x in files if x[1] == l10n]

>+if "base" in args:
>+  wixDocBase = GenerateWixDocument()
>+  for group in groups:
>+    if not group == l10n:
>+      ProcessFileTree(group, BuildDirectoryTree(map(lambda (f, g): f, filter(lambda (f, g): g == group, files))),
>+                      FileCallback, wixDocBase)
>+  outputfile = output + appname + "_base.wxs"
>+  file = open(outputfile, 'w')
>+  file.write(wixDocBase.toprettyxml())
>+  file.close()

Feels like you could probably combine these two blocks (l10n/base) into a single function with a little refactoring (they are practically identical).

r- for some cleanup, but it's a good start (I'm just picky about my Python!)
Attachment #440729 - Flags: review?(ted.mielczarek) → review-
Comment on attachment 440730 [details] [diff] [review]
Part 5: Connect Parts 3 and 4

>diff --git a/toolkit/mozapps/installer/windows/msi/pyheat.py b/toolkit/mozapps/installer/windows/msi/pyheat.py
>--- a/toolkit/mozapps/installer/windows/msi/pyheat.py
>+++ b/toolkit/mozapps/installer/windows/msi/pyheat.py
>@@ -147,16 +148,71 @@ def ProcessDirectoryTree(tree, document)
>     dirrefElement.appendChild(node)
> 
>   # Generate a fragment element
>   dirsFragment = document.createElement("Fragment")
>   dirsFragment.appendChild(dirrefElement)
> 
>   document.documentElement.appendChild(dirsFragment)
> 
>+
>+def HarvestNativeDll(name, document, componentElement, fileElement):
>+  # This is messy because the _winreg module uses exceptions for control flow
>+  # The general idea is to call the selfregcap helper executable to divert
>+  # the registry modifications made by DllRegisterServer to
>+  # HKCU\Software\Mozilla\SelfRegCap where we can then examine it and emit the
>+  # appropriate WiX markup.  Calling selfregcap again then cleans up the
>+  # registry entries.

Please make this a docstring comment. Also, you aren't kidding that this is ugly with the Exceptions.

>+  os.system("selfregcap.exe " + name);

I'd prefer you
from subprocess import check_call
check_call(["selfregcap.exe", name])

>+  try:
>+    regkey = _winreg.OpenKey(_winreg.HKEY_CURRENT_USER, "Software\\Mozilla\\SelfRegCap")

Where does the "Mozilla" key come from? Is that just hardcoded in selfregcap.exe?

>         componentElement.appendChild(fileElement)
>+        if file[-3:] == "dll":

if file.endswith("dll"):

r=me with those fixes.
Attachment #440730 - Flags: review?(ted.mielczarek) → review+
Comment on attachment 440733 [details] [diff] [review]
Part 6: Handle directories and files removed in point releases 

>diff --git a/toolkit/mozapps/installer/windows/msi/pyheat.py b/toolkit/mozapps/installer/windows/msi/pyheat.py
>--- a/toolkit/mozapps/installer/windows/msi/pyheat.py
>+++ b/toolkit/mozapps/installer/windows/msi/pyheat.py
>@@ -271,40 +271,53 @@ def GenerateWixDocument():
>   return wixDoc
> 
> # Create a true condition element on each file
> def FileCallback(document, element):
>   trueNode = document.createTextNode("2=2")
>   condNode = document.createElement("Condition")
>   condNode.appendChild(trueNode)
>   element.appendChild(condNode)
>+# Create a false condition element on each file
>+def RmFileCallback(document, element):

Needs a comment like I said about the other function in my previous review.

>+  falseNode = document.createTextNode("1=0")

This is weird. That's just how WiX works?

>+# We only need to handle removed files for "rm" and "dir" modes.  This lets us
>+# not build that list during l10n handling.
>+if "rm" in args or "dir" in args:
>+ for line in fileinput.input([removed_files]):
>+    sline = line.strip()
>+    removedfiles.append(sline)

You could write this as:
removedfiles = [line.strip() for line in open(removed_files, 'r')]

>+if "dir" in args:
>+  wixDocDir = GenerateWixDocument()
>+  tree = BuildDirectoryTree(map(lambda (f, g): f, files) + removedfiles)
>+  ProcessDirectoryTree(tree, wixDocDir)
>+  outputfile = output + appname + "_dir.wxs"
>+  file = open(outputfile, 'w')
>+  file.write(wixDocDir.toprettyxml())
>+  file.close()
>+
>+if "rm" in args:
>+  wixDocRm = GenerateWixDocument()
>+  removedTree = BuildDirectoryTree(removedfiles)
>+  ProcessFileTree("removed-files", removedTree, RmFileCallback, wixDocRm)
>+  outputfile = output + appname + "_rm.wxs"
>+  file = open(outputfile, 'w')
>+  file.write(wixDocRm.toprettyxml())
>+  file.close()

I really think you need to abstract this logic into a function.

r+ with those changes.
Attachment #440733 - Flags: review?(ted.mielczarek) → review+
 (In reply to comment #325)
> (From update of attachment 440729 [details] [diff] [review])
> This would make a nice start towards replacing Packager.pm in Python. :)

It probably would be.
> 
> >+def BuildDirectoryTree(files):
> >+  tree = dict()
> >+  for file in files:
> >+    workingnode = tree
> >+    (root, sep, tail) = file.partition("/")
> >+    while sep == "/":
> >+      if root not in workingnode:
> >+        workingnode[root] = dict()
> >+      workingnode = workingnode[root]
> 
> I was going to suggest a defaultdict here, but I guess you need an arbitrarily
> nested dict, which would mean subclassing it (shown at the bottom of this post:
> http://personalpages.tds.net/~kent37/kk/00013.html )

Will take a look at this.
 
> >+# Create a true condition element on each file
> 
> This comment could stand to be clarified. What does a true condition element
> do?

See below

> r- for some cleanup, but it's a good start (I'm just picky about my Python!)

Please be.  This is the first script of substance I've written in it.

(In reply to comment #326)
> (From update of attachment 440730 [details] [diff] [review])
> >diff --git a/toolkit/mozapps/installer/windows/msi/pyheat.py b/toolkit/mozapps/installer/windows/msi/pyheat.py
> >--- a/toolkit/mozapps/installer/windows/msi/pyheat.py
> >+++ b/toolkit/mozapps/installer/windows/msi/pyheat.py
> >@@ -147,16 +148,71 @@ def ProcessDirectoryTree(tree, document)
> >     dirrefElement.appendChild(node)
> > 
> >   # Generate a fragment element
> >   dirsFragment = document.createElement("Fragment")
> >   dirsFragment.appendChild(dirrefElement)
> > 
> >   document.documentElement.appendChild(dirsFragment)
> > 
> >+
> >+def HarvestNativeDll(name, document, componentElement, fileElement):
> >+  # This is messy because the _winreg module uses exceptions for control flow
> >+  # The general idea is to call the selfregcap helper executable to divert
> >+  # the registry modifications made by DllRegisterServer to
> >+  # HKCU\Software\Mozilla\SelfRegCap where we can then examine it and emit the
> >+  # appropriate WiX markup.  Calling selfregcap again then cleans up the
> >+  # registry entries.
> 
> Please make this a docstring comment. Also, you aren't kidding that this is
> ugly with the Exceptions.

I know.  I was tempted to rewrite _winreg and submit a patch.  It's god awful as is.

> >+  try:
> >+    regkey = _winreg.OpenKey(_winreg.HKEY_CURRENT_USER, "Software\\Mozilla\\SelfRegCap")
> 
> Where does the "Mozilla" key come from? Is that just hardcoded in
> selfregcap.exe?

Yes.

(In reply to comment #327)
> (From update of attachment 440733 [details] [diff] [review])
> 
> >+  falseNode = document.createTextNode("1=0")
> 
> This is weird. That's just how WiX works?

So because of the absolutely braindead way Windows Installer handles components, you have to keep components that you've removed in a point release around but inactive.  This condition does that.  From our perspective, it's required, even though it's awful.


Everything else sounds good.  Thanks for the reviews!
Comment on attachment 443023 [details] [diff] [review]
Part 9: Provide app-agnostic MSI building capability V1.2

>diff --git a/toolkit/mozapps/installer/windows/msi/wix.mk b/toolkit/mozapps/installer/windows/msi/wix.mk
>new file mode 100644
>--- /dev/null
>+++ b/toolkit/mozapps/installer/windows/msi/wix.mk
>+# The application's MSI makefile should include this makefile.  
>+# It should define MOZ_MSI_L10N_FILES and MOZ_MSI_NOL10N_FILES 
>+
>+# We want to remove the minor version number, but preserve any suffixes
>+# We do this to generate the same product code for an entire release branch
>+MOZ_APP_SHORT_VERSION = $(shell echo $(MOZ_APP_VERSION) | $(PERL) -pe "s/(\.\d+)(\.\d+)/\1/")

Perl seems like overkill here. Just use sed, maybe?

>+
>+MSI_PRODUCTCODE = $(shell $(PYTHON) $(topsrcdir)/toolkit/mozapps/installer/windows/msi/makeproductcode.py "www.mozilla.org/msi-namespace/$(MOZ_APP_NAME)/$(MOZILLA_VERSION)/$(TARGET_CPU)/$(AB_CD)")

I don't think we should be hardcoding www.mozilla.org here.

If these variables will get used more than once you might want to make them :=, which will execute them when the makefile is parsed, instead of when they're referenced.

>+MSI_DEFINES = \
>+  -DMOZ_PKG_VERSION=$(MOZ_PKG_VERSION) \
>+  -DMOZ_MSI_BRANDING=$(DIST)/branding \
>+  -DMOZ_MSI_BIN=$(DIST)/bin \
>+  -DMOZ_MSI_PKG_PREFIX=$(MOZ_MSI_PKG_PREFIX) \
>+  -DMOZ_MSI_TOOLKIT_PATH=$(topsrcdir)/toolkit/mozapps/installer/windows/msi \
>+  -DMSI_PRODUCTCODE=$(MSI_PRODUCTCODE)
>+  -DTARGET_CPU=$(TARGET_CPU)
>+
>+MSI_LINK_OPTS += -b $(DIST)/bin
>+
>+%.wixobj: %.wxs
>+	candle -nologo -I$(topsrcdir)/toolkit/mozapps/installer/windows/msi -I$(DIST)/branding -I. $<

We should probably check for the 'candle' program in configure, similar to how we check for makensis:
http://mxr.mozilla.org/mozilla-central/source/configure.in#6421

>+
>+$(MOZ_APP_NAME)_%.wxs:
>+	cd $(DIST)/bin; \
>+	$(PYTHON) $(topsrcdir)/toolkit/mozapps/installer/windows/msi/pyheat.py -n "$(MOZ_APP_NAME)" -m $(MOZ_MSI_PACKAGE_MANIFEST) -r $(MOZ_MSI_REMOVED_FILES) -l "$(AB_CD)" -o $(MOZ_MSI_PYHEAT_OUTPUT_DIR) $*

Where do all these other MOZ_MSI variables get set?

Also, you should write the script so that it doesn't need to be run from dist/bin. Pass it a path to $(DIST) or $(DIST)/bin as an argument if you need to.

>+msi:: defines.wxi $(DIST)/$(PKG_BASENAME).msi

Is defines.wxi required to build the .msi? If so, you should instead make it a prerequisite of the .msi rule below.

>+
>+productcode::
>+	echo $(MSI_PRODUCTCODE)
>+
>+$(DIST)/$(PKG_BASENAME).msi: $(MOZ_MSI_L10N_FILES) $(MOZ_APP_NAME)_base.wixlib $(MOZ_APP_NAME)_l10n.wixobj 
>+	light -nologo $(MSI_LINK_OPTS) $^ -out $@

We should check for this binary as well.

>+
>+ifneq ($(MOZ_MSI_L10N),1)

If you just want to check that a variable is set you can just say:
ifdef MOZ_MSI_L10N
(unset variables are undefined)
Attachment #443023 - Flags: review?(ted.mielczarek) → review-
Comment on attachment 443983 [details] [diff] [review]
Part 15a: Export brand.dtd and brand.properties to $(DIST)/branding

>diff --git a/browser/branding/nightly/Makefile.in b/browser/branding/nightly/Makefile.in
>--- a/browser/branding/nightly/Makefile.in
>+++ b/browser/branding/nightly/Makefile.in
>@@ -81,17 +81,16 @@ LINUX_BRANDING_FILES = \
> 	$(NULL)
> 
> OS2_BRANDING_FILES = \
> 	firefox-os2.ico \
> 	document-os2.ico \
> 	$(NULL)
> 
> export::
>-	$(NSINSTALL) -D $(DIST)/branding

Leave this in. It shouldn't error if the directory already exists.
Attachment #443983 - Flags: review?(ted.mielczarek) → review+
Review comments addressed.  This is built on my latest patch to Bug 511648.
Attachment #440729 - Attachment is obsolete: true
Attachment #448235 - Flags: review?(ted.mielczarek)
Review comments addressed.  Carrying forward r+ because changes are minor.
Attachment #440730 - Attachment is obsolete: true
Attachment #448236 - Flags: review+
Review comments addressed.  This has changed enough to need another look over IMO.
Attachment #448237 - Flags: review?(ted.mielczarek)
Review comments addressed and configure checks added.
Attachment #440733 - Attachment is obsolete: true
Attachment #443023 - Attachment is obsolete: true
Attachment #448238 - Flags: review?(ted.mielczarek)
Attached patch Part 11: Support l10n packaging (obsolete) — Splinter Review
Attachment #442946 - Attachment is obsolete: true
Attachment #448239 - Flags: review?(ted.mielczarek)
Attachment #448239 - Flags: feedback?(l10n)
Attachment #442946 - Flags: review?(jmathies)
Asking review from Ted for the makefile hackery, Jim for the WiX markup.
Attachment #448240 - Flags: review?(ted.mielczarek)
Attachment #448240 - Flags: feedback?(jmathies)
This adds a new update manifest action "msiexec" that causes the updater to trigger the installation of a MSI package.  See http://msdn.microsoft.com/en-us/library/aa367574%28v=VS.85%29.aspx and http://msdn.microsoft.com/en-us/library/aa367571%28v=VS.85%29.aspx.  MSDN actually recommends against doing it this way for a major upgrade (3.6.x->4.0.y) but the only downside is that the upgrade is not uninstallable.  Since none of our updates are currently uninstallable I think we can live with that.
Attachment #448241 - Flags: review?(robert.bugzilla)
Eventually we'll want to turn off the CAB compression and compress the entire MSI in a 7zip self extractor, but I'm punting that to Bug 543393 so this will keep our download size from being absurd in the interim.
Attachment #448242 - Flags: review?
Attachment #448242 - Attachment is patch: true
Attachment #448242 - Attachment mime type: application/octet-stream → text/plain
Attachment #448242 - Flags: review? → review?(jmathies)
This creates a toolkit DLL that we can store our custom actions in and implements a custom action to invoke IApplicationAssociationRegistration::SetAsDefaultAppAll, the function that sets default apps on Vista+.  This will be linked into the MSI and stored as a binary resource within it and never installer or shipped alongside the actual product.
Attachment #448243 - Flags: review?(jmathies)
hg addremove
hg qrefresh

really helps.
Attachment #448243 - Attachment is obsolete: true
Attachment #448244 - Flags: review?(jmathies)
Attachment #448243 - Flags: review?(jmathies)
I've published a repo containing the patches here and the series you should apply them in at http://hg.mozilla.org/users/me_kylehuey.com/msi-patches/
Comment on attachment 448240 [details] [diff] [review]
Part 15b: Provide fully localized shortcuts

>Bug 231062: Part 15b - Provide fully localized shortcuts. r?=ted,jmathies
>
>diff --git a/browser/installer/windows/msi/Makefile.in b/browser/installer/windows/msi/Makefile.in
<...>
>+ifdef LOCALE_MERGEDIR
>+VPATH += $(LOCALE_MERGEDIR)/installer
>+endif
>+VPATH += $(LOCALE_SRCDIR)/installer

This should also append the en-US locale srcdir to VPATH if LOCALE_MERGEDIR is on, after LOCALE_SRCDIR. ifdef: merge, localesrc, ifdef: en-US. Tricky to write, sadly. There's another pattern to use $(wildcard) and $(firstword). http://mxr.mozilla.org/mozilla-central/source/toolkit/locales/l10n.mk#169 has an example, even though that's not perfect as it lacks the mergedir. The intention was good, as those files don't actually merge, but wrong as we copy en-US in that case to fix problems.

>diff --git a/browser/locales/Makefile.in b/browser/locales/Makefile.in
>--- a/browser/locales/Makefile.in
>+++ b/browser/locales/Makefile.in
>@@ -251,17 +251,18 @@ package-msi: $(MSI_INSTALLER_IN) $(SUBMA
> # Bug 231062.
> 	$(MAKE) -C ../installer/windows CONFIG_DIR=l10ngen uninstaller
> 	$(NSINSTALL) -D l10n-stage/uninstall
> 	cp ../installer/windows/l10ngen/helper.exe l10n-stage/uninstall
> 	$(MAKE) -C ../installer/windows/msi msi MSI_LINK_OPTS="-b $(_ABS_L10NSTAGE)" \
> 		MOZ_MSI_L10N=1 \
> 		MOZ_MSI_BASE_DIR=$(_ABS_L10NSTAGE) \
> 		MOZ_MSI_PACKAGE_MANIFEST=../../installer/package-manifest \
>-		MOZ_MSI_PYHEAT_OUTPUT_DIR=../../installer/windows/msi/
>+		MOZ_MSI_PYHEAT_OUTPUT_DIR=../../installer/windows/msi/ \
>+		MOZ_MSI_LOCALE_PATH=$(LOCALE_SRCDIR) \

Not sure how this blends with l10n merge?
Comment on attachment 448239 [details] [diff] [review]
Part 11: Support l10n packaging

I don't think we should glue the MSI download with RETRIEVE_WINDOWS_INSTALLER on the toolkit level, use two distinct defines for that?

Also, this assumes that Firefox_base.wixlib is uploaded to ftp, right? I lost track of what code is actually responsible for that, and if this needs tweaks to the post_upload logic of our build automation. Check in with coop or so?
Attachment #448239 - Flags: feedback?(l10n) → feedback-
Interesting work, and exciting to watch it progress, but I don't think this blocks Firefox 4 according to the current product plan.
blocking2.0: ? → -
(In reply to comment #346)
> Interesting work, and exciting to watch it progress, but I don't think this
> blocks Firefox 4 according to the current product plan.

I there anyway to change that so as that it does block Firefox 4 final? Currently due to lack of this, Firefox is shipping without enterprise support.
(In reply to comment #347)
> I there anyway to change that so as that it does block Firefox 4 final?
> Currently due to lack of this, Firefox is shipping without enterprise support.
Firefox needs more than an msi installer to have "enterprise support".
(In reply to comment #348)
> (In reply to comment #347)
> > I there anyway to change that so as that it does block Firefox 4 final?
> > Currently due to lack of this, Firefox is shipping without enterprise support.
> Firefox needs more than an msi installer to have "enterprise support".

Also, given that the work I'm doing on this at the moment is pretty much limited to dumping my patch queue and addressing review comments it's not realistic to be planning to hold a release on this.
I landed the very first patch (the cruft removal) as http://hg.mozilla.org/mozilla-central/rev/e92012b73cbb to kickstart builds on m-c.
Now a version that actually builds.
Attachment #448241 - Attachment is obsolete: true
Attachment #456806 - Flags: review?(robert.bugzilla)
Attachment #448241 - Flags: review?(robert.bugzilla)
Addressed Axel's comments.
Attachment #448239 - Attachment is obsolete: true
Attachment #456812 - Flags: review?(ted.mielczarek)
Attachment #456812 - Flags: feedback?(l10n)
Attachment #448239 - Flags: review?(ted.mielczarek)
Attachment #448242 - Flags: review?(jmathies) → review+
Attachment #448244 - Flags: review?(jmathies) → review+
Comment on attachment 448246 [details] [diff] [review]
Part 20: Defaults and shell integration

I can review syntax and such, but I think rs should look at this to be sure it matches up with the existing install.
Attachment #448246 - Flags: review?(robert.bugzilla)
Is IApplicationAssociationRegistrationSetAppAsDefaultAll the only registration interface we'll support here? I think there are cases where we only want to register for the current user.
(In reply to comment #354)
> Is IApplicationAssociationRegistrationSetAppAsDefaultAll the only registration
> interface we'll support here? I think there are cases where we only want to
> register for the current user.

That's not how IApplicationAssociationRegstration (that's a mouthful) works.  It sets per user settings only.  The "all" comes from setting the application as the default for everything it's signed up to be an application for.  See http://msdn.microsoft.com/en-us/library/bb776337.aspx for more info.
Comment on attachment 448245 [details] [diff] [review]
Part 19: Link the custom action DLL into the nonlocalized library

(In reply to comment #355)
> (In reply to comment #354)
> > Is IApplicationAssociationRegistrationSetAppAsDefaultAll the only registration
> > interface we'll support here? I think there are cases where we only want to
> > register for the current user.
> 
> That's not how IApplicationAssociationRegstration (that's a mouthful) works. 
> It sets per user settings only.  The "all" comes from setting the application
> as the default for everything it's signed up to be an application for.  See
> http://msdn.microsoft.com/en-us/library/bb776337.aspx for more info.

Ah, yes. My memory of these apis is a little gray.
Attachment #448245 - Flags: review?(jmathies) → review+
(In reply to comment #353)
> Comment on attachment 448246 [details] [diff] [review]
> Part 20: Defaults and shell integration
> 
> I can review syntax and such, but I think rs should look at this to be sure it
> matches up with the existing install.

Is there any way we could provide the option of opting out of DDE registration here? Not sure if installers have control over these "common defaults". (We were working toward removing this in the nsis install, see bug 491947.)
Comment on attachment 456812 [details] [diff] [review]
Patch 11: Support l10n packaging V1.1

Yep, addresses the comments I had, so feedback+ on this. Didn't do a review, though.
Attachment #456812 - Flags: feedback?(l10n) → feedback+
Mark this bug as "parity Chrome" and ship this baby ASAP http://www.google.com/chrome/eula.html?msi=true
(In reply to comment #360)
> Mark this bug as "parity Chrome" and ship this baby ASAP
> http://www.google.com/chrome/eula.html?msi=true

Interesting.  From poking around in their MSI it appears that they've just taken their setup EXE and wrapped it in MSI's clothing.
It's been around awhile. There is some policy support built into Chrome as well. See: http://dev.chromium.org/administrators for additional info.
(In reply to comment #361)
> (In reply to comment #360)
> > Mark this bug as "parity Chrome" and ship this baby ASAP
> > http://www.google.com/chrome/eula.html?msi=true
> 
> Interesting.  From poking around in their MSI it appears that they've just
> taken their setup EXE and wrapped it in MSI's clothing.

What's the difference in terms of administrator-experience between doing it that way and the strategy we're taking with this bug?

(In reply to comment #362)
> It's been around awhile. There is some policy support built into Chrome as
> well. See: http://dev.chromium.org/administrators for additional info.

That might serve as a good roadmap to copy-and-paste if we believe that those are the most important things for administrators. We've been looking for that information for a while, and to my knowledge (which can be happily corrected!) has not yet been obtained/documented.
(In reply to comment #363)
> (In reply to comment #361)
> > (In reply to comment #360)
> > > Mark this bug as "parity Chrome" and ship this baby ASAP
> > > http://www.google.com/chrome/eula.html?msi=true
> > 
> > Interesting.  From poking around in their MSI it appears that they've just
> > taken their setup EXE and wrapped it in MSI's clothing.
> 
> What's the difference in terms of administrator-experience between doing it
> that way and the strategy we're taking with this bug?

Some of the people on this bug who've actually done sysadmin/corporate admin work can probably answer this better than me, but if you do it their way you lose all of the repair/reinstall functionality that MSI provides you.  You also probably lose the ability to install updates without privilege (which isn't an issue for Chrome since they do per user installs anyways).
(In reply to comment #363)
> (In reply to comment #361)
> > Interesting.  From poking around in their MSI it appears that they've just
> > taken their setup EXE and wrapped it in MSI's clothing.
> What's the difference in terms of administrator-experience between doing it
> that way and the strategy we're taking with this bug?

There are various things like the repair / reinstall functionality and the inability to install updates as non-admins as mentioned before, but 9 times out of 10 the main problem is that they just don't work - Adobe have been doing this for ages and they almost always break some part of the MSI installation procedure at the same time - whether it's silent installs or performing upgrades/downgrades, or the fact that you can't create 'transforms' to change functionality very easily any more.

Speaking as a sysadmin, I just don't trust installers that wrap EXEs into MSIs, for the above reasons. It's an abuse of the process and just Not Right(TM).
> That might serve as a good roadmap to copy-and-paste if we believe that those
> are the most important things for administrators. We've been looking for that
> information for a while, and to my knowledge (which can be happily corrected!)
> has not yet been obtained/documented.

For group policy support, it's a good roadmap to start with, but that's OOS for the MSI installer and this bug (unless we want to start adding command line config options, which I don't think is a good play for a variety of reasons). Blocklist and add-on management need to be on that map as well, and we'd definitely need to mimic the ability to administratively disable private browsing that chrome has for regulatory/policy compliance.
OK, sounds like there are good reasons to continue with the investment Kyle's been making here (whew).

Kev: noted; I might chase you to put something similar together for our wiki so we're approaching this in a co-ordinated fashion. You're one of the most co-ordinated people I know! :)
(In reply to comment #364)
> You also probably lose the ability to install updates without privilege (which isn't an issue for Chrome since they do per user installs anyways).

The Chrome MSI is a system-level installation and includes auto-update (I think using a google update windows service).
(In reply to comment #368)
> The Chrome MSI is a system-level installation and includes auto-update (I think
> using a google update windows service).

MSI doesn't allow self-auto-update by programs. There is MSP file which is applied by sysadmins directly or via group policies. IIUC, MS WUS installs each fix as a separate program. I've heard MS Office can updated via WUS, but not sure since I'm personally not a w32/64 user.
(In reply to comment #369)
> (In reply to comment #368)
> > The Chrome MSI is a system-level installation and includes auto-update (I think
> > using a google update windows service).
> 
> MSI doesn't allow self-auto-update by programs. There is MSP file which is
> applied by sysadmins directly or via group policies. IIUC, MS WUS installs each
> fix as a separate program. I've heard MS Office can updated via WUS, but not
> sure since I'm personally not a w32/64 user.

Given that their MSI is just a wrapper around their EXE, I would imagine that their updater service just does the update itself.  I don't have time to verify that hunch though.
(In reply to comment #369)
> IIUC, MS WUS installs each fix as a separate program.
> I've heard MS Office can updated via WUS ...

Office can indeed be updated by WU (Windows Update).

OS updates to Win XP and prior are usually distributed as self-extracting EXEs which then run HOTFIX.EXE, the OS patch installer.  OS updates to Vista and later are usually distributed as .MSU files, which are specific to the new OS "component-based servicing stack" introduced with Vista.  Office updates are usually distributed as self-extracting EXEs which unpack an MSP and then call MSIEXEC on that.

The above applies equally to "manual" update download-and-install as well as WU in all its forms (manually driven as well as Auto Update).

(In reply to comment #364)
>>> Interesting.  From poking around in their MSI it appears that they've just
>>> taken their setup EXE and wrapped it in MSI's clothing.
>> 
>> What's the difference in terms of administrator-experience between doing it
>> that way and the strategy we're taking with this bug?
> 
> Some of the people on this bug who've actually done sysadmin/corporate admin
> work can probably answer this better than me ...

I prolly qualify for that.  I've been watching this bug with great interest because I'm a member of the target audience.  I don't have the time or experience to help with development, but I hope to help with testing.

But anyway, your analysis is spot-on.  MSI is sort of like rpm/dpkg, in that it can keep track of what's installed down to the file level.  This lets it do detect-and-repair, handle optional features, handle patching, do some reporting, etc.  You loose all that with a traditional EXE-based installer, same as you don't get the benefits of rpm/dpkg when you  do a "make install".

As a sysadmin, one thing I would particularly dislike is the ability to open up an MSI, review what it does, and generate changesets as MSTs (transforms).  For example, when I deploy the FrontMotion Firefox MSIs, I remove the desktop icon by dropping a row from the "Registry" table in the MSI.  That gets done  by an MSI I generate against the MSI using ORCA.

Wrapping an EXE in an MSI really misses the point, to the extend that I would say "don't bother".  So I agree that Kyle's efforts are well worth it to us corporate IT weenies, and I staunchly support them.
Thanks, everyone - we don't need additional advocacy for the current plan in this bug of not just wrapping the EXE in the MSI (unless it carries new information about stuff we're not seeing). I'll file another bug for the group policy and report back here when that's done.
Attachment #448235 - Flags: review?(ted.mielczarek) → review+
Attachment #448237 - Flags: review?(ted.mielczarek) → review+
Comment on attachment 456812 [details] [diff] [review]
Patch 11: Support l10n packaging V1.1

The l10n stuff needs to be redone in the post-omnijar world.
Attachment #456812 - Attachment is obsolete: true
Attachment #456812 - Flags: review?(ted.mielczarek)
Attachment #448238 - Flags: review?(ted.mielczarek) → review+
Attachment #448240 - Flags: review?(ted.mielczarek) → review+
Moving this out.  This isn't going to make 4 given that feature freeze is next Monday and a lot of this needs to be redone after omnijar.
Target Milestone: Firefox 4.0 → Future
I guess Omnijar isn't just fewer files? How much needs to be redone?

I agree that this probably won't make Firefox 4 ship, but it doesn't need to be tied to the release. We can get to it right after, and perhaps should.
The l10n repackaging is done completely differently now (in a way that is better in the end, but means that parts of this need to be redone).

We should chat sometime about when/how we want to ship this.
I see no reason why this must coincide with the release of a new version of Firefox, major or minor. However, I've spoken to quite a few network admins that clearly indicate that the one thing they need to roll out Firefox on the network is a package suitable for AD group policies. The when question therefore is ASAP! And if it can be done today for 3.6, why not ship it today?
(In reply to comment #378)
> I see no reason why this must coincide with the release of a new version of
> Firefox, major or minor. However, I've spoken to quite a few network admins
> that clearly indicate that the one thing they need to roll out Firefox on the
> network is a package suitable for AD group policies. The when question
> therefore is ASAP! And if it can be done today for 3.6, why not ship it today?

This requires significant changes to the packaging system to support it.  It's not impossible, and highly desirable, but it's not a trivial amount of effort.
Google just announced support for Group Policy Administration and MSI deployment in Chrome.
http://googleenterprise.blogspot.com/2010/12/chrome-is-ready-for-business.html
What can be done right now to help getting this bug to RESOLVED?
As an enterprise, we can't support FireFox without enterprise deployument and management tools.  This includes an MSI based installer which can be tuned to our specific needs and Group Policy templates for both enforced policies and preferences.
WRT Group Policies, it would be helpful to have specific other bugs filed on the specific preferences and policies over which you want control.  Thanks!
Please make any bugs created for GPO support block bug 267888.
Firefox 4 has been launched. Firefox 5 has been launched and big organizations complain that the new fast upgrade cycle is bad for them. Providing MSI packages is the one single thing that can be done to help them.

It is time to take a new approach to this bug. Perfect is the enemy of good. Never mind being able to install multiple versions, having the most optimum package format, etc. Let's quickly spec a bare minimum and SHIP IT and improve afterwards.
What's the point in setting a flag for something that's nowhere near ready? The tracking flag is to track landed released, not to nominate things for landing. Please be responsible with them.

In regards to 'just landing this', let's not be disrespectful of all the hard work that's gone in thus far. I'm sure that for all of the eagerness to get this landed KHuey would be happy for any assistance in the form of patches and testing. Pretty much anything and everything other than throwing toys out the pram and demanding this now.
Let's stop criticising mozilla and try to do something productive instead.

How to generate msi files

I was told it's possible to run Firefox from a network share. Yes indeed. Works fine on freshly installed XP-SP3. Need not be the portable Firefox version (that probably only affects the place for the profile).

This means that all it needs is to have the msi file create the directories and files in 
"%ProgramFiles%\Mozilla Firefox"
The windows installer will automatically take care to select the correct local path, depending on local language, boot drive etc.

The only disadvantage that I have seen so far, is that this lacks setting Firefox as default browser. That can be achieved by either adding some registry keys (surprisingly few, mostly khcr/htm,html,http,https/shell/open), or let firefox do it by running once this command:
firefox.exe -silent -setDefaultBrowser
(found here: http://kb.mozillazine.org/Setting_Your_Default_Browser)
Such a command can be integrated into a msi file (says google). Again the windows installer will automagically know from where to execute the firefox.exe file, the one that it just copied before. But this would probably harm the uninstall capability of the msi. So better set the registry keys. Find them in the Firefox source by tracking the command line option -setDefaultBrowser, or compare registry before and after the above command.

Idea: start with one of the demo msi files that microsoft provides in the platform sdk. Then let a script for windows scripting host in a loop issue the sql commands to add the directories, files and the final command to the relevant tables in the msi file. Must run on a computer where firefox has been installed by running the setup. 

So far I have personally only inserted Property rows this way. The complete msi structure is a bit more complex, e.g. all included files must be named in yet another table and referenced by that name etc., but it's managable.

This is just a sketch based on my so far limited msi knowledge. I guess a guru could hack a basic version in less than an hour. Please somebody with more knowledge comment on this. What did I miss? Then maybe we could get msi files really soon. This would for many IT persons reduce the RR pain and help make more people in corporate environments test beta and aurora versions.
Google are aggressively marketing their Business edition of Google Chrome, and sadly we are not there yet. 

http://googleenterprise.blogspot.com/2012/03/secretary-clinton-announces-state.html
https://www.google.com/intl/en/chrome/business/browser/
Tomer, so sad to hear that because Firefox wasn't mentioned :(

So, the point now is: 

What the current status? 
What's needed to speed up the process? 
Do we need to do a call for more resources in the community to help current maintainers?
I haven't been actively working on this in a while.  It needs a new owner.
Assignee: khuey → nobody
Status: ASSIGNED → NEW
Wow, in eight years there is still no MSI and no sign of a MSI being in sight for the near future.  That really is pathetic.
Comments like comment #392 are beyond belief unhelpful. If you want to complain, take it to your blog, forums, newgroups, mailists, etc. otherwise you can either help fix this bug with actual work or just hold your peace.

https://bugzilla.mozilla.org/page.cgi?id=etiquette.html
(In reply to BW~Merlin from comment #392)
> Wow, in eight years there is still no MSI and no sign of a MSI being in
> sight for the near future.  That really is pathetic.

IIUC, to all intents and purposes Firefox 3.6.27 is the current MSI. Of course it includes none of the novelties introduced in Firefox 4 or later. This of course in no way detracts from the validity of comment #393.
Probably part of why this was not fixed is that Firefox MSIs would still be useless in a corporate environment because Firefox does not behave sanely on Windows.

It saves the Firefox profile (including any possible extensions and caches) into the users's roaming profile.

As there is hardcoded limit on size of this profile on Windows which is exceeded by the default Firefox installation you would have to configure Firefox for each user as well, not just push out the binaries.

It might be possible to include a pre-created Firefox profile in the default user profile but last time I saw discussion of this it concluded Firefox does not really provide options to make this work in any sane way even if you bothered to go this route.

Note that on OS X Firefox properly saves its cache to the user's cache directory but not on Windows.
(In reply to Michal 'hramrach' Suchanek from comment #395)
> Probably part of why this was not fixed is that Firefox MSIs would still be
> useless in a corporate environment because Firefox does not behave sanely on
> Windows.

That's not the reason.  By now you should be familiar with the Mozilla process. If there is a missing feature for changing where Firefox stores the cache, submit a new bug for it.  Building an MSI is independent from that.
(In reply to Michal 'hramrach' Suchanek from comment #395)
> Probably part of why this was not fixed is that Firefox MSIs would still be
> useless in a corporate environment because Firefox does not behave sanely on
> Windows.
> 
> It saves the Firefox profile (including any possible extensions and caches)
> into the users's roaming profile.

What is wrong with saving the extensions in the user roaming profile?

As for caches, yes, these should be in the Local Settings part of the profile, but any sane Windows Admin will implement Redirected Folders (for at least Application Data and My Documents folders, but I also do the Desktop folder).

The above invalidates the rest of your complaints too.
(In reply to alanjstr from comment #396)
> (In reply to Michal 'hramrach' Suchanek from comment #395)
> > Probably part of why this was not fixed is that Firefox MSIs would still be
> > useless in a corporate environment because Firefox does not behave sanely on
> > Windows.
> 
> That's not the reason.  By now you should be familiar with the Mozilla
> process. If there is a missing feature for changing where Firefox stores the
> cache, submit a new bug for it.  Building an MSI is independent from that.

Sure it is independent but since there is no point you will have hard time convincing somebody to work on it.

(In reply to Charles from comment #397)
> (In reply to Michal 'hramrach' Suchanek from comment #395)
> > Probably part of why this was not fixed is that Firefox MSIs would still be
> > useless in a corporate environment because Firefox does not behave sanely on
> > Windows.
> > 
> > It saves the Firefox profile (including any possible extensions and caches)
> > into the users's roaming profile.
> 
> What is wrong with saving the extensions in the user roaming profile?
> 
> As for caches, yes, these should be in the Local Settings part of the
> profile, but any sane Windows Admin will implement Redirected Folders (for
> at least Application Data and My Documents folders, but I also do the
> Desktop folder).
> 
> The above invalidates the rest of your complaints too.

It does not. Redirecting AppData is not always practical, and other browsers work fine without redirecting AppData whereas Firefox reliably fails.
(In reply to Michal 'hramrach' Suchanek from comment #398)
> (In reply to Charles from comment #397)
>> (In reply to Michal 'hramrach' Suchanek from comment #395)
>>> Probably part of why this was not fixed is that Firefox MSIs would still be
>>> useless in a corporate environment because Firefox does not behave sanely on
>>> Windows.
>>> 
>>> It saves the Firefox profile (including any possible extensions and caches)
>>> into the users's roaming profile.

>> What is wrong with saving the extensions in the user roaming profile?

You didn't answer this question...

>> As for caches, yes, these should be in the Local Settings part of the
>> profile, but any sane Windows Admin will implement Redirected Folders (for
>> at least Application Data and My Documents folders, but I also do the
>> Desktop folder).
>> 
>> The above invalidates the rest of your complaints too.

> It does not.

It does if you implement Redirected Folders.

> Redirecting AppData is not always practical,

In an enterprise environment, it is not only almost always practical, it is almost always strongly recommended, unless you have *very* unusual circumstances...

> and other browsers work fine without redirecting AppData whereas Firefox
> reliably fails.

It is very uncommon for Firefox cached data to be all that much - I believe the default is 50MB? That is not very 'heavy' even if it is in the roaming profile, and shouldn't cause any problems whatsoever, unless your environment is very poorly configured and/or the server that stores the roaming profiles is extremely underpowered or over-utilized.

I've *never* seen a problem with Firefox (because its cache is limited to 50MB by default), but have definitely seen some problems with huge IMAP stores in *Thunderbird* profiles that were getting stored in the roaming portion in environments that were not using Redirected Folders.
Guys, while the profile conversation is interesting, it is has no bearing on fixing *THIS* bug. There are 185 people CCed to this bug - please show courtesy to them by not adding more off-topic comments to this bug. While it's obvious there is desire to see this bug fixed, burying the useful information in off-topic comments is only making it harder for any interested parties to do so.

If there isn't a bug for the cache issue already, please file one and mark it as blocking bug 720362.
Hello, everyone - 

My name's Mike Hoye, and I run a startup called Bespoke I/O. We've built a tool that customizes .MSIs of Firefox (and, in beta now, Thunderbird) for enterprise deployment. Preconfigured bookmarks, settings and upgrade behavior, preloaded certificates and extensions, lots of other options.

It's a paid-subscription-based service, but in the spirit of Mozilla's mission we're offering it at an deeply discounted rate to publicly-funded colleges and universities, and to K-12 (or your regional equivalent) public schools and public libraries for free.

We have some demo .MSIs available here: http://bespokeio.com/msi/

If you'd like to take a look at it to see if it fits your needs, please contact me at mhoye@bespokeio.com and I'd be glad to set you up with a trial account. You can see more information about it at http://bespokeio.com and feel free to contact me directly with any questions.
(In reply to Mike Hoye from comment #401)
> It's a paid-subscription-based service, but in the spirit of Mozilla's
> mission we're offering it at an deeply discounted rate to publicly-funded
> colleges and universities, and to K-12 (or your regional equivalent) public
> schools and public libraries for free.

Hello,

This is not the correct place to advertise your product, please use the enterprise mailing list. This bug has a lot of people subscribed that expect news about this development, not a discussion.

Also when I read "in the spirit of Mozilla's mission" I was expecting you were open sourcing the code and sharing with the global community of volunteers ;)

I don't want to be mean, it's just my opinion.

Regards.
If you liken java api's to this msi build source from chromium, getting firefox to install from an MSI should be a snap. http://code.google.com/p/omaha/source/browse/trunk/enterprise/installer/
Why not wrap the exe from an administrative install into an MSI that supports uninstalling and having itself not list in Uninstall? At such a time that a new MSI supports customization, the wrapping of the administrative install can be abandoned?

Everyone can have their cake and eat it too. Now.
You don't wrap random junk in MSI but the other way around. That's what MSI is about.

Microsoft invested some effort into making MSIs work, and by now they pretty much do.

So when you get a MSI it very likely works, the same way all other MSIs do and you don't have to deal with that broken junk of all those other installers. Not to say that some of them don't work to some extent but still have to research every single one. And most of them are broken anyway, even if  you look up all that is known about them you can't install software with them.
(In reply to havealoha from comment #404)
> Why not wrap the exe from an administrative install into an MSI that
> supports uninstalling and having itself not list in Uninstall?

Because that gets us no advantages over just scripting a silent install from the EXE.  Those of us who want MSI want easy upgrades most of all, and also easy modifications, easy uninstall, easy tracking of what's installed, and so on.  There's no point to just wrapping an MSI around what we have now.
Most of us with 10ESR want a wrapped MSI for using a GPO to push out  silently from the administrative install. This will support minor updates and uninstall and upgrade. New major (17)? New policy. Problem solved.
The original URL in the bug appears to be dead. I believe this one is the new location of the Windows Installer documentation:

http://msdn.microsoft.com/en-us/library/windows/desktop/aa372845.aspx
(In reply to travis from comment #408)
> The original URL in the bug appears to be dead. I believe this one is the
> new location of the Windows Installer documentation:
> 
> http://msdn.microsoft.com/en-us/library/windows/desktop/aa372845.aspx

What you said seems correct.
Why has this not been prioritised? Chrome is overtaking Firefox, and this is probably why! They have a truly awesome msi installer that just works, as well as a really nice set of Group Policy templates.

As a sys admin, I'm forced to use abandon FF in favour of Chrome.
"Why has this not been prioritised? Chrome is overtaking Firefox, and this is probably why!"

All the available data I have, and I have a fair bit, says otherwise.

Hi, everyone - Mike Hoye here again, of Bespoke I/O. The project I mentioned earlier, the MSI generation web service, has been shut down largely due to a lack of interest. Even so, though, some good people did a lot of hard work to get it going, and I didn't want their contributions to wither on the vine. 

The source code to the web service, along with instructions for setting it up on a Fedora-based VM, is here:

https://github.com/mhoye/Bespoke_IO

It works with stock Mozilla-provided binaries and gives you a fairly fine-grained way of customizing Firefox (4.0 or later, including ESR) and Thunderbird (10.0 or later, including ESR) for sane, sensibly-managed corporate deployment. Have fun with it. Email me if you have any problems setting it up, and I'll try to help.

I'll leave it to some authority at Mozilla to determine if this is grounds for closing the bug.
Hi Mike,

Would you might to send an announcement to Enterprise mailing list?

https://mail.mozilla.org/listinfo/enterprise

Is there any place to see some images or how is it working in a more visual way? It would be great for most of us to see it in action :D
"Why has this not been prioritised?"

Please do not rush the Firefox developers.
I chose my words carefully. Prioritised doesn't mean "rush it", it means put it up higher on the to-do list. I.e. put a hold on new features and get this done first.
FYI I have now achieved automatic conversion of firefox-setup.exe into a native MSI-file. With native I mean that it does *not* run custom actions, everything is done by windows-installer.

It's trivial to insert config changes, if you know how to do it on a locally installed Firefox. My version disables auto-update, and declares Firefox as default browser for all users.

The same method works with Thunderbird as well.

So far tested with german Firefox 10 in WinXP, but should work with any version in any language, and in Win7 as well, because windows-installer adjusts everything to local conventions. Except the conditional settings that Mozilla has in their installer. I hope they are only for changes introduced with Win8. If you have Win8, you must probably use a Win8-PC to create a separate MSI-file, and use that for Win8-PCs. I don't expect that Win7-64 needs a separate MSI-file as well, but haven't tested yet.

I plan to document everything on a web page within the next days, and offer a zip-archive with all tools that you need to run this yourself. I don't plan to offer MSI-files, but I suggest Mozilla should routinely run my scripts for each esr-version and every language, it can easily be made fully automated. I don't know how to build localized MSI-files with all languages in one file, and I refuse to dig any further into the WiX toolkit. Also I had to remove one localized string from the registry, because WiX refuses to read files with german umlaut characters.
Great to hear that hartnegg :)

And I think the strategy here should be provide a basic official MSI and then improve it over time, but at least have this option for enterprise users now.
I agree that my script cannot be counted as fix for this bug.

It can make life easier while waiting for the real thing.

Here's the first version that passed a basic test:
http://www.klaus-hartnegg.de/gpo/msi_firefox_wix.html
I already do this using Flexera's AdminStudio Repackager. I created an .msi yesterday and it took me about 20 minutes to capture the installation. It's actually very simple and straightforward. Feel free to shoot me questions about the process.
dps002, a link with instructions would be nice :)
Unless the NSIS-based process for installation, removal, and self-updating from a limited user account will be dropped completely, why duplicate functionality?

The attachment that accompanies this comment contains a WIX 3.7 file that will:

Install Firefox 25.0.1 (x86 en-US) in unattended/silent mode
Allow modification of the installation parameters using an INI file (which is itself generated from MSI properties, so systems administrators can create MSTs for use in their environments)
Remove the non-MSI entry from the list of installed programs
Run Firefox's uninstaller when removing the package
Perform upgrades/repairs by running the uninstall process followed by running the installer.
Use WIX variables for details such as the language, version, platform, package/upgrade codes, and installer file, to ease turning the file into a template.

Thoughts?
Is it possible to see the latest versions of the files that were created so far?
http://lxr.mozilla.org/seamonkey/source/browser/installer/windows
does contain a directory named msi, but the files there contain only a fraction of the data from the patches here.

One of the last comments from Kyle Huey refers to Omnijar and says 
"The l10n repackaging is done completely differently now".
Is the new versus old method described somewhere?

The l10n part is the thing from here that I have understood least. There must currently be a script that creates one exe installer per language. I cannot find anything looping over languages. Is the plan to integrate the msi creation into the same script? Or shall this become a different script, that has to also go through the languages itself?

Another issue that I see is that the list of files is generated dynamicly, but the registry keys seem to be hardcoded. The source for the nsis installer contains some conditional code for different versions of windows. Is this a problem?

Overall this appears to have already come quite far. Which steps other than omnijar-adaption are missing to finish this (assuming that active directory integration is a separate project)?
(In reply to Ariel Poliak from comment #420)
> Created attachment 8340663 [details]
> WIX 3.7 installation wrapper

Wrapping can be a useful trick. But it is a hack. It is less reliable and less flexible.  See also previous comments, for example comment 406 and comment 365. Also it appears that most of the work to do it as true msi-file has already been done. I wonder what precisely is missing.
Idea/suggestion regarding the l10n issues caused by the omnijar change:
It seems to be planned anyway to *not* replace the exe-installer anytime soon. In this case the file selection for the msi could be made by creating the exe-installer first, then simply unzip that into a temporary directory, and tell WiX to just grab all files from there. This is how my msi-script has survived mostly unchanged both directory structure changes between Firefox versions 10 and 24. In this case there would be no need to duplicate the file selection process, and to adapt it each time something changes. Only if the set of registry keys change, such a change would have to be replicated in the config file for WiX.

The only features that I know are missing in my script are
- set slightly different registry keys in XP, Vista, 7.
- undo the default browser setting during uninstall.
> It seems to be planned anyway to *not* replace the exe-installer anytime soon.
> In this case the file selection for the msi could be made by creating the exe-installer first,
> then simply unzip that into a temporary directory, and tell WiX to just grab all files from there.

I think that's a good approach. You'd need to remove the uninstaller, though.

FWIW, this is how the l10n build works (worked?) as well: Take the English installer exe, unpack it, replace locale files, repack.
Hello, i am new here and would like to contribute to this. 

But i see the bug was raised in 2004 and no updates since 2014. Can someone please let me know what is the current status on this requirement ?
(In reply to monish.k from comment #425)
> Hello, i am new here and would like to contribute to this. 
> 
> But i see the bug was raised in 2004 and no updates since 2014. Can someone
> please let me know what is the current status on this requirement ?

I would like to know too, 12 years old, it´s a teenage bug.
And I would like to suggest that msi format include a option to not install pdf viewer too, because pdf viewer not respect PDF rights (bug id 792816) and Its necessary in all most company that use ISO Quality in Brazil.
As my team is responsible for the updater, and we've had several conversations about this topic (and this bug), I'm closing this as WONTFIX.

There are a variety of solutions for those who need verbatim MSI (some mentioned in these comments, as well as our own mhoye's bespoke.io -- see http://exple.tive.org/blarg/2011/03/22/here-we-go/).

For everyone's edification: the common perception is that MSI would automatically provide everyone with what they need. That is only partly true: it is not automatic -- specific functionality would need to be both created and supported, and that could (and probably would) vary for each consumer of the common use case, which is handling enterprise deployments. We are not well suited or staffed to keep up with individual enterprise solutions in perpetuity.

[Accordingly, you can refer to bug 1034988 for a kind of contract that we could conform to in the future.]

At any rate, keeping this bug open ongoing doesn't help anyone, and only serves to make people wonder what our intentions and opinions are. If anyone can make a case for keeping this bug open, I'm all ears; in the meantime, it's closed.

---

Four days ago, I mistakenly put this comment on and closed bug 267888, the Group Policy bug. There was a comment there from Vadim Rapp about using transforms to apply per-use changes to a base MSI package, who also volunteered to take this on.

While my team is not going to work on this, I would encourage anyone who chooses to do so to read all the comments above as well as those on bug 267888 and bug 1034988 before determining if just having an MSI is going to solve the issues for everyone who is requesting an MSI or just a subset. That determination should define whether the MSI is something Mozilla incorporates or is just an external option.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → WONTFIX
FTR, a good argument would be that Chrome supports it: https://enterprise.google.com/chrome/chrome-browser/

As a former Windows domain admin, I think a basic MSI package is a big plus, event just for the purposes of allowing easy GPO distribution and update. Eventually, allowing customizations would be nice as well, but currently it is difficult to deploy Firefox in a Windows domain environment.
(In reply to David Durst [:ddurst] from comment #427)
> At any rate, keeping this bug open ongoing doesn't help anyone, and only
> serves to make people wonder what our intentions and opinions are. If anyone
> can make a case for keeping this bug open, I'm all ears; in the meantime,
> it's closed.

Just as Nochum said, deploying Firefox in a Windows Domain environment is extremely difficult without an MSI. It wouldn't even really need any features beyond simply installing/updating the downloaded Firefox version in the background, but a lack of that has made the job of many domain admins much, MUCH more difficult. Instead, many of us just switch to installing Chrome on our clients computers instead of Firefox.

Until there is an easy way for Firefox to be deployed in a a Windows enterprise environment, even if it's a very basic deployment, Chrome will become the default for most businesses.
For everything that's been said in this bug about "what businesses want", it turns out that an excellent indicator of what businesses actually want is what they're willing to spend money to have. 

A number of people have tried to make a modest living off Firefox enterprise support, including myself.  And while I can't speak for the others, I can tell you for sure that however many people _said_ "businesses need" the toolchain and service I built to solve this precise problem, a much lower number of actual businesses were willing to so much as trial, much less pay for, that service. 

That number was exactly zero.

Which, it turns out, is also the number of people who have ever asked me for help standing up an in-house version of that toolchain, which has long been free-as-in-everything except for the hour or so it takes to set up on a vanishingly small VM. 

It did a bunch of other neat things, too. Preloaded extensions. Preloaded certs for the Sarbanes/Oxley crowd who need internal traffic monitoring tools. Preconfigured homepages, preloaded bookmarks. Preconfiguring a Firefox Sync server, so you could deploy those internally too. 

As far as I can tell, nobody's ever take that hour to set it up.

God, I don't know why I still follow this bug.

That software is almost certainly moribund now - it might work, but likely won't play well with modern cryptosigned addons and definitely not with modern Firefox Accounts servers. But I'd appreciate it if we could all stop talking about "what businesses want" in this bug like that's the least bit true, or would sway anyone's decision if it was. Businesses don't want this enough to spend more than zero dollars on it. Sysadmins at those businesses don't want this enough to spend an hour of their time making it work. 

If you disagree with that assessment, high five, but I've got years of my life saying otherwise. And while I don't regret giving this a shot, I definitely regret the time I've spent watching people in this bug ask for the same things I built for them, over and over again since.
(In reply to Mike Hoye [:mhoye] from comment #430)
> Businesses don't want this
> enough to spend more than zero dollars on it.
Exactly, which is why we'll switch to Chrome before we waste both time and money on some third party software we can't trust.

Why spend money on something that you can get for free, quickly, and with much less risk? It'd be stupid to.
I subscribed to this ticket several years ago when I was running IT for a small school. I no longer do so, but others on this ticket might find my experience instructive…

I initially installed Firefox manually on all ~50 computers and taught everyone to use it. As time passed, it became harder to find the time to manually update Firefox everywhere, so I found a source for prebuilt MSI installers, and started using them. However, it was always a little behind.

Eventually, Chrome became popular, and since it was trivial to install the MSI, I added that. Eventually, because of the overhead, I had to choose one browser to maintain and train people on, and I chose the one with the MSI and Group Policy support built in, namely Chrome. No computer in the school has Firefox on it anymore, to my knowledge.

At this point, it isn't a matter of businesses wanting Firefox enough to pay for it. For MSIE/Edge and Google Chrome, MSI and Group Policy are table stakes. Firefox not having it in core just means businesses don't even consider it. There are exceptions of course, but I find this to be mostly true.

Thanks for all of your work on this, and the rest of Firefox! I understand a lack of resources, and I trust that you all will make the best prioritization decisions you can, regardless of the final status of this ticket.
(In reply to Mike Hoye [:mhoye] from comment #430)
> For everything that's been said in this bug about "what businesses want", it
> turns out that an excellent indicator of what businesses actually want is
> what they're willing to spend money to have. 

I don't think that this is true any more.

Exhibit 1: The most successful web services, including ones heavily used by business users, are ones which do not get any money from their users. And an increasing number of popular web services are blocking users who install adblockers.

Exhibit 2: There are companies who do get payed big money for the online use of their products. But these are mostly ones which already had a large user base for these products before Internet use got widespread. Examples: Microsoft, Sony, Bloomberg, Elsevier. And even they struggle to adapt their business model.

I even reject the often cited example of Redhat. Because their customer number did not explode proportionally with the number of Linux users, which is now several billion. Yes this comparison is unfair, but it is relevant. Both personal and business users do in most cases not pay for getting support. There are a lot of chromebooks in businesses, because they do not need expensive support.

Microsoft not offering MSI files for all their products is not a good counter example. They do go great length to prodivde automatic deployment, automatic updating, and remote configuration of their products, eventhough they are often deployed together with the hardware and then not touched anymore. 


By the way another reason for offering MSI files is that of the more than 1 billion PCs being in use worldwide, somewhere around 45 percent stand in business environments. The two biggest competitors of Firefox do have a huge number of business users. Not having them is not the only reason why Firefox has fewer users, but it is a large part of it.
You could also look at this from a pure business perspective for Firefox.
There's little doubt Firefox could gain users by adding this feature, maybe more so than any other single feature in itself. Now the only question is how many. One million world wide is a VERY conservative estimate, it could probably be tens of millions. Surely money wise that pays off by itself to implement.
Thanks for all the feedback. This is appreciated!

I am available by email (s@mozilla.com) if you have different feedback but don't want to share it publicly.
Please check this page and fill it now: https://qsurvey.mozilla.com/s3/Enterprise-2017 - this is the MSI/MCX/autoconfig package survey!
Copy of an email that I sent to the enterprise:

We made a simple MSI wrapper of Firefox 52.2.0 installer. It is available on this URL:

https://drive.google.com/file/d/0B9mXnKFQuivXd19YT1NhNHpVRkk/view

sha256: e9b31370ee13ffd74d5cb14143aa49aff353dac7d6fe707eb53460fc7482aa31

Now, we would appreciate your feedback on this implementation.
Please use this form to tell us what you think:

https://docs.google.com/forms/d/e/1FAIpQLSevoDXIWq4d5vbBR41rQSNhZiInw2CNqYBwQUbnlj1xP5wEdw/viewform

Please note that this is a proof of concept and that should not be used in production.
Thanks for the efforts. But I'm actually not able to test a proof of concept as it should be.

As for my part is clear, that an MSI wrapper makes only sense for the ESR-Versions. Admx should be implemented only for some "easy Features" (No automatic Update, no Know-your-rights-Popup, fix a start page, accept automatic AD-Login, etc).
Thx.
Looks like this bug is attracting spam comments. Going to nip that in the bud with restrict comments. If you have a useful comment contact me and I will likely grant editbugs to your account or go through the formal process at https://bugzilla.mozilla.org/page.cgi?id=get_permissions.html .
Restrict Comments: true
Per bug 1426870, I'm asking :Felipe if they will accept a patch. 

:Felipe, if a patch is acceptable to you as part of Enterprise work, please reopen set the priority on this bug to P5. If your team would like to take on this work, set to P3 or higher.

Thank you!
Flags: needinfo?(felipc)
(In reply to Emma Humphries, Bugmaster ☕️ (she/her) [:emceeaich] (UTC-8) needinfo? me from comment #445)
> Per bug 1426870, I'm asking :Felipe if they will accept a patch. 
> 
> :Felipe, if a patch is acceptable to you as part of Enterprise work..

The topics are related on a general sense, but the enterprise policy work that I'm involved is independent.

The installer team has been looking into MSI work, but my understanding is that MSI is generally incompatible with how our update system and multi-install support works, so it's not just about accepting an MSI installer
Flags: needinfo?(felipc)
See Also: → 1553076
You need to log in before you can comment on or make changes to this bug.