186703 - Update mozilla installer to work with GRE

Status: VERIFIED FIXED

(SeaMonkey :: Installer, defect)

Description

[Sean Su]

Mozilla installer needs to be updated to leverage GRE already on the system.  If
it's not there, install a copy of GRE.  There are 2 parts to this:
 1) Update the installer code to be able to load xpinstall engine in GRE
 2) Update installer build scripts to build mozilla and gre together.

GRE installer needs to be updated to package up and install xpcom.xpi.  This is
required for mozilla installer to work at all.

Comment 1

[Sean Su]

Attachment: patch v0.1

This patch is not complete yet.  Things that this patch contains:
 * Win32 installer now supports using either xpcom.xpi or the GRE installer
   to install mozilla/ns product.
 * Current GRE build process now uses and installs xpcom.xpi.  It used to
   only use xpcom.xpi for installation purposes without actually installing
   it.	Installation of xpcom.xpi is now required because the win32 native
   installer will use the existing GRE on the system to install files.
 * New .pl scripts to generate staging areas for GRE and Mozilla.  Release
   eng. should no longer call pkgcpy.pl directly.  'make_stage.pl [product]'
   should now be called (where currently supported products are 'gre' and
   'mozilla').

Things that still needs to be done:
 * fix mozilla installer build process (update the make*.pl files) to
   build GRE and Mozilla installers together.  Also make sure they are
   crossplatform and can be run using cygwin perl.  I think I've encountered
   a bug with the cygwin perl (as compared to ActiveState perl).  If I find
   any more info on it, I'll post it here.
 * add more comments to the new code.

Comment 2

[Chak Nanga]

Since i'm not qualified to comment on the installer changes, i'll just comment
on the changes to "embedding/config/basebrowser-win". 

Making changes to basebrowser-win (by commenting out dll inclusions) will break
the nightly base embedding distributions. A lot of users in the Mozilla
community use the base embedding distributions pretty regularly to test the
embedding functionality. With the proposed changes they will miss key dlls such
as the xpcom.dll etc. and will be unable to run their embedding apps.

Comment 3

[Sean Su]

chak, I commented them out of:
 embedding/config/basebrowser-win

and moved them into:
 xpinstall/packager/xpcom-win.pkg

I've also updated 'xpinstall/wizard/windows/builder/build_gre.pl' to use
'xpcom-win.pkg'.  'xpcom-win.pkg' will get installed by the GRE installer.

I assumed that the nightly embedding distributions are distributed only with
installers and not outside of the installer scope.  If this is an incorrect
assumption, please let me know what other distributions there are because they
will need to be udpated to use 'xpcom-win.pkg' as well.

Comment 4

[Chak Nanga]

The embedding/basebrowser-[mac|unix|win] files are the way folks build a
base/minimal embed distributions which are needed for a particular platform -
there's no installer. All one has to do is to run "make" in the embedding/config
directory and it will generate a "embed" dir under "dist" for them which they
use as a base embedding file set.

Removing some of the files from basebrowser-win will break this functionality. 

Comment 5

[Sean Su]

Attachment: patch v0.2

Updated patch to not use the following to build the GRE installer:
  basebrowser-win
  gre-win

GRE installer build process will now use:
  basebrower-installer-win
  gre-installer-win

This is because the current GRE build process is using the package lists
incorrectly.  The package lists were meant for creating a staging area for
building the installers, not to create a dist area.

The new set of package lists will only be used for installer purposes while the
original package lists will be used as they were, to create the embedding dist
area.

Comment 6

[Chak Nanga]

Initially, when Curt worked on the GRE installer he had a set of manifest files
for the GRE installer - eventhough the exact same file list was available in
gre-win/basebrowser-win. This caused missing file issues with the installer
since we have duplicate set of manifests - folks now had to update two manifest
files instead of one.

That's the reason why currently the GRE installers and the nightly embed dist
builder scripts share gre-win/basebrowser-win.

By going back to separate manifests we'll be re-opening this issue.

On a general note, why do we need to change the way the GRE installer is
currently being built?

1. We have a working process by which GRE installers are being built.
2. We have a working process by which Mozilla installers are being built.
3. Is'nt all that's needed here is to modify the Mozilla installer:
    - By adding some logic to invoke the GRE installer if a GRE is not  
present[The MfcEmbed installer follows this process]  
   - Make some additional changes to the Mozilla packaging manifests to remove
files from it which are already being installed as part of the GRE installer process

Comment 7

[Sean Su]

There is already duplication of package lists.  The embed build's package list
contains a subset of mozilla's package list.  If the mozilla package list is
changed, the embed's package list could be out of sync (depending on which files
were changed).

The main problem here is that the embed build process is using the package lists
to create its dist area.  The package lists were not designed for this purpose.
 The embed build process should be using each components' Makefile.in to deliver
it to dist/embed (or dist/gre) in addition to dist/bin.

For example, to deliver xpcom.dll to dist/embed, xpcom/build/Makefile.in should
have been modified to deliver the .dll to both dist/bin and dist/embed.

I had to change the way GRE is built for a couple of reasons:
  * I'm trying to have mozilla's installer build with the GRE installer.  Right
now the GRE installer is dependent on the mozilla installer build process, which
is backwards.
  * The download size of the GRE installer is larger than it should be because
it's packaging several files more than once.

(adding dveditz to the cc: list for input on this issue)

Comment 8

[Sean Su]

adding some release people to the cc: list because the patch to this bug will
affect them greatly.

Comment 9

[Sean Su]

Attachment: patch v0.3

updated patch.	still not ready for testing quite yet.	forgot to mention that
my patches also fix bug 173122 - make install scripts work with cygwin perl.

Comment 10

[Jon Granrose]

sounds like this will require changes to the automation.

Sean - please work with Leaf to make sure those changes go in at the same time
as your patch so we don't break the verification builds.  We should probably
clobber dist and do a test build on the build systems after these changes go in.
 If we want to be really safe, we should do it before.

Comment 11

[Chak Nanga]

Cc:ing Charles and Jimmy so they're in the loop as well. 

Basically, Charles has to verify that the nightly GRE/MfcEmbed installers are
kosher once these changes land.

Comment 12

[Sean Su]

Attachment: patch v0.4

Updated patch.	Still not the final patch.  For those inside the firewall,
there is a test build off of this patch at:
  http://jazz.mcom.com/users/ssu/publish/mozilla-gre/2002-01-08/

Comment 13

[Sean Su]

Attachment: patch v0.9

This patch is pretty much ready for review.  I'll be working with sgehani and
dveditz on the reivews.

For those inside the firewall, an optimized win32 mozilla test build of this
patch can be found here:
  http://jazz.mcom.com/users/ssu/publish/mozilla-gre/2002-01-14

This build has a test patch that allows mozilla.exe to locate and run with a
GRE installed not in the same dir as itself.  This test patch is not part of
this fix, but just to show that the installer patch is doing what it's suppose
to do.

Comment 14

[Sean Su]

Attachment: patch v0.9.1

Comment 15

[Sean Su]

Attachment: patch v1.0

I've found a few more problems with the previous patch.  This patch addresses
problems with using cygwin perl, download archive feature (stub installer), and
gre build integration.

I've tested this for both mozilla and ns builds.
The ns patch is attached here: http://bugscape.mcom.com/show_bug.cgi?id=21994

Comment 16

[Sean Su]

Attachment: patch v1.1

just added code to offer [GRE PATH] support to config.ini.

Comment 17

[Sean Su]

Attachment: (real) patch v1.1

sorry, forgot to diff the previous v1.1 patch with -N

Comment 18

[Daniel Veditz [:dveditz]]

Don't know why this is "critical", I don't see any crashing or dataloss
currently. Maybe you meant high priority, but priority is currently set to '-'

Comment 19

[Daniel Veditz [:dveditz]]

Comment on attachment 112175 [details] [diff] [review]
(real) patch v1.1

>Index: xpinstall/packager/StageUtils.pm
>Index: xpinstall/packager/make_pkgs_from_list.pl

You need comments, at least at the file and routine level. Perl is cryptic
enough, give the next guy a fighting chance. I'm not talking a lot, but
something that explains how the routines relate to each other or how you use
them. The one-liner in makeall.pl, for example.

>+sub PrintUsage

Your usage printouts do a good job with the args, but they say nothing about
what the programs themselves are supposed to do.

>Index: xpinstall/packager/windows/config.it
>===================================================================
>+C1=Component GRE
>+C2=Component Navigator
>+C3=Component PSM

I thought you said PSM was now part of the GRE?

What happens to the commercial build after this whackage?

sr=dveditz

Comment 20

[Sean Su]

I've addressed dveditz's comments.
* added more comments to the new .pl scripts
* removed Component PSM from the config.it files.

new patch coming up

Comment 21

[Sean Su]

Attachment: patch v1.2

this new patch addresses comments from sgehani and dvetiz.

Comment 22

[Sean Su]

Comment on attachment 112263 [details] [diff] [review]
patch v1.2

moving sr=dveditz to here and got verbal r=[e-mail]

Comment 23

[Sean Su]

patch checked in.  marking this fixed.

Comment 24

[Mel Reyes]

The latest daily builds are installing but are not running, I'm rolling back to 
yesterdays builds.  Mozilla runs and then just exits.  This is one of the 
potential culprits, because I download the 6am 0121 build and it failed after 
that and I notice the GRE stuff had been added.

Help????

Comment 25

[Chak Nanga]

I ran the latest Mozilla  installer from
ftp://ftp.mozilla.org/pub/mozilla/nightly/latest-trunk/mozilla-win32-installer-sea.exe
and noticed the following:

It does not install the GRE. The C:\Program Files\mozilla.org\GRE\1.3b dir has
just two files - "install_status" and "install_wizard". 

Comment 26

[Sean Su]

I'm on it.

Comment 27

[Sean Su]

found the problem.  patch coming up.

Comment 28

[Sean Su]

Attachment: patch to fix GRE not installing

Comment 29

[Sean Su]

patch checked in. a=lpham

Comment 30

[Mel Reyes]

Ok, not sure if this patch is in the latest build yet, but besides increasing 
the download by almost 2 megs, can someone point out what the GRE benefits are 
( you can do this offline if you'd like )

Also the standalone GRE 1.3.0 installer does not install either.

Just waiting to see if the latest build will be refreshed so that I can 
download and install, thanks.

Comment 31

[Mel Reyes]

Attachment: xpcom.dll error installed latest daily build (w/ gre)

Thanks for the recompile on the daily build, here's a followup error, talkback
id TB16484050W.

I've attached screenshot of one of the xpcom.dll entry point errors that came
up.

So that that both moz and gre are installed, must say, not overly excited about
the duplicate dll's, etc.  Is this a proposed feature set to have the dll's in
two different directories?

Comment 32

[Sean Su]

mel, did you install mozilla into a clean dir?  I think the duplicate files are
because you might have installed ontop of a previous mozilla build which did not
have its GRE files elsewhere.

Sounds like the installer will need to deal with this upgrade scenario.  bug
190144 filed.

Comment 33

[Mel Reyes]

Sorry for the spam:

1) but uninstalled and did a clean install and now I don't get the xpcom error,
now I get a bad image on CustomImages.dll on startup and shutdown.  There should
be some kind of warning or check to do the one time uninstall for you.

2) https sites are not working now

3) JavaScript Debugger venkman generates an error and doesn't load

Comment 34

[Peter Van der Beken [:peterv]]

This caused bug 191996 by removing dom_xpath.xpt from packages-win but not
adding it anywhere else. Hopefully you didn't forget to add any of the other
files you removed from packages-win!

Comment 35

[Grace Bush]

verified 2003030708