955017 - Re-sync installer with comm-central

Status: RESOLVED FIXED

(Instantbird Graveyard :: Other, defect)

Description

[Patrick Cloke [:clokep]]

*** Original post on bio 1587 at 2012-07-12 12:23:00 UTC ***

After we next update Mozilla, we should resync our build scripts with <whatever version of comm-central that matches>. Hopefully this will ensure we stay better in line with the Thunderbird build process and should make porting any changes to build scripts a bit easier.

(This doesn't really need to wait until we update Mozilla again, but it seems to make sense to...)

Comment 1

[Florian Quèze [:florian]]

*** Original post on bio 1587 at 2012-11-05 23:34:35 UTC ***

Realistically, this isn't going to be ready for 1.3, moving to 1.4-wanted.

Comment 2

[Patrick Cloke [:clokep]]

*** Original post on bio 1587 at 2013-04-09 14:49:53 UTC ***

s/1.4-wanted/1.5-wanted/ in the whiteboard for bugs that probably won't happen before 1.4 is released.

Comment 3

[Patrick Cloke [:clokep]]

*** Original post on bio 1587 at 2013-05-31 22:02:18 UTC ***

This was done as part of bug 955394 (bio 1956) (specifically http://hg.instantbird.org/instantbird/rev/9ed5d3abbe8a).

Comment 4

[Patrick Cloke [:clokep]]

*** Original post on bio 1587 at 2013-06-01 14:24:31 UTC ***

I'm going to reopen this as we still need to sync instantbird/installer and instantbird/locales

See http://log.bezut.info/instantbird/130531/#m742 for more information.

Comment 5

[Florian Quèze [:florian]]

*** Original post on bio 1587 at 2013-06-01 18:35:21 UTC ***

It would also be very useful to sync build/macosx/universal/flight.mk, as http://hg.mozilla.org/comm-central/rev/d394e3684954 is the fix for our current Mac onCommit failures :-).

Comment 6

[Patrick Cloke [:clokep]]

Attachment: WIP v1

*** Original post on bio 1587 as attmnt 2532 at 2013-06-28 21:00:00 UTC ***

This is my attempt at syncing some of the directories. This specifically does NOT update removed-files.in, by the way.

Comment 7

[Patrick Cloke [:clokep]]

Attachment: Diff with c-r (TB 21)

*** Original post on bio 1587 as attmnt 2533 at 2013-06-28 21:01:00 UTC was without comment, so any subsequent comment numbers will be shifted ***

Comment 8

[Patrick Cloke [:clokep]]

*** Original post on bio 1587 at 2013-06-29 12:24:41 UTC ***

port 'Bio 797452 Intermittent build/macosx/universal/unify failure: "copyIfIdentical: files differ" - sort manifest files for comparison when comparing them.'

http://hg.instantbird.org/instantbird/rev/7be18e1919f7

Comment 9

[Patrick Cloke [:clokep]]

Comment on attachment 8354300 [details] [diff] [review]
WIP v1

*** Original change on bio 1587 attmnt 2532 at 2013-07-01 14:28:31 UTC ***

make package failed when I tried this.

Comment 10

[Patrick Cloke [:clokep]]

Attachment: Part 1 v1

*** Original post on bio 1587 as attmnt 2560 at 2013-07-09 15:29:00 UTC ***

Port some c-c changesets we had not yet ported.

Comment 11

[Patrick Cloke [:clokep]]

Attachment: Part 2 v1

*** Original post on bio 1587 as attmnt 2561 at 2013-07-09 15:31:00 UTC ***

This sync instantbird/installer/Makefile.in. It does NOT include the changes to instantbird/installer/windows, these need to be debugged.

Comment 12

[Patrick Cloke [:clokep]]

Comment on attachment 8354300 [details] [diff] [review]
WIP v1

*** Original change on bio 1587 attmnt 2532 at 2013-07-09 15:31:50 UTC was without comment, so any subsequent comment numbers will be shifted ***

Comment 13

[Patrick Cloke [:clokep]]

*** Original post on bio 1587 at 2013-07-09 15:33:24 UTC ***

(In reply to comment #11)
> Created attachment 8354329 [details] [diff] [review] (bio-attmnt 2561) [details]
> Part 2 v1
> 
> This sync instantbird/installer/Makefile.in. It does NOT include the changes to
> instantbird/installer/windows, these need to be debugged.

And package-manifest.in, this was mostly just reorganizing our list of files to match those in c-c. The diff in attachment 8354328 [details] [diff] [review] (bio-attmnt 2560) should be helpful here.

Comment 14

[Florian Quèze [:florian]]

Comment on attachment 8354329 [details] [diff] [review]
Part 2 v1

*** Original change on bio 1587 attmnt 2561 at 2013-08-02 17:12:17 UTC ***

>+++ b/instantbird/installer/Makefile.in

>-ifdef NECKO_WIFI
>-DEFINES += -DNECKO_WIFI
>-endif
>-

Why?

> ifdef MOZ_PKG_MANIFEST_P
> MOZ_PKG_MANIFEST = package-manifest
> 
> $(MOZ_PKG_MANIFEST): $(MOZ_PKG_MANIFEST_P) $(GLOBAL_DEPS)
> 	$(PYTHON) $(MOZILLA_SRCDIR)/config/Preprocessor.py $(DEFINES) $(ACDEFINES) $< > $@
>-
>-GARBAGE += $(MOZ_PKG_MANIFEST)

Why?

>diff --git a/instantbird/installer/package-manifest.in b/instantbird/installer/package-manifest.in

>@@ -30,6 +29,8 @@
> @APPNAME@/Contents/Info.plist
> @APPNAME@/Contents/PkgInfo
> @APPNAME@/Contents/Resources/
>+@APPNAME@/Contents/_CodeSignature/CodeResources

I think this only exists if we sign our builds.

>+@APPNAME@/Contents/Library/

I don't think Instantbird has this folder.


> @BINPATH@/chrome/@AB_CD@.manifest
> @BINPATH@/@PREF_DIR@/all-l10n.js
> @BINPATH@/searchplugins/*
>-@BINPATH@/defaults/profile/localstore.rdf
>-@BINPATH@/defaults/profile/mimeTypes.rdf
>-@BINPATH@/defaults/profile/chrome/*

Do you know why these 3 lines aren't needed any more?


> @BINPATH@/application.ini
>+@BINPATH@/update-settings.ini
>+@BINPATH@/blocklist.xml

Do these 2 file exist for us?

>-; [Components]
>+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
>+; Instantbird Specific Files
>+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

> @BINPATH@/components/ibCommandLineHandler.manifest
> @BINPATH@/components/ibStatusCommandLineHandler.js
> @BINPATH@/components/ibStatusCommandLineHandler.manifest
>-@BINPATH@/components/irc.js
>-@BINPATH@/components/irc.manifest
>+@BINPATH@/components/xpcom_base.xpt
>+@BINPATH@/components/xpcom_system.xpt
>+@BINPATH@/components/xpcom_components.xpt
>+@BINPATH@/components/xpcom_ds.xpt
>+@BINPATH@/components/xpcom_io.xpt
>+@BINPATH@/components/xpcom_threads.xpt
>+@BINPATH@/components/xpcom_xpti.xpt
>+@BINPATH@/chrome/toolkit@JAREXT@
>+@BINPATH@/chrome/toolkit.manifest
>+@BINPATH@/chrome/instantbird@JAREXT@
>+@BINPATH@/chrome/instantbird.manifest

How come we have xpcom_*.xpt files in the middle of Instantbird specific files?


>+; shell icons
>+#ifdef XP_UNIX
>+#ifndef XP_MACOSX
>+@BINPATH@/icons/*.xpm
>+@BINPATH@/icons/*.png
>+#endif
>+#endif

You are packaging @BINPATH@/icons/*.png twice.

Comment 15

[Florian Quèze [:florian]]

Comment on attachment 8354328 [details] [diff] [review]
Part 1 v1

*** Original change on bio 1587 attmnt 2560 at 2013-08-02 17:19:53 UTC ***

The commit messages in these changesets are horrible, and some of these changesets undo what was done in previous changesets, so I think we would be better off merging everything into the other part.


>+MOZ_PKG_FATAL_WARNINGS = 1

Setting this without fixing first the existing warnings is asking for trouble.

Current warnings on linux:
Warning: package-manifest:30: Missing file(s): bin/defaults/profile/chrome/*
Warning: package-manifest:75: Missing file(s): bin/components/content_xtf.xpt
Warning: package-manifest:206: Missing file(s): bin/components/nsProxyAutoConfig.manifest
Warning: package-manifest:207: Missing file(s): bin/components/nsProxyAutoConfig.js
Warning: package-manifest:267: Missing file(s): bin/extensions/{972ce4c6-7e08-4474-a285-3208198ce6fd}/icon.png
Warning: package-manifest:418: Missing file(s): bin/crashreporter-override.ini
Warning: package-manifest:421: components/libmozgnome.so already added

On Windows:
Warning: package-manifest:30: Missing file(s): bin/defaults/profile/chrome/*
Warning: package-manifest:35: Missing file(s): bin/uninstall/helper.exe
Warning: package-manifest:66: Missing file(s): bin/components/accessibility-msaa.xpt
Warning: package-manifest:83: Missing file(s): bin/components/content_xtf.xpt
Warning: package-manifest:212: Missing file(s): bin/components/nsProxyAutoConfig.manifest
Warning: package-manifest:213: Missing file(s): bin/components/nsProxyAutoConfig.js
Warning: package-manifest:272: Missing file(s): bin/extensions/{972ce4c6-7e08-4474-a285-3208198ce6fd}/icon.png
Warning: package-manifest:413: Missing file(s): bin/crashreporter-override.ini

On Mac:
Warning: package-manifest:34: Missing file(s): @BINPATH@/defaults/profile/chrome/*
Warning: package-manifest:77: Missing file(s): @BINPATH@/components/content_xtf.xpt
Warning: package-manifest:207: Missing file(s): @BINPATH@/components/nsProxyAutoConfig.manifest
Warning: package-manifest:208: Missing file(s): @BINPATH@/components/nsProxyAutoConfig.js
Warning: package-manifest:268: Missing file(s): @BINPATH@/extensions/{972ce4c6-7e08-4474-a285-3208198ce6fd}/icon.png
Warning: package-manifest:405: Missing file(s): @BINPATH@/crashreporter-override.ini

Comment 16

[Patrick Cloke [:clokep]]

*** Original post on bio 1587 at 2013-09-03 23:09:44 UTC ***

(In reply to comment #13)
> Comment on attachment 8354329 [details] [diff] [review] (bio-attmnt 2561) [details]
> Part 2 v1
> 
> >+++ b/instantbird/installer/Makefile.in
> 
> >-ifdef NECKO_WIFI
> >-DEFINES += -DNECKO_WIFI
> >-endif
> >-
> 
> Why?
This was added in bug 954773 (bio 1340). As far as I can tell this was never added to Thunderbird's Makefile.in.

> 
> > ifdef MOZ_PKG_MANIFEST_P
> > MOZ_PKG_MANIFEST = package-manifest
> > 
> > $(MOZ_PKG_MANIFEST): $(MOZ_PKG_MANIFEST_P) $(GLOBAL_DEPS)
> > 	$(PYTHON) $(MOZILLA_SRCDIR)/config/Preprocessor.py $(DEFINES) $(ACDEFINES) $< > $@
> >-
> >-GARBAGE += $(MOZ_PKG_MANIFEST)
> 
> Why?
This was added in http://hg.instantbird.org/instantbird/rev/a05f56f9408c, there's no bug and I can't seem to find this in Thunderbird's installer (it doesn't seem like it was ever there).

Would you like me to add these back?

> >diff --git a/instantbird/installer/package-manifest.in b/instantbird/installer/package-manifest.in
> 
> >@@ -30,6 +29,8 @@
> > @APPNAME@/Contents/Info.plist
> > @APPNAME@/Contents/PkgInfo
> > @APPNAME@/Contents/Resources/
> >+@APPNAME@/Contents/_CodeSignature/CodeResources
> 
> I think this only exists if we sign our builds.
> 
> >+@APPNAME@/Contents/Library/
> 
> I don't think Instantbird has this folder.
I've removed these since I can't check this since I don't have a Mac.


> > @BINPATH@/chrome/@AB_CD@.manifest
> > @BINPATH@/@PREF_DIR@/all-l10n.js
> > @BINPATH@/searchplugins/*
> >-@BINPATH@/defaults/profile/localstore.rdf
> >-@BINPATH@/defaults/profile/mimeTypes.rdf
> >-@BINPATH@/defaults/profile/chrome/*
> 
> Do you know why these 3 lines aren't needed any more?
localstore.rdf and mimeTypes.rdf were supposed to be moved, I've now done that. There doesn't even seem to be a chrome/* subfolder of instantbird/app/profile.


> > @BINPATH@/application.ini
> >+@BINPATH@/update-settings.ini
> >+@BINPATH@/blocklist.xml
> 
> Do these 2 file exist for us?
No, I've removed them.

> >-; [Components]
> >+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
> >+; Instantbird Specific Files
> >+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
> 
> > @BINPATH@/components/ibCommandLineHandler.manifest
> > @BINPATH@/components/ibStatusCommandLineHandler.js
> > @BINPATH@/components/ibStatusCommandLineHandler.manifest
> >-@BINPATH@/components/irc.js
> >-@BINPATH@/components/irc.manifest
> >+@BINPATH@/components/xpcom_base.xpt
> >+@BINPATH@/components/xpcom_system.xpt
> >+@BINPATH@/components/xpcom_components.xpt
> >+@BINPATH@/components/xpcom_ds.xpt
> >+@BINPATH@/components/xpcom_io.xpt
> >+@BINPATH@/components/xpcom_threads.xpt
> >+@BINPATH@/components/xpcom_xpti.xpt
> >+@BINPATH@/chrome/toolkit@JAREXT@
> >+@BINPATH@/chrome/toolkit.manifest
> >+@BINPATH@/chrome/instantbird@JAREXT@
> >+@BINPATH@/chrome/instantbird.manifest
> 
> How come we have xpcom_*.xpt files in the middle of Instantbird specific files?
This is where comm-release has these files, we can separate them out if you want.


> >+; shell icons
> >+#ifdef XP_UNIX
> >+#ifndef XP_MACOSX
> >+@BINPATH@/icons/*.xpm
> >+@BINPATH@/icons/*.png
> >+#endif
> >+#endif
> 
> You are packaging @BINPATH@/icons/*.png twice.
I removed the one next to the packaging of run-mozilla.sh.

Comment 17

[Patrick Cloke [:clokep]]

*** Original post on bio 1587 at 2013-09-03 23:43:56 UTC ***

(In reply to comment #14)
> The commit messages in these changesets are horrible, and some of these
> changesets undo what was done in previous changesets, so I think we would be
> better off merging everything into the other part.
I qfolded all of these patches together.


> >+MOZ_PKG_FATAL_WARNINGS = 1
> 
> Setting this without fixing first the existing warnings is asking for trouble.
> 
> Current warnings on linux:
> Warning: package-manifest:267: Missing file(s):
> bin/extensions/{972ce4c6-7e08-4474-a285-3208198ce6fd}/icon.png
With this patch almost all of these were already fixed, we'd have to add something as icon.png though, any ideas?

Comment 18

[Patrick Cloke [:clokep]]

Attachment: Sync installer v2

*** Original post on bio 1587 as attmnt 3026 at 2013-11-10 02:24:00 UTC ***

This seems to compile successfully on Windows. I did explicitly ignore the stuff that deals with the maintenance service, this would need to be done in a separate bug (and is something I should get around to). I'll also upload a diff between ib (with this patch) and c-r (TB 25).

This does not attempt to handle removed-files.in, I think Florian said we should be able to script this somehow, but last time I tried it wasn't possible on Windows since there's no way to open the Mac package (I think).

Comment 19

[Patrick Cloke [:clokep]]

Attachment: Diff with c-r (TB 25)

*** Original post on bio 1587 as attmnt 3027 at 2013-11-10 02:24:00 UTC was without comment, so any subsequent comment numbers will be shifted ***

Comment 20

[Patrick Cloke [:clokep]]

Comment on attachment 8354328 [details] [diff] [review]
Part 1 v1

*** Original change on bio 1587 attmnt 2560 at 2013-11-10 02:24:38 UTC was without comment, so any subsequent comment numbers will be shifted ***

Comment 21

[Patrick Cloke [:clokep]]

Comment on attachment 8354329 [details] [diff] [review]
Part 2 v1

*** Original change on bio 1587 attmnt 2561 at 2013-11-10 02:24:38 UTC was without comment, so any subsequent comment numbers will be shifted ***

Comment 22

[Patrick Cloke [:clokep]]

Comment on attachment 8354301 [details] [diff] [review]
Diff with c-r (TB 21)

*** Original change on bio 1587 attmnt 2533 at 2013-11-10 02:24:58 UTC was without comment, so any subsequent comment numbers will be shifted ***

Comment 23

[Patrick Cloke [:clokep]]

Attachment: Sync installer v3

*** Original post on bio 1587 as attmnt 3031 at 2013-11-11 13:35:00 UTC ***

This takes into account the changes from bug 955634 (bio 2189).

Comment 24

[Patrick Cloke [:clokep]]

Comment on attachment 8354807 [details] [diff] [review]
Sync installer v2

*** Original change on bio 1587 attmnt 3026 at 2013-11-11 13:35:43 UTC was without comment, so any subsequent comment numbers will be shifted ***

Comment 25

[Patrick Cloke [:clokep]]

Attachment: Diff with c-r (TB 25) v3

*** Original post on bio 1587 as attmnt 3032 at 2013-11-11 13:36:00 UTC was without comment, so any subsequent comment numbers will be shifted ***

Comment 26

[Patrick Cloke [:clokep]]

Comment on attachment 8354808 [details] [diff] [review]
Diff with c-r (TB 25)

*** Original change on bio 1587 attmnt 3027 at 2013-11-11 13:36:04 UTC was without comment, so any subsequent comment numbers will be shifted ***

Comment 27

[Florian Quèze [:florian]]

*** Original post on bio 1587 at 2013-11-11 15:14:44 UTC ***

(In reply to comment #15)
> (In reply to comment #13)
> > Comment on attachment 8354329 [details] [diff] [review] (bio-attmnt 2561) [details]
> > Part 2 v1
> > 
> > >+++ b/instantbird/installer/Makefile.in
> > 
> > >-ifdef NECKO_WIFI
> > >-DEFINES += -DNECKO_WIFI
> > >-endif
> > >-
> > 
> > Why?
> This was added in bug 954773 (bio 1340). As far as I can tell this was never added to
> Thunderbird's Makefile.in.
> 
> > 
> > > ifdef MOZ_PKG_MANIFEST_P
> > > MOZ_PKG_MANIFEST = package-manifest
> > > 
> > > $(MOZ_PKG_MANIFEST): $(MOZ_PKG_MANIFEST_P) $(GLOBAL_DEPS)
> > > 	$(PYTHON) $(MOZILLA_SRCDIR)/config/Preprocessor.py $(DEFINES) $(ACDEFINES) $< > $@
> > >-
> > >-GARBAGE += $(MOZ_PKG_MANIFEST)
> > 
> > Why?
> This was added in http://hg.instantbird.org/instantbird/rev/a05f56f9408c,
> there's no bug and I can't seem to find this in Thunderbird's installer (it
> doesn't seem like it was ever there).
> 
> Would you like me to add these back?

Yes, for both.

Comment 28

[Patrick Cloke [:clokep]]

Attachment: Sync installer v4

*** Original post on bio 1587 as attmnt 3034 at 2013-11-11 15:56:00 UTC ***

I had some comments from Florian over IRC, this adds back the NECKO_WIFI change in installer/Makefile.in and the change to GARBAGE.

This also removes the ifdef around components.manifest.

See http://log.bezut.info/instantbird/131111/#m92

This needs to be tested on Mac (and maybe Linux) before pushing.

Comment 29

[Patrick Cloke [:clokep]]

Comment on attachment 8354812 [details] [diff] [review]
Sync installer v3

*** Original change on bio 1587 attmnt 3031 at 2013-11-11 15:56:42 UTC was without comment, so any subsequent comment numbers will be shifted ***

Comment 30

[Patrick Cloke [:clokep]]

Attachment: Diff with c-r (TB 25) v4

*** Original post on bio 1587 as attmnt 3035 at 2013-11-11 15:57:00 UTC was without comment, so any subsequent comment numbers will be shifted ***

Comment 31

[Patrick Cloke [:clokep]]

Comment on attachment 8354813 [details] [diff] [review]
Diff with c-r (TB 25) v3

*** Original change on bio 1587 attmnt 3032 at 2013-11-11 15:57:01 UTC was without comment, so any subsequent comment numbers will be shifted ***

Comment 32

[Florian Quèze [:florian]]

Attachment: Fix for Mac

*** Original post on bio 1587 as attmnt 3041 at 2013-11-11 21:46:00 UTC ***

Additional changes to attachment 8354815 [details] [diff] [review] (bio-attmnt 3034) to fix make package on mac. Some of these changes were requested in comment 13.

Comment 33

[Florian Quèze [:florian]]

Comment on attachment 8354815 [details] [diff] [review]
Sync installer v4

*** Original change on bio 1587 attmnt 3034 at 2013-11-11 21:47:33 UTC ***

r=me with the changes of attachment 8354822 [details] [diff] [review] (bio-attmnt 3041) included.

Comment 34

[Patrick Cloke [:clokep]]

*** Original post on bio 1587 at 2013-11-11 21:53:23 UTC ***

http://hg.instantbird.org/instantbird/rev/cd4a67ecb4af

Do we want to update / sync the removed-files.in?

Comment 35

[Florian Quèze [:florian]]

*** Original post on bio 1587 at 2013-11-11 21:55:11 UTC ***

(In reply to comment #26)

> Do we want to update / sync the removed-files.in?

Someday, yes. In a different bug? :)

Comment 36

[Patrick Cloke [:clokep]]

*** Original post on bio 1587 at 2013-11-11 22:02:08 UTC ***

(In reply to comment #27)
> (In reply to comment #26)
> 
> > Do we want to update / sync the removed-files.in?
> 
> Someday, yes. In a different bug? :)

I filed bug 955696 (bio 2248).