835309 - make package fails on mac universal builds because of extensions such as test pilot

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Mark Banner (:standard8)]

+++ This bug was initially created as a clone of Bug #713133 +++

Bug 713133 is enabling fatal warnings for Windows and Linux, but currently on try server, Mac is failing with:

/builds/slave/tb-try-c-cen-osx64/build/objdir-tb/i386/_virtualenv/bin/python /builds/slave/tb-try-c-cen-osx64/build/mozilla/toolkit/mozapps/installer/packager.py -DMOZ_THUNDERBIRD=1 -DOSTYPE=\"Darwin\" -DOSARCH=Darwin -DNO_NSPR_10_SUPPORT -DAB_CD=en-US -DMOZ_APP_NAME=thunderbird -DPREF_DIR=defaults/pref -DJAREXT= -DMOZ_UPDATER=1 -DMOZ_CHILD_PROCESS_NAME=plugin-container -DMOZ_SHARED_MOZGLUE=1 -DDLL_PREFIX=lib -DDLL_SUFFIX=.dylib -DBIN_SUFFIX= -DAPPNAME=Daily.app -DBINPATH=/Daily.app/Contents/MacOS \
		--format omni \
		--removals /builds/slave/tb-try-c-cen-osx64/build/mail/installer/removed-files.in \
		 \
		 \
		--jarlogs /builds/slave/tb-try-c-cen-osx64/build/objdir-tb/i386/mail/installer/../../jarlog//en-US \
		--optimizejars \
		--unify /builds/slave/tb-try-c-cen-osx64/build/objdir-tb/i386/../x86_64/mozilla/dist \
		package-manifest ../../mozilla/dist ../../mozilla/dist/universal/thunderbird \
		--non-resource defaults/messenger/mailViews.dat
Error: /builds/slave/tb-try-c-cen-osx64/build/objdir-tb/i386/mail/installer/package-manifest:137: Can't unify Daily.app/Contents/MacOS/distribution/extensions/[e-mail]: file differs between /builds/slave/tb-try-c-cen-osx64/build/objdir-tb/i386/../x86_64/mozilla/dist and ../../mozilla/dist
--- /builds/slave/tb-try-c-cen-osx64/build/objdir-tb/i386/../x86_64/mozilla/dist/Daily.app/Contents/MacOS/distribution/extensions/[e-mail]
+++ ../../mozilla/dist/Daily.app/Contents/MacOS/distribution/extensions/[e-mail]
@@ -1,4 +1,4 @@

(followed by a diff)

I don't know why that would fail when Mac Nightly builds fine.

Comment 1

[Mike Hommey [:glandium]]

Nightlies probably don't have the test pilot extension.

Comment 2

[Mark Banner (:standard8)]

(In reply to Mike Hommey [:glandium] from comment #1)
> Nightlies probably don't have the test pilot extension.

Good idea, but actually they do for Thunderbird. The diff shows the xpi diffs as well.

Comment 3

[Mike Hommey [:glandium]]

I guess unify.py just needs a better test for zip archives.

Comment 4

[Mike Hommey [:glandium]]

Attachment: Test patch

Can you test this? (completely untested)

Comment 5

[Mark Banner (:standard8)]

Yep, pushed to try:

https://tbpl.mozilla.org/?tree=Thunderbird-Try&rev=f3c87f45ad67

Comment 6

[Mark Banner (:standard8)]

 File "/builds/slave/tb-try-c-cen-osx64/build/mozilla/python/mozbuild/mozpack/unify.py", line 176, in unify_file
    jar1 = JarReader(fileobj=file1.open())
NameError: global name 'JarReader' is not defined

https://tbpl.mozilla.org/php/getParsedLog.php?id=19234855&tree=Thunderbird-Try

Comment 7

[Mike Hommey [:glandium]]

Attachment: Test patch

Try this revised version (not in a landable state, but should work)

Comment 8

[Mark Banner (:standard8)]

Pushed the new patch to try:

https://tbpl.mozilla.org/?tree=Thunderbird-Try&rev=15867e3e1fa5

Comment 9

[Mark Banner (:standard8)]

Great! that passed.

Comment 10

[Mike Hommey [:glandium]]

Attachment: Fix JarFileReader.readlines() and make flake8 happier.

Comment 11

[Mike Hommey [:glandium]]

Attachment: Look at .xpi file contents when unifying them for universal builds.

Comment 12

[Mike Hommey [:glandium]]

Attachment: Look at .xpi file contents when unifying them for universal builds.

Comment 13

[Gregory Szorc [:gps]]

Comment on attachment 708981 [details] [diff] [review]
Fix JarFileReader.readlines() and make flake8 happier.

Review of attachment 708981 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks for making flake8 happier!

Comment 14

[Gregory Szorc [:gps]]

Comment on attachment 708983 [details] [diff] [review]
Look at .xpi file contents when unifying them for universal builds.

Review of attachment 708983 [details] [diff] [review]:
-----------------------------------------------------------------

LGTM!

Comment 15

[Mike Hommey [:glandium]]

https://hg.mozilla.org/mozilla-central/rev/6474019340d3
https://hg.mozilla.org/mozilla-central/rev/d949c8ee39da