1362611 - Thunderbird 54.0b1 build2: build step failed on win32

Status: RESOLVED FIXED

(Release Engineering :: Release Automation, defect, P2)

Description

[Wayne Mery (:wsmwk)]

+++ This bug was initially created as a clone of Bug #1349525 +++

rail	haven't seen that before.. a wild guess would be two single quotes in 

pymake\..\..\mozmake.exe: Entering directory 'c:/builds/moz2_slave/tb-rel-c-beta-w32_bld-00000000/build/objdir-tb/tools/update-packaging'
mkdir -p 'c:/builds/moz2_slave/tb-rel-c-beta-w32_bld-00000000/build/objdir-tb/dist/update/win32/en-US/'
test -f 'c:/builds/moz2_slave/tb-rel-c-beta-w32_bld-00000000/build/objdir-tb/dist/win32/en-US/Thunderbird Setup 54.0b1.exe'
rm -f -rf 'c:/builds/moz2_slave/tb-rel-c-beta-w32_bld-00000000/build/objdir-tb/dist/thunderbird'
cd c:/builds/moz2_slave/tb-rel-c-beta-w32_bld-00000000/build/objdir-tb/dist && c:/builds/moz2_slave/tb-rel-c-beta-w32_bld-00000000/build/objdir-tb/_virtualenv/Scripts/python.exe -m mozbuild.action.7z_exe_extract ''c:/builds/moz2_slave/tb-rel-c-beta-w32_bld-00000000/build/objdir-tb/dist/win32/en-US/Thunderbird Setup 54.0b1.exe'' 'thunderbird'
Makefile:58: recipe for target 'complete-patch' failed
pymake\..\..\mozmake.exe: Leaving directory 'c:/builds/moz2_slave/tb-rel-c-beta-w32_bld-00000000/build/objdir-tb/tools/update-packaging'
Usage: 7z_exe_extract.py <package> <target>
pymake\..\..\mozmake.exe: *** [complete-patch] Error 1
program finished with exit code 2
elapsedTime=1.732000
========= master_lag: 0.13 =========
========= Finished 'python c:/builds/moz2_slave/tb-rel-c-beta-w32_bld-00000000/build/build/pymake/make.py ...' failed (results: 2, elapsed: 1

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

53.0b9 [1]:
cd c:/builds/moz2_slave/tb-rel-c-beta-w32_bld-00000000/build/objdir-tb/dist &&  7z x 'c:/builds/moz2_slave/tb-rel-c-beta-w32_bld-00000000/build/objdir-tb/dist/win32/en-US/Thunderbird Setup 53.0b1.exe' core && mv core thunderbird

54.0b1 build2 [2]:
cd c:/builds/moz2_slave/tb-rel-c-beta-w32_bld-00000000/build/objdir-tb/dist && c:/builds/moz2_slave/tb-rel-c-beta-w32_bld-00000000/build/objdir-tb/_virtualenv/Scripts/python.exe -m mozbuild.action.7z_exe_extract ''c:/builds/moz2_slave/tb-rel-c-beta-w32_bld-00000000/build/objdir-tb/dist/win32/en-US/Thunderbird Setup 54.0b1.exe'' 'thunderbird'

ie extract core and move to thunderbird vs extract thunderbird. A little earlier in the 54.0b1 build2 log, the files are still in core:

Creating archive c:/builds/moz2_slave/tb-rel-c-beta-w32_bld-00000000/build/objdir-tb/mail/installer/windows/instgen/app.7z

Compressing  core\distribution\extensions\{e2fda1a4-762b-4020-b5ad-a41df1933103}\components\icaljs-manifest
Compressing  core\precomplete
Compressing  core\removed-files
....

I suggest looking for the build system change which changed the extract call.


[1] https://archive.mozilla.org/pub/thunderbird/candidates/53.0b1-candidates/build3/logs/release-comm-beta-win32_build-bm73-build1-build6.txt.gz
[2] https://archive.mozilla.org/pub/thunderbird/candidates/54.0b1-candidates/build2/logs/release-comm-beta-win32_build-bm71-build1-build7.txt.gz

Comment 2

[Wayne Mery (:wsmwk)]

(In reply to Nick Thomas [:nthomas] from comment #1)
> ...
> I suggest looking for the build system change which changed the extract call.

bug 1337145?
https://hg.mozilla.org/mozilla-central/rev/5579ec1531b5
installer-stage: prepare-package
 ifndef MOZ_PKG_MANIFEST
 	$(error MOZ_PKG_MANIFEST unspecified!)
 endif
 	@rm -rf $(DEPTH)/installer-stage $(DIST)/xpt
 	@echo 'Staging installer files...'
 	@$(NSINSTALL) -D $(DEPTH)/installer-stage/core
 	@cp -av $(DIST)/$(MOZ_PKG_DIR)$(_BINPATH)/. $(DEPTH)/installer-stage/core
+	@(cd $(DEPTH)/installer-stage/core && $(CREATE_PRECOMPLETE_CMD))
 ifdef MOZ_SIGN_PREPARED_PACKAGE_CMD
 # The && true is necessary to make sure Pymake spins a shell

Comment 3

[Ronald Claveau [:sousmangoosta]]

Attachment: 1362611.patch

For me the build failed here : https://hg.mozilla.org/releases/mozilla-beta/file/bad375b8eabf/toolkit/mozapps/installer/upload-files.mk#l124

Please find a patch in attachment, and let me know.

Comment 4

[Wayne Mery (:wsmwk)]

sousmangoosta found the regression in Bug 1287881 - add mach repackage so windows builds can be signed as separate step from build with move to taskcluster

Comment 5

[Philipp Kewisch [:Fallen][📱📆]]

comm try build: https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=c26bd2336ca0b530ff33a21400157191ac28bf18
mozilla try build: https://treeherder.mozilla.org/#/jobs?repo=try&revision=8bd7c6f93882fd5ff943fdec282314df82667dea

Remind me to check the try builds if I don't get back to you. The final review will have to come from a m-c build peer.

Comment 6

[Ronald Claveau [:sousmangoosta]]

Mike, Could you please review the patch ?

Thx

Comment 7

[Mike Hommey [:glandium]]

Comment on attachment 8866723 [details] [diff] [review]
1362611.patch

Review of attachment 8866723 [details] [diff] [review]:
-----------------------------------------------------------------

Redirecting to mshal, who reviewed bug 1287881, that introduced that code. That being said, the unzip branch seems fishy too. It doesn't help that the UNPACKAGE variable itself has the single quotes. Not sure why bug 1287881 added those quotes to INNER_*_PACKAGE.

Comment 8

[Michael Shal [:mshal]]

Comment on attachment 8866723 [details] [diff] [review]
1362611.patch

It looks like the problem is that l10n passes in UNPACKAGE without quotes (it escapes the spaces instead), so it is is fine to add single quotes when using UNPACKAGE in that context, though they aren't necessary. But INNER_UNMAKE_PACKAGE is also used during mar packaging, which has UNPACKAGE defined with quotes, so adding extra quotes when using $(UNPACKAGE) causes the problem you saw.

I agree that make_unzip should also be fixed, as well as unpackge_dmg. Can you do that as part of this patch?

Comment 9

[Ronald Claveau [:sousmangoosta]]

Attachment: 1362611_1.patch

Comment 10

[Michael Shal [:mshal]]

Comment on attachment 8868263 [details] [diff] [review]
1362611_1.patch

This looks good to me. Can you do a try push to make sure there aren't any unintended consequences?

Comment 11

[Ronald Claveau [:sousmangoosta]]

Try is running here : https://treeherder.mozilla.org/#/jobs?repo=try&revision=bd5bd42f8e31453342f3469b76ed5d2bb36768e2

Comment 12

[Wayne Mery (:wsmwk)]

try build looks pretty good

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/2fe44b09de7c
Fix packager quoting in the installer. r=mshal

Comment 14

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/2fe44b09de7c

Comment 15

[Ronald Claveau [:sousmangoosta]]

The log is ok on moz-central build

mozbuild.action.7z_exe_extract 'c:/builds/moz2_slave/tb-c-cen-w32-ntly-000000000000/build/objdir-tb/dist/install/sea/thunderbird-55.0a1.en-US.win32.installer.exe' thunderbird


Here is the full log:
https://archive.mozilla.org/pub/thunderbird/nightly/2017/05/2017-05-18-03-02-10-comm-central/comm-central-win32-nightly-bm77-build1-build6.txt.gz

Ok for other platforms too

Comment 16

[Carsten Book [:Tomcat]]

adding checkin needed since ryan wanted to take a look

Comment 17

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/54f218cec92f