902091 - failed l10n repacks for firefox 24.0b1 on windows

Status: RESOLVED FIXED

(Release Engineering :: Release Automation, defect)

Description

[Massimo Gervasini [:massimo]]

l10n repacks for firefox 24.0b1 failed on windows with the following error:

client.mk:90: *** Pymake is required to build on Windows. Run |./mach build| to automatically use pymake. Or, invoke pymake directly via |python build/pymake/make.py|..  Stop.

Full log here:
http://buildbot-master57.srv.releng.use1.mozilla.com:8001/builders/release-mozilla-beta-win32_repack_8%2F10/builds/2/steps/run_script/logs/stdio

Comment 1

[Aki Sasaki (not active)]

Attachment: windows_tools

Allow for --use-pymake in the release repack script.
That way we can turn this on for beta, and not affect esr or release.

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 786529 [details] [diff] [review]
windows_tools

Review of attachment 786529 [details] [diff] [review]:
-----------------------------------------------------------------

r+ but please test in staging to confirm it works as expected.

Comment 3

[Aki Sasaki (not active)]

Attachment: windows_configs

Comment 4

[Aki Sasaki (not active)]

Attachment: windows_custom

Comment 5

[Aki Sasaki (not active)]

Attachment: windows_custom

Comment 6

[Aki Sasaki (not active)]

Attachment: windows_configs

Still testing :\

Comment 7

[Armen [:armenzg]]

Looking to help for a bit.

Comment 8

[Armen [:armenzg]]

I've refreshed aki's buildbot-config repo.
I think that things were committed and tagged in default but I could be misreading.

Comment 9

[Armen [:armenzg]]

Changing the l10n base to match what is in the tree (even though mozilla-release was the same):
-ac_add_options --with-l10n-base=..
+ac_add_options --with-l10n-base=../../l10n
http://hg.mozilla.org/users/asasaki_mozilla.com/buildbot-configs/raw-diff/0e1c3bca0230/mozilla2/win32/mozilla-beta/release/l10n-mozconfig

Then, it failed because we're building on win64 machines:
> configure: error: You are targeting i386 but using the 64-bit compiler.
so I decided to look into the L10n nightly's l10n-mozconfig which we run on win64 machines:
http://hg.mozilla.org/users/asasaki_mozilla.com/buildbot-configs/rev/f0872aedf3f9
http://buildbot-master65.srv.releng.usw2.mozilla.com:8001/builders/Firefox%20mozilla-central%20win32%20l10n%20nightly/builds/565/steps/cat_mozconfig/logs/stdio

diff --git a/mozilla2/win32/mozilla-beta/release/l10n-mozconfig b/mozilla2/win32/mozilla-beta/release/l10n-mozconfig
--- a/mozilla2/win32/mozilla-beta/release/l10n-mozconfig
+++ b/mozilla2/win32/mozilla-beta/release/l10n-mozconfig
@@ -1,5 +1,19 @@
-ac_add_options --enable-application=browser
-ac_add_options --enable-update-channel=beta
+. "$topsrcdir/browser/config/mozconfigs/common"
+
+#ac_add_options --enable-application=browser
+ac_add_options --enable-update-channel=${MOZ_UPDATE_CHANNEL}
 ac_add_options --enable-update-packaging
-ac_add_options --enable-official-branding
+#ac_add_options --enable-official-branding
 ac_add_options --with-l10n-base=../../l10n
+ac_add_options --enable-metro
+ac_add_options --with-windows-version=601
+
+export MOZILLA_OFFICIAL=1
+
+if test "$PROCESSOR_ARCHITECTURE" = "AMD64" -o "$PROCESSOR_ARCHITEW6432" = "AMD64"; then
+  . $topsrcdir/build/win32/mozconfig.vs2010-win64
+else
+  . $topsrcdir/build/win32/mozconfig.vs2010
+fi
+
+. "$topsrcdir/build/mozconfig.common.override"

Comment 10

[Armen [:armenzg]]

Then I merged from production to default because that is what we use.

Then I tried to make things closer to what aki had:
http://hg.mozilla.org/users/asasaki_mozilla.com/buildbot-configs/rev/8bbf2e22c436

Comment 11

[Armen [:armenzg]]

and now putting the l10n base back to where it was:
http://hg.mozilla.org/users/asasaki_mozilla.com/buildbot-configs/rev/fa13cd09bbfc

Comment 12

[Armen [:armenzg]]

and removing tier_base:
http://hg.mozilla.org/users/asasaki_mozilla.com/tools/rev/ede65b2397eb

e:\builds\moz2_slave\rel-m-beta-w32_rpk_1-000000000\mozilla-beta\config\rules.mk:1140:0: command 'e:/builds/moz2_slave/rel-m-beta-w32_rpk_1-000000000/mozilla-beta/obj-l10n/_virtualenv/Scripts/python.exe -O e:/builds/moz2_slave/rel-m-beta-w32_rpk_1-000000000/mozilla-beta/build/cl.py cl -Fovmwarerecordinghelper.obj -c -D_HAS_EXCEPTIONS=0 -I../../../dist/stl_wrappers  -DNO_NSPR_10_SUPPORT  -Ie:/builds/moz2_slave/rel-m-beta-w32_rpk_1-000000000/mozilla-beta/build/win32/vmwarerecordinghelper -I. -I../../../dist/include  -Ie:/builds/moz2_slave/rel-m-beta-w32_rpk_1-000000000/mozilla-beta/obj-l10n/dist/include/nspr -Ie:/builds/moz2_slave/rel-m-beta-w32_rpk_1-000000000/mozilla-beta/obj-l10n/dist/include/nss        -wd4099 -TP -nologo -W3 -Gy -Fdgenerated.pdb -wd4251 -wd4244 -wd4345 -wd4351 -wd4482 -wd4800 -wd4819 -we4553 -GR-  -DNDEBUG -DTRIMMED -Zi -UDEBUG -DNDEBUG -O1 -Oy -MT            -FI ../../../dist/include/mozilla-config.h -DMOZILLA_CLIENT  e:/builds/moz2_slave/rel-m-beta-w32_rpk_1-000000000/mozilla-beta/build/win32/vmwarerecordinghelper/vmwarerecordinghelper.cpp' failed, return code 2
e:\builds\moz2_slave\rel-m-beta-w32_rpk_1-000000000\mozilla-beta\config\makefiles\target_libs.mk:59:0: command 'c:/mozilla-build/python27/python.exe e:/builds/moz2_slave/rel-m-beta-w32_rpk_1-000000000/mozilla-beta/build/pymake/pymake/../make.py -C vmwarerecordinghelper libs' failed, return code 2
e:\builds\moz2_slave\rel-m-beta-w32_rpk_1-000000000\mozilla-beta\config\makefiles\target_libs.mk:59:0: command 'c:/mozilla-build/python27/python.exe e:/builds/moz2_slave/rel-m-beta-w32_rpk_1-000000000/mozilla-beta/build/pymake/pymake/../make.py -C win32 libs' failed, return code 2
e:\builds\moz2_slave\rel-m-beta-w32_rpk_1-000000000\mozilla-beta\config\makefiles\target_libs.mk:17:0: command 'c:/mozilla-build/python27/python.exe e:/builds/moz2_slave/rel-m-beta-w32_rpk_1-000000000/mozilla-beta/build/pymake/pymake/../make.py -C build libs' failed, return code 2
evaluation from e:\builds\moz2_slave\rel-m-beta-w32_rpk_1-000000000\mozilla-beta\config\rules.mk:737:24:23:0: command 'c:/mozilla-build/python27/python.exe e:/builds/moz2_slave/rel-m-beta-w32_rpk_1-000000000/mozilla-beta/build/pymake/pymake/../make.py libs_tier_base' failed, return code 2
command: ERROR

Comment 13

[Armen [:armenzg]]

I'm trying now without "tier_nspr" (just to try):
http://hg.mozilla.org/users/asasaki_mozilla.com/tools/rev/6a01445e3dcc

and now with "mozilla-release" instead:
http://hg.mozilla.org/users/asasaki_mozilla.com/buildbot-configs/rev/594bad4eeb7a

I'm going to grab something to eat and then join a Vidyo room to see who can join and discuss the problem.

Comment 14

[Armen [:armenzg]]

I've switched back to win32 machines since coop is working on bug 898099.

Comment 15

[Armen [:armenzg]]

Now going back to a mozconfig similar to mozilla-aurora's nightly:
http://hg.mozilla.org/users/asasaki_mozilla.com/buildbot-configs/rev/22b5c3af6713
and adding tier_nspr and tier_base back:
http://hg.mozilla.org/users/asasaki_mozilla.com/tools/rev/537587f38451

Comment 16

[Armen [:armenzg]]

Switching to mozilla-release seems to get us moving forward.

Comment 17

[Aki Sasaki (not active)]

Attachment: windows_tools, with successful make installers-%s

Comment 18

[Aki Sasaki (not active)]

Attachment: windows_configs

Comment 19

[Aki Sasaki (not active)]

Comment on attachment 787171 [details] [diff] [review]
windows_tools, with successful make installers-%s

http://hg.mozilla.org/build/tools/rev/962220abc86e

Comment 20

[Aki Sasaki (not active)]

Comment on attachment 787173 [details] [diff] [review]
windows_configs

http://hg.mozilla.org/build/buildbot-configs/rev/5701eb77c611

Comment 21

[Aki Sasaki (not active)]

Comment on attachment 786702 [details] [diff] [review]
windows_custom

http://hg.mozilla.org/build/buildbotcustom/rev/27ad13d23bea

Comment 22

[Aki Sasaki (not active)]

Due to conflicts I missed this part of the tools patch:
http://hg.mozilla.org/build/tools/rev/868dda8c4aad

Comment 23

[Rail Aliiev [:rail]]

Fixed in 24.0b2