962420 - repack failure during 2.24b1 release

Status: RESOLVED FIXED

(SeaMonkey :: Build Config, defect)

Description

[Edmund Wong (:ewong)]

Currently we're having repack problems with our 2.24b1 release.

make.py[0]: Entering directory 'e:\builds\slave\rel-c-beta-w32-rpk\build\comm-beta\obj-l10n\suite\locales'
No rule to make target 'installers-be' needed by ['<command-line>', 'installers-be']
Traceback (most recent call last):
  File "e:/builds/slave/rel-c-beta-w32-rpk/build/comm-beta/build/pymake/make.py", line 35, in <module>
    main(sys.argv[1:])
  File "e:/builds/slave/rel-c-beta-w32-rpk/build/comm-beta/build/pymake/make.py", line 32, in main
    subprocess.check_call([sys.executable, PYMAKE] + args)
  File "D:\mozilla-build\python27\lib\subprocess.py", line 511, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['D:\\mozilla-build\\python27\\python2.7.exe', 'e:\\builds\\slave\\rel-c-beta-w32-rpk\\build\\comm-beta\\mozilla\\build\\pymake\\make.py', 'installers-be', 'LOCALE_MERGEDIR=e:/builds/slave/rel-c-beta-w32-rpk/build/merged']' returned non-zero exit status 2
program finished with exit code 1
elapsedTime=0.938000

Comment 1

[Justin Wood (:Callek)]

Attachment: tmp.txt

I'm getting waaaay too tired, but I narrowed this down to a combination of two issues:

we download (some) artifacts to the wrong location during a release, so needs fixing.

we don't escape spaces in paths in many places.

Here is a raw-dump from my "current state" of objdir/suite/locales/Makefile  (yes not the .in so not a patch):

[[[[see attachment which I was going to paste into bug, until I realized how long it was]]]]

Comment 2

[Justin Wood (:Callek)]

Current State has:

     remove: D3DCompiler_42.dll
     remove: ; shared mail libraries
     remove: ; pre-static-build, pre-libxul shared libraries
     remove: ; pre-libxul static mail library
     remove: ; Library files which used to have a short name.
     remove: .autoreg
E:/builds/slave/rel-c-beta-w32-rpk/build/comm-beta/mozilla/tools/update-packagin
g/make_full_update.sh: line 109: e:/builds/slave/rel-c-beta-w32-rpk/build/comm-b
eta/obj-l10n/mozilla/dist/host/bin/mar.exe: No such file or directory
mv: cannot stat `E:/builds/slave/rel-c-beta-w32-rpk/build/comm-beta/obj-l10n/sui
te/locales/../../mozilla/dist/l10n-stage/seamonkey.work/output.mar': No such fil
e or directory

Finished
make.py[3]: Leaving directory 'E:\builds\slave\rel-c-beta-w32-rpk\build\comm-bet
a\obj-l10n\suite\locales\../../mozilla/tools/update-packaging'
E:\builds\slave\rel-c-beta-w32-rpk\build\comm-beta\mozilla\toolkit\locales\l10n.
mk:136:0$ e:/builds/slave/rel-c-beta-w32-rpk/build/comm-beta/obj-l10n/mozilla/_v
irtualenv/Scripts/python.exe E:/builds/slave/rel-c-beta-w32-rpk/build/comm-beta/
mozilla/config/nsinstall.py -D ../../mozilla/dist/win32/be/
E:\builds\slave\rel-c-beta-w32-rpk\build\comm-beta\mozilla\toolkit\locales\l10n.
mk:137:0$ mv -f "../../mozilla/dist/l10n-stage/win32/be/seamonkey-2.24b1.exe" "E
:/builds/slave/rel-c-beta-w32-rpk/build/comm-beta/obj-l10n/suite/locales/../../m
ozilla/dist/win32/be/SeaMonkey?Setup?2.24b1.exe"
mv: cannot move `../../mozilla/dist/l10n-stage/win32/be/seamonkey-2.24b1.exe' to
 `E:/builds/slave/rel-c-beta-w32-rpk/build/comm-beta/obj-l10n/suite/locales/../.
./mozilla/dist/win32/be/SeaMonkey?Setup?2.24b1.exe': No such file or directory
E:\builds\slave\rel-c-beta-w32-rpk\build\comm-beta\mozilla\toolkit\locales\l10n.
mk:137:0: command 'mv -f "../../mozilla/dist/l10n-stage/win32/be/seamonkey-2.24b
1.exe" "E:/builds/slave/rel-c-beta-w32-rpk/build/comm-beta/obj-l10n/suite/locale
s/../../mozilla/dist/win32/be/SeaMonkey?Setup?2.24b1.exe"' failed, return code 1

E:\builds\slave\rel-c-beta-w32-rpk\build\comm-beta\obj-l10n\suite\locales\Makefi
le:169:0: command 'D:/mozilla-build/python27/python2.7.exe e:/builds/slave/rel-c
-beta-w32-rpk/build/comm-beta/mozilla/build/pymake/pymake/../make.py repackage-z
ip \
  AB_CD=be \
  MOZ_PKG_FORMAT=SFX7Z \
  ZIP_IN=E:/builds/slave/rel-c-beta-w32-rpk/build/comm-beta/obj-l10n/suite/local
es/../../mozilla/dist/win32/en-US/SeaMonkey?Setup?2.24b1.exe \
  ZIP_OUT="E:/builds/slave/rel-c-beta-w32-rpk/build/comm-beta/obj-l10n/suite/loc
ales/../../mozilla/dist/win32/be/SeaMonkey?Setup?2.24b1.exe" \
  SFX_HEADER="E:/builds/slave/rel-c-beta-w32-rpk/build/comm-beta/obj-l10n/suite/
locales/../installer/windows/l10ngen/7zSD.sfx \
              E:/builds/slave/rel-c-beta-w32-rpk/build/comm-beta/suite/installer
/windows/app.tag"' failed, return code 2
E:\builds\slave\rel-c-beta-w32-rpk\build\comm-beta\obj-l10n\suite\locales\Makefi
le:179:0: command 'D:/mozilla-build/python27/python2.7.exe e:/builds/slave/rel-c
-beta-w32-rpk/build/comm-beta/mozilla/build/pymake/pymake/../make.py repackage-w
in32-installer AB_CD=be WIN32_INSTALLER_IN="E:/builds/slave/rel-c-beta-w32-rpk/b
uild/comm-beta/obj-l10n/suite/locales/../../mozilla/dist/win32/en-US/SeaMonkey S
etup 2.24b1.exe"' failed, return code 2
Traceback (most recent call last):
  File "e:/builds/slave/rel-c-beta-w32-rpk/build/comm-beta/build/pymake/make.py"
, line 35, in <module>
    main(sys.argv[1:])
  File "e:/builds/slave/rel-c-beta-w32-rpk/build/comm-beta/build/pymake/make.py"
, line 32, in main
    subprocess.check_call([sys.executable, PYMAKE] + args)
  File "D:\mozilla-build\python27\lib\subprocess.py", line 511, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['D:\\mozilla-build\\python27\\python2.7
.exe', 'e:\\builds\\slave\\rel-c-beta-w32-rpk\\build\\comm-beta\\mozilla\\build\
\pymake\\make.py', '-dd', 'repackage-win32-installer-be', 'LOCALE_MERGEDIR=e:/bu
ilds/slave/rel-c-beta-w32-rpk/build/merged']' returned non-zero exit status 2

Comment 3

[Frank Wein [:mcsmurf]]

So current state of the build is that it fails to find e:/builds/slave/rel-c-beta-w32-rpk/build/comm-b
eta/obj-l10n/mozilla/dist/host/bin/mar.exe and E:/builds/slave/rel-c-beta-w32-rpk/build/comm-beta/obj-l10n/sui
te/locales/../../mozilla/dist/l10n-stage/seamonkey.work/output.mar ?

Comment 4

[Justin Wood (:Callek)]

Attachment: [comm-beta] WIP 1

I still need to acclimate this through a few more minor checks, and run this as part of our beta 1 (on a relbranch) so it checks against other platforms for bustage, but this WIP is pretty close to final

Comment 5

[Justin Wood (:Callek)]

Attachment: [mozilla-beta] WIP 1

Similarly I think this one is pretty much final, but will be on a beta relbranch of mine before final checkin on trunk

Comment 6

[Justin Wood (:Callek)]

Comment on attachment 8364077 [details] [diff] [review]
[mozilla-beta] WIP 1

Review of attachment 8364077 [details] [diff] [review]:
-----------------------------------------------------------------

::: toolkit/locales/l10n.mk
@@ +176,5 @@
>  	$(error Wget not installed)
>  endif
>  	$(NSINSTALL) -D $(_ABS_DIST)/$(PKG_PATH)
> +	(cd $(_ABS_DIST)/$(PKG_PATH) && $(WGET) --no-cache -nv -O "$$(basename $(PACKAGE))" -N  "$(EN_US_BINARY_URL)/$(PACKAGE)")
> +	@echo "Downloaded $(EN_US_BINARY_URL)/$(PACKAGE) to $(_ABS_DIST)/$(PKG_PATH)/$$(basename $(PACKAGE))"

before you ask this was needed because it was downloading ..../win32/be/<file>.zip to ..../win32/be/win32/be/<file>.zip

since we already cd in here, and $(PACKAGE) contains a path in this case.

Comment 7

[Axel Hecht [:Pike]]

Did you try to make the quoting and deps in suite/locales/Makefile.in match what we do in browser? I noticed a few differences still. Those combinations are very subtle, and changing the ones in l10n.mk might break browser.

Comment 8

[Justin Wood (:Callek)]

(In reply to Axel Hecht [:Pike] from comment #7)
> Did you try to make the quoting and deps in suite/locales/Makefile.in match
> what we do in browser? I noticed a few differences still. Those combinations
> are very subtle, and changing the ones in l10n.mk might break browser.

I tried (my best) for the parts that failed, while the same code path in browser/ is never utilized in automation (we don't use this with PRETTYNAMES anymore) so we never call these paths with spaces in them or with a release-dir-path.

Comment 9

[Mike Hommey [:glandium]]

Comment on attachment 8364077 [details] [diff] [review]
[mozilla-beta] WIP 1

Review of attachment 8364077 [details] [diff] [review]:
-----------------------------------------------------------------

::: toolkit/locales/l10n.mk
@@ +134,4 @@
>  endif
>  endif
>  	$(NSINSTALL) -D $(DIST)/$(PKG_PATH)
> +	mv -f "$(DIST)/l10n-stage/$(PACKAGE)" "$(call ESCAPE_SPACE,$(ZIP_OUT))"

For all shell purposes, "foo\ bar" is the same as "foo bar". That being said, those should be single quotes instead of double quotes, as per bug 943728.

@@ +176,5 @@
>  	$(error Wget not installed)
>  endif
>  	$(NSINSTALL) -D $(_ABS_DIST)/$(PKG_PATH)
> +	(cd $(_ABS_DIST)/$(PKG_PATH) && $(WGET) --no-cache -nv -O "$$(basename $(PACKAGE))" -N  "$(EN_US_BINARY_URL)/$(PACKAGE)")
> +	@echo "Downloaded $(EN_US_BINARY_URL)/$(PACKAGE) to $(_ABS_DIST)/$(PKG_PATH)/$$(basename $(PACKAGE))"

I see how that's "necessary" in the echo (but shouldn't affect whether that works or not, it's just a display issue), I don't see why you need to change the wget invocation. wget http://foo/bar/baz/qux.zip will save to qux.zip, not baz/qux.zip (if EN_US_BINARY_URL=http://foo/bar and PACKAGE=baz/qux.zip)

All that being said, like Pike, I don't think this is the right time to risk breaking browser builds, so i'd prefer if you avoided l10n.mk changes altogether.

Comment 10

[Mike Hommey [:glandium]]

Comment on attachment 8364075 [details] [diff] [review]
[comm-beta] WIP 1

Review of attachment 8364075 [details] [diff] [review]:
-----------------------------------------------------------------

I think you should mimick what browser/locales/Makefile.in does, which is not quite the same as what you have. That /should/ even work without l10n.mk changes, since browser does with without l10n.mk changes (obviously)

Comment 11

[Justin Wood (:Callek)]

Attachment: [comm-beta] v1.0 (better patch)

This was also tested, and seems to work!

Comment 12

[Philip Chee]

> 22:19	Callek	RattyAway: if you get a chance tonight, can you please land
> http://hg.mozilla.org/releases/comm-beta/rev/d9c53409d7b2 on aurora, and trunk

Pushed to comm-aurora and trunk:
https://hg.mozilla.org/releases/comm-aurora/rev/0c0348549dd4
https://hg.mozilla.org/comm-central/rev/5ea6651ce415

FYI:
Patch was pushed to comm-beta by Callek on 23rd Jan 2014
http://hg.mozilla.org/releases/comm-beta/rev/d9c53409d7b2

Comment 13

[Edmund Wong (:ewong)]

Is this bug considered fixed?

Comment 14

[Justin Wood (:Callek)]

I was leaving this open to make sure we land the buildbot changes we did, I'm going to do them in seperate bug(s) at this point, since its a pretty mess in there