1179945 - gen_test_packages_manifest.py failing on 40.0b1 release builds

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[[e-mail] (:bhearsum)]

Looks like something isn't coping with spaces in filenames properly:
make -C ./testing/web-platform stage-package
../../config/nsinstall -D ../../dist/test-stage/web-platform
./config/nsinstall -D dist/test-stage/luciddream
/builds/slave/rel-m-beta-m64_bld-00000000000/build/obj-firefox/i386/_virtualenv/bin/python /builds/slave/rel-m-beta-m64_bld-00000000000/build/build/gen_test_packages_manifest.py --common Firefox 40.0b1.tests.zip --jsshell jsshell-mac.zip --dest-file dist/mac/en-US//test_packages.json
usage: gen_test_packages_manifest.py [-h] --common TESTS_COMMON --jsshell
                                     JSSHELL --dest-file DESTFILE
gen_test_packages_manifest.py: error: unrecognized arguments: 40.0b1.tests.zip

Comment 1

[[e-mail] (:bhearsum)]

Chris, looks like you added this target in http://hg.mozilla.org/mozilla-central/diff/a7a6d91116ab/testing/testsuite-targets.mk

Comment 2

[Chris Manchester (limited bugmail, email directly)]

Attachment: Deal with spaces in file names when generating test archive dependencies

I wasn't able to reproduce the issue locally, but this ought to address it.

Comment 3

[[e-mail] (:bhearsum)]

Comment on attachment 8629063 [details] [diff] [review]
Deal with spaces in file names when generating test archive dependencies

Review of attachment 8629063 [details] [diff] [review]:
-----------------------------------------------------------------

Yeah, this should work for central...per IRC we'll need a different one for aurora and beta.

Comment 4

[Chris Manchester (limited bugmail, email directly)]

Attachment: Deal with spaces in file names when generating test archive dependencies (patch for beta)

Fix for beta (the first applies to central).

Comment 5

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/d2c37bf5d4a6

Comment 6

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Comment on attachment 8629067 [details] [diff] [review]
Deal with spaces in file names when generating test archive dependencies (patch for beta)

This is blocking 40.0b1 release. Approving for uplift to Beta.

Comment 7

[Chris Manchester (limited bugmail, email directly)]

remote:   https://hg.mozilla.org/releases/mozilla-beta/rev/e22cb5ee5134

Comment 8

[Ryan VanderMeulen [:RyanVM]]

Aurora nomination coming too? :)

Comment 9

[Nick Thomas [:nthomas] (UTC+12)]

We hit another error with that fix in:

/builds/slave/rel-m-beta-m64_bld-00000000000/build/obj-firefox/i386/_virtualenv/bin/python /builds/slave/rel-m-beta-m64_bld-00000000000/build/build/gen_test_packages_manifest.py --common 'Firefox 40.0b1.tests.zip' --jsshell 'jsshell-mac.zip' --dest-file dist/mac/en-US//test_packages.json
Traceback (most recent call last):
  File "/builds/slave/rel-m-beta-m64_bld-00000000000/build/build/gen_test_packages_manifest.py", line 62, in <module>
    with open(args.destfile, 'w') as of:
IOError: [Errno 2] No such file or directory: 'dist/mac/en-US//test_packages.json'
make[3]: *** [test-packages-manifest] Error 1
make[2]: *** [postflight_all] Error 2
make[1]: *** [realbuild] Error 2
make: *** [build] Error 2

Full log:
http://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/40.0b1-candidates/build2/logs/release-mozilla-beta-macosx64_build-bm84-build1-build4.txt.gz

We're not actually running tests on these release builds so this would not be a problem, except that it blocks other packaging and uploading.

Comment 10

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Create directory before trying to write to it

r? rail to try to get this beta out this week

Comment 11

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8629216 [details] [diff] [review]
Create directory before trying to write to it

https://hg.mozilla.org/releases/mozilla-beta/rev/962491662436 
(and removed the release tags to that with 2455ab8cb01c)

Comment 12

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8629216 [details] [diff] [review]
Create directory before trying to write to it

This works for me on a borrowed slave, but I want to run it by you before landing on default on beta, and possibly elsewhere.

Comment 13

[Mike Hommey [:glandium]]

Comment on attachment 8629216 [details] [diff] [review]
Create directory before trying to write to it

Review of attachment 8629216 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/testsuite-targets.mk
@@ +418,5 @@
>  
>  test-packages-manifest:
>  	@rm -f $(MOZ_TEST_PACKAGES_FILE)
> +ifndef UNIVERSAL_BINARY
> +	$(NSINSTALL) -D $(DIST)/$(PKG_PATH)

$(dir $(MOZ_TEST_PACKAGES_FILE))

Comment 14

[Nick Thomas [:nthomas] (UTC+12)]

Fix on 'default' of beta:
https://hg.mozilla.org/releases/mozilla-beta/rev/7b632d539a00

Comment 15

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/d2c37bf5d4a6

Comment 16

[Chris Manchester (limited bugmail, email directly)]

Comment on attachment 8629216 [details] [diff] [review]
Create directory before trying to write to it

Review of attachment 8629216 [details] [diff] [review]:
-----------------------------------------------------------------

We'll need this for aurora too. I'll file a follow up for the fix to central.

Comment 17

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to Chris Manchester [:chmanchester] from comment #16)

Thanks for catching this. Please note that what landed is slightly different from attachment 8629216 [details] [diff] [review]. See https://hg.mozilla.org/releases/mozilla-beta/rev/7b632d539a00.

Comment 18

[Chris Manchester (limited bugmail, email directly)]

The rest of bug 917999 is on aurora now, I'll do the rest of the uplifts in bug 1181732

Comment 19

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-esr140/rev/359c711c088f