1042432 - make upload ignores UPLOAD_EXTRA_FILES

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Massimo Gervasini [:massimo]]

We are building firefox 32.0b1 build1 and the release-mozilla-beta-updates step fails because there are no *info.txt files in: 

http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/32.0b1-candidates/build1/

the files are generated during the build process:

echo buildID=20140722093241 > linux_info.txt,

and they are in UPLOAD_EXTRA_FILES

UPLOAD_EXTRA_FILES="linux_info.txt update/linux-i686/en-US/firefox-31.0b8-32.0b1.partial.mar ..."

but they are not present in the output of make upload:

http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/32.0b1-candidates/build1/linux-i686/en-US/firefox-32.0b1.tar.bz2
http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/32.0b1-candidates/build1/update/linux-i686/en-US/firefox-32.0b1.complete.mar
http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/32.0b1-candidates/build1/linux-i686/xpi/en-US.xpi
http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/32.0b1-candidates/build1/linux-i686/en-US/firefox-32.0b1.tests.zip
http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/32.0b1-candidates/build1/linux-i686/en-US/firefox-32.0b1.crashreporter-symbols.zip
http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/32.0b1-candidates/build1/linux-i686/en-US/firefox-32.0b1.txt
http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/32.0b1-candidates/build1/linux-i686/en-US/firefox-32.0b1.json
http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/32.0b1-candidates/build1/jsshell-linux-i686.zip
http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/32.0b1-candidates/build1/host/bin/mar
http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/32.0b1-candidates/build1/host/bin/mbsdiff
http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/32.0b1-candidates/build1/linux-i686/en-US/firefox-32.0b1.checksums
http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/32.0b1-candidates/build1/linux-i686/en-US/firefox-32.0b1.checksums.asc
Uploading /builds/slave/rel-m-beta-lx_bld-000000000000/build/obj-firefox/dist/linux-i686/en-US/firefox-32.0b1.tar.bz2

The same applies for partial files too, they are generated but they are not uploaded.

full log here:
http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/32.0b1-candidates/build1/logs/release-mozilla-beta-linux_build-bm84-build1-build7.txt.gz

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

glandium points to bug 1005495 regressing this.

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: [m-b] Bandaid for beta - remove AC_SUBST from bug 1005495

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8460696 [details] [diff] [review]
[m-b] Bandaid for beta - remove AC_SUBST from bug 1005495

http://hg.mozilla.org/releases/mozilla-beta/rev/6a87b4799b21

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

[Tracking Requested - why for this release]: we should track a proper fix for this bug, to land on central and aurora this cycle. Beta 32 has a bandaid so is OK.

Comment 5

[Sylvestre Ledru [:Sylvestre]]

Nick, are you going to take care of a proper fix?

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

No, it would be best if glandium or another build system person did that.

Comment 7

[Mike Hommey [:glandium]]

Attachment: Change how UPLOAD_EXTRA_FILES is passed down from mozconfig.cache to the build system

Interestingly, I tried that at the time the bug was filed and that failed. I don't know what I was doing wrong, but this appears to work just fine: the sccache.log.gz is in the build directory after a try build.

Comment 8

[Mike Hommey [:glandium]]

Comment on attachment 8469144 [details] [diff] [review]
Change how UPLOAD_EXTRA_FILES is passed down from mozconfig.cache to the build system

Nick, would you mind testing this in real conditions on a staged release?

Comment 9

[Michael Shal [:mshal]]

Comment on attachment 8469144 [details] [diff] [review]
Change how UPLOAD_EXTRA_FILES is passed down from mozconfig.cache to the build system

This seems to add sccache.log.gz to UPLOAD_EXTRA_FILES multiple times, at least when going through client.mk. I think in part this is because there's no include guards on .mozconfig.mk, but I can't account for all the extra occurrences. I might just be testing it incorrectly - I'm just printing out $(UPLOAD_EXTRA_FILES) at the top of obj/Makefile and testing it with/without it set in the environment.

If it's not easy to only add it once, we may want to $(sort) UPLOAD_EXTRA_FILES when adding to UPLOAD_FILES to prune the duplicates.

nthomas, do you see sccache.log.gz getting uploaded multiple times with this patch? If it's just something on my end, I'll change to r+.

Comment 10

[Mike Hommey [:glandium]]

Attachment: Change how UPLOAD_EXTRA_FILES is passed down from mozconfig.cache to the build system

Try this.

Comment 11

[Michael Shal [:mshal]]

Comment on attachment 8469670 [details] [diff] [review]
Change how UPLOAD_EXTRA_FILES is passed down from mozconfig.cache to the build system

Chatted with glandium in IRC - so the duplicates were coming from client.mk re-invoking itself and re-including .mozconfig.mk, but since we don't actually run 'make upload' through client.mk, that shouldn't matter. The new patch unexport's UPLOAD_EXTRA_FILES to match the environment that client.mk had before this patch.

nit: A comment in client.mk of why we're unexporting it will be helpful in the future.

Comment 12

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8469670 [details] [diff] [review]
Change how UPLOAD_EXTRA_FILES is passed down from mozconfig.cache to the build system

Will run this in a staging release on the current tip of mozilla-beta.

Comment 13

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8469670 [details] [diff] [review]
Change how UPLOAD_EXTRA_FILES is passed down from mozconfig.cache to the build system

Linux64 en-US build uploaded the same set of files in staging as the production.

Comment 14

[Mike Hommey [:glandium]]

https://hg.mozilla.org/integration/fx-team/rev/fab1f203cab0

Comment 15

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/fab1f203cab0

Comment 16

[Sylvestre Ledru [:Sylvestre]]

Mike, can we have an uplift for aurora? Thanks

Comment 17

[Mike Hommey [:glandium]]

Comment on attachment 8469670 [details] [diff] [review]
Change how UPLOAD_EXTRA_FILES is passed down from mozconfig.cache to the build system

Approval Request Comment
[Feature/regressing bug #]: regression bug 1005495
[User impact if declined]: results of the build are not all uploaded for releases/betas
[Describe test coverage new/current, TBPL]: tested by nthomas on staging.
[Risks and why]: No risk. This doesn't touch anything that is really important to the build.
[String/UUID change made/needed]: None

Comment 18

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/83f0220caa12