The wrong build locale gets offered as fallback update for Nightly Italian version

RESOLVED FIXED

Status

Release Engineering
General
P2
normal
RESOLVED FIXED
5 years ago
13 days ago

People

(Reporter: Daniela Petrovici, Assigned: coop)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Reporter)

Description

5 years ago
Created attachment 725432 [details]
screenshot with error after update

The Mozmill fallback update tests started failing on Nightly IT with today's build. 

It appears that the Italian locale is being offered the wrong fallback update:
https://aus3.mozilla.org/update/3/Firefox/21.0a2/20130313042013/WINNT_x86-msvc/it/aurora/Windows_NT%206.1/default/default/update.xml?force=1

Mozmill report:
http://mozmill-crowd.blargon7.com/#/update/report/2a6536e9db9f5f44ed48c58510b605d1

Jenkins link: http://mm-ci-master.qa.scl3.mozilla.com:8080/job/mozilla-central_update/1377/console

The error can be reproduced locally by running the Mozmill tests and after update Firefox cannot be opened (I have attached the screenshot with the error received when trying to open Firefox after update).
Component: Application Update → Release Engineering: Releases
Product: Toolkit → mozilla.org
QA Contact: bhearsum
Version: 22 Branch → other
I'm disabling nightly updates while we investigate this.
Component: Release Engineering: Releases → Release Engineering: Automation (General)
QA Contact: bhearsum → catlee
So, looking at logs I see:
========= Started find filepath (results: 0, elapsed: 0 secs) (at 2013-03-15 08:24:59.857282) =========
'bash' '-c' 'find build/mozilla-central/obj-firefox/dist/update -maxdepth 1 -type f -name *.complete.mar'
 in dir e:\builds\moz2_slave\m-cen-w32-l10n-ntly-0000000000\. (timeout 1200 secs)
build/mozilla-central/obj-firefox/dist/update/firefox-22.0a1.br.win32.complete.mar
build/mozilla-central/obj-firefox/dist/update/firefox-22.0a1.et.win32.complete.mar
build/mozilla-central/obj-firefox/dist/update/firefox-22.0a1.he.win32.complete.mar
build/mozilla-central/obj-firefox/dist/update/firefox-22.0a1.it.win32.complete.mar

and we end up taking the first one as the complete MAR.

So, either something isn't cleaning up like it did before, or we changed something in our buildbot code that's making us less choosey about the MAR.
Looks like this is related to bug 784848

13:14 < bhearsum> i see "'sh' '-c' 'if [ -d mozilla-central/dist/update ]; then rm -rf mozilla-central/dist/update; fi'" is yesterday's l10n nightly....
13:14 < bhearsum> coop: ah, whic his why https://github.com/mozilla/build-buildbotcustom/commit/1115642ad5573710d40280c6c603d034af3e1a81#L1R2439 got added, i guess?
13:15 < bhearsum> i see this in today's nightly: "'sh' '-c' 'if [ -d obj-firefox/dist/update ]; then rm -rf obj-firefox/dist/update; fi'"
13:15 < bhearsum> which gets run in  in dir e:\builds\moz2_slave\m-cen-w32-l10n-ntly-0000000000\build (timeout 1200 secs)
13:15 <@coop> yes
13:15 < bhearsum> so that dir probably doesn't exist...needs to be mozilla-central/obj-firefox
13:15 <@coop> that's not the correct dir, yeah
13:15 < bhearsum> boo
Assignee: nobody → coop
After this gets fixed and a new set of nightlies get run we need to chmod the snippet directory to enable nightlies again:
chmod 775 /opt/aus2/incoming/2/Firefox/mozilla-central

should do it.
(Assignee)

Comment 7

5 years ago
Created attachment 725595 [details] [diff] [review]
Specify the proper path to the dist dir for pre-build cleanup
Attachment #725595 - Flags: review?(bhearsum)

Updated

5 years ago
Attachment #725595 - Flags: review?(bhearsum) → review+
(Assignee)

Comment 8

5 years ago
Comment on attachment 725595 [details] [diff] [review]
Specify the proper path to the dist dir for pre-build cleanup

https://hg.mozilla.org/build/buildbotcustom/rev/84f077434979
Attachment #725595 - Flags: checked-in+
(Assignee)

Comment 9

5 years ago
In production now, although bug 851720 may prevent aurora updates from happening tonight.
Status: NEW → ASSIGNED
Priority: -- → P2
Updates for nightly still seem to be off. Can we turn them back on now?
(In reply to Reed Loden [:reed] from comment #10)
> Updates for nightly still seem to be off. Can we turn them back on now?

I just asked about this and was told that bug 851793 also need to be fixed first.
Blocks: 852106
(Assignee)

Comment 12

5 years ago
We'll track further issues via bug 852106.
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
Component: General Automation → General
Product: Release Engineering → Release Engineering
You need to log in before you can comment on or make changes to this bug.