1168999 - Enable mozharness desktop l10n repacks on mozilla-aurora

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Rail Aliiev [:rail]]

m-c is green enough, we should enable them on m-a too!

Comment 1

[Chris AtLee [:catlee]]

Attachment: aurora-l10n.diff

could it be this easy?

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

rail mentioned there's some bustage on central right now, bug 1169937 I think. I suggest we wait until that's resolved until we push this change, given we're doing some marketing for DevEdition starting June 2nd.

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8612610 [details] [diff] [review]
aurora-l10n.diff

This landed at 
  http://hg.mozilla.org/build/buildbot-configs/rev/7a5654e7329e
at 
  Mon, 01 Jun 2015 13:48:01 -0400

We may back it out to get the DevEdition whatsnew out.

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8612610 [details] [diff] [review]
aurora-l10n.diff

I'm pretty sure we're going run into more issues because there is no aurora equivalent of 
 http://hg.mozilla.org/build/mozharness/file/476752fc4ea8/configs/single_locale/mozilla-central.py

Backing out as least scary choice at this point:
https://hg.mozilla.org/build/buildbot-configs/rev/c1695b4e384b
https://hg.mozilla.org/build/buildbot-configs/rev/9354dbb04817

Will go live in the automated reconfig in a few mins.

Comment 5

[Chris AtLee [:catlee]]

Attachment: aurora-mozharness.patch

Comment 6

[Rail Aliiev [:rail]]

Comment on attachment 8615332 [details] [diff] [review]
aurora-mozharness.patch

Do you know why we use http:// instead of https:// to download the en-US binaries?

Comment 7

[Chris AtLee [:catlee]]

I think wget/curl fails with SSL errors on some platform :\

Comment 8

[Chris Cooper [:coop] (he/him)]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/964e9e789fdb

Comment 9

[Rail Aliiev [:rail]]

Attachment: aurora_l10n.diff

The repacks are failing because they are trying to clone from https://hg.mozilla.org/l10n-central/$locale. Looks like configs/single_locale/$platform.py files contain branch specific information, which is already in confgs/single_locale/mozilla-aurora.py.

Comment 10

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-aurora&job_id=903429
repository: mozilla-aurora
start_time: 2015-06-17T02:22:56
who: tomcat[at]mozilla[dot]com
machine: bld-lion-r5-055
buildname: Firefox mozilla-aurora macosx64 l10n nightly-1
revision: 37d5ec17d2fc

abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
Automation Error: Can't checkout https://hg.mozilla.org/l10n-central/an!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/an to /builds/hg-shared/l10n-central/an!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/an to /builds/hg-shared/l10n-central/an!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/an to /builds/hg-shared/l10n-central/an!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/an to /builds/hg-shared/l10n-central/an!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/an to /builds/hg-shared/l10n-central/an!
Running post_fatal callback...
Exiting -1

Comment 11

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-aurora&job_id=903430
repository: mozilla-aurora
start_time: 2015-06-17T02:22:56
who: tomcat[at]mozilla[dot]com
machine: bld-lion-r5-085
buildname: Firefox mozilla-aurora macosx64 l10n nightly-2
revision: 37d5ec17d2fc

abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
Automation Error: Can't checkout https://hg.mozilla.org/l10n-central/hsb!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/hsb to /builds/hg-shared/l10n-central/hsb!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/hsb to /builds/hg-shared/l10n-central/hsb!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/hsb to /builds/hg-shared/l10n-central/hsb!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/hsb to /builds/hg-shared/l10n-central/hsb!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/hsb to /builds/hg-shared/l10n-central/hsb!
Running post_fatal callback...
Exiting -1

Comment 12

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-aurora&job_id=903428
repository: mozilla-aurora
start_time: 2015-06-17T02:23:10
who: tomcat[at]mozilla[dot]com
machine: bld-lion-r5-043
buildname: Firefox mozilla-aurora macosx64 l10n nightly-3
revision: 37d5ec17d2fc

abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
Automation Error: Can't checkout https://hg.mozilla.org/l10n-central/sat!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/sat to /builds/hg-shared/l10n-central/sat!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/sat to /builds/hg-shared/l10n-central/sat!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/sat to /builds/hg-shared/l10n-central/sat!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/sat to /builds/hg-shared/l10n-central/sat!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/sat to /builds/hg-shared/l10n-central/sat!
Running post_fatal callback...
Exiting -1

Comment 13

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-aurora&job_id=903467
repository: mozilla-aurora
start_time: 2015-06-17T03:18:36
who: tomcat[at]mozilla[dot]com
machine: bld-linux64-spot-163
buildname: Firefox mozilla-aurora linux64 l10n nightly-2
revision: 37d5ec17d2fc

abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
Automation Error: Can't checkout https://hg.mozilla.org/l10n-central/hsb!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/hsb to /builds/hg-shared/l10n-central/hsb!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/hsb to /builds/hg-shared/l10n-central/hsb!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/hsb to /builds/hg-shared/l10n-central/hsb!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/hsb to /builds/hg-shared/l10n-central/hsb!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/hsb to /builds/hg-shared/l10n-central/hsb!
Running post_fatal callback...
Exiting -1

Comment 14

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-aurora&job_id=903468
repository: mozilla-aurora
start_time: 2015-06-17T03:18:37
who: tomcat[at]mozilla[dot]com
machine: bld-linux64-spot-083
buildname: Firefox mozilla-aurora linux64 l10n nightly-3
revision: 37d5ec17d2fc

abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
Automation Error: Can't checkout https://hg.mozilla.org/l10n-central/sat!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/sat to /builds/hg-shared/l10n-central/sat!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/sat to /builds/hg-shared/l10n-central/sat!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/sat to /builds/hg-shared/l10n-central/sat!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/sat to /builds/hg-shared/l10n-central/sat!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/sat to /builds/hg-shared/l10n-central/sat!
Running post_fatal callback...
Exiting -1

Comment 15

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-aurora&job_id=903509
repository: mozilla-aurora
start_time: 2015-06-17T03:18:46
who: tomcat[at]mozilla[dot]com
machine: bld-linux64-spot-012
buildname: Firefox mozilla-aurora linux l10n nightly-3
revision: 37d5ec17d2fc

abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
Automation Error: Can't checkout https://hg.mozilla.org/l10n-central/sat!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/sat to /builds/hg-shared/l10n-central/sat!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/sat to /builds/hg-shared/l10n-central/sat!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/sat to /builds/hg-shared/l10n-central/sat!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/sat to /builds/hg-shared/l10n-central/sat!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/sat to /builds/hg-shared/l10n-central/sat!
Running post_fatal callback...
Exiting -1

Comment 16

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-aurora&job_id=903466
repository: mozilla-aurora
start_time: 2015-06-17T03:18:36
who: tomcat[at]mozilla[dot]com
machine: bld-linux64-spot-151
buildname: Firefox mozilla-aurora linux64 l10n nightly-1
revision: 37d5ec17d2fc

abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
Automation Error: Can't checkout https://hg.mozilla.org/l10n-central/an!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/an to /builds/hg-shared/l10n-central/an!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/an to /builds/hg-shared/l10n-central/an!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/an to /builds/hg-shared/l10n-central/an!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/an to /builds/hg-shared/l10n-central/an!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/an to /builds/hg-shared/l10n-central/an!
Running post_fatal callback...
Exiting -1

Comment 17

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-aurora&job_id=903581
repository: mozilla-aurora
start_time: 2015-06-17T04:12:34
who: tomcat[at]mozilla[dot]com
machine: b-2008-ix-0128
buildname: Firefox mozilla-aurora win32 l10n nightly-3
revision: 37d5ec17d2fc

abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
abort: HTTP Error 404: Not Found
Automation Error: hg not responding
Return code: 255
Automation Error: Can't checkout https://hg.mozilla.org/l10n-central/sat!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/sat to c:/builds/hg-shared\l10n-central/sat!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/sat to c:/builds/hg-shared\l10n-central/sat!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/sat to c:/builds/hg-shared\l10n-central/sat!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/sat to c:/builds/hg-shared\l10n-central/sat!
Caught exception: Unable to clone https://hg.mozilla.org/l10n-central/sat to c:/builds/hg-shared\l10n-central/sat!
Running post_fatal callback...
Exiting -1

Comment 18

[Rail Aliiev [:rail]]

Comment on attachment 8623598 [details] [diff] [review]
aurora_l10n.diff

Transplanted to production:

remote:   https://hg.mozilla.org/build/mozharness/rev/585c42501cbd
remote:   https://hg.mozilla.org/build/mozharness/rev/be59230d72ce

Comment 19

[Rail Aliiev [:rail]]

Now it fails on mac:

05:46:09     INFO - Running command: ['make', 'unpack'] in /builds/slave/m-aurora-m64-l10n-ntly-1-00000/build/mozilla-aurora/obj-l10n/browser/locales
...
05:46:09     INFO -  + /builds/slave/m-aurora-m64-l10n-ntly-1-00000/build/mozilla-aurora/build/package/mac_osx/unpack-diskimage /builds/slave/m-aurora-m64-l10n-ntly-1-00000/build/mozilla-aurora/obj-l10n/dist/firefox-40.0a2.en-US.mac.dmg /tmp/firefox-unpack /builds/slave/m-aurora-m64-l10n-ntly-1-00000/build/mozilla-aurora/obj-l10n/dist/unpack.tmp
....
05:46:45     INFO -  FirefoxDeveloperEdition.app/Contents/_CodeSignature/CodeResources
05:46:45     INFO -  sent 316238387 bytes  received 2830 bytes  14708893.81 bytes/sec
05:46:45     INFO -  total size is 316190418  speedup is 1.00
05:46:45     INFO -  "disk3" unmounted.
05:46:45     INFO -  "disk3" ejected.
05:46:50     INFO -  + rsync -a /builds/slave/m-aurora-m64-l10n-ntly-1-00000/build/mozilla-aurora/obj-l10n/dist/unpack.tmp/FirefoxFirefoxDeveloperEdition.app firefox
05:46:50     INFO -  rsync: link_stat "/builds/slave/m-aurora-m64-l10n-ntly-1-00000/build/mozilla-aurora/obj-l10n/dist/unpack.tmp/FirefoxFirefoxDeveloperEdition.app" failed: No such file or directory (2)
05:46:50     INFO -  rsync error: some files could not be transferred (code 23) at /SourceCache/rsync/rsync-42/rsync/main.c(992) [sender=2.6.9]
05:46:50     INFO -  make: *** [/builds/slave/m-aurora-m64-l10n-ntly-1-00000/build/mozilla-aurora/obj-l10n/dist/l10n-stage/firefox/FirefoxFirefoxDeveloperEdition.app/Contents/Resources] Error 23
05:46:50    ERROR - Return code: 2
05:46:50    ERROR - 2 not in success codes: [0]
05:46:50  WARNING - setting return code to 2
05:46:50    FATAL - Halting on failure while running ['make', 'unpack']
05:46:50    FATAL - Running post_fatal callback...
05:46:50    FATAL - Exiting 2
program finished with exit code 2

Comment 20

[Rail Aliiev [:rail]]

Full log is here: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2015/06/2015-06-17-00-40-06-mozilla-aurora-l10n/mozilla-aurora-macosx64-l10n-nightly-1-unknown-bm82-build1-build0.txt.gz

Comment 21

[Rail Aliiev [:rail]]

<catlee> MOZ_UPDATE_CHANNEL=nightly
<catlee> triggers this:
<catlee> 05:46:05     INFO - if test "${MOZ_UPDATE_CHANNEL}" = "nightly"; then
<catlee> 05:46:05     INFO - ac_add_options --with-macbundlename-prefix=Firefox
<catlee> 05:46:05     INFO - fi


remote:   https://hg.mozilla.org/build/mozharness/rev/74410fe01e45
remote:   https://hg.mozilla.org/build/mozharness/rev/6f313e1b7c40

Comment 22

[Rail Aliiev [:rail]]

I pointed aurora updates to Firefox-mozilla-aurora-nightly-20150616004005 for now

Comment 23

[Rail Aliiev [:rail]]

I pointed the updates back to latest.

Comment 24

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-aurora&job_id=904671
repository: mozilla-aurora
start_time: 2015-06-17T11:45:10
who: rail[at]mozilla[dot]com
machine: b-2008-ix-0014
buildname: Firefox mozilla-aurora win32 l10n nightly-3
revision: 37d5ec17d2fc

command timed out: 10800 seconds elapsed, attempting to kill

Comment 25

[Rail Aliiev [:rail]]

all green today \o/