desktop firefox beta repacks failing to some locale's complete mar



3 years ago
2 years ago


(Reporter: jlund, Assigned: jlund)


Firefox Tracking Flags

(Not tracked)




3 years ago
Traceback (most recent call last):
  File "c:/builds/moz2_slave/rel-m-beta-w64_rpk_1-000000000/scripts/scripts/l10n/", line 386, in <module>
  File "c:/builds/moz2_slave/rel-m-beta-w64_rpk_1-000000000/scripts/scripts/l10n/", line 183, in createRepacks
    "Failed locales: %s" % " ".join([x for x, _ in failed]))
Exception: Failed locales: be


3 years ago
Assignee: nobody → jlund

Comment 2

3 years ago
it looks like we are failing to run `mar -t $archive`[1] against some previous complete mars.

command: START
command: perl ../../../tools/update-packaging/ ../../../..//
command: cwd: /builds/slave/rel-m-beta-lx_rpk_1-0000000000/mozilla-beta/obj-l10n/dist/previous
... # env
command: output:
Couldn't run "/builds/slave/rel-m-beta-lx_rpk_1-0000000000/mozilla-beta/obj-l10n/dist/host/bin/mar" -t at ../../../tools/update-packaging/ line 51.
command: ERROR

looking for that mar, it seems like it downloaded okay, and I can verify it exists:
Downloading to

If they all look like that it's bug 1214971, which we need to reopen with details including estimated time, which datacenter the slave is in. We've mostly seen this on linux repacks up till now so it's interesting that it was mostly windows this time. An mtr/traceroute to would be useful, given bug 1170832 and the weird routing for Cloudfront (which is serving archive.m.o).

To recover the release, make a list of all the locales that failed for each platform, and run the standalone repack jobs
You'll need to force the repack_complete for each platform after the standlone job is done, otherwise the updates job won't start.

Comment 4

3 years ago
I'll re-open 1214971 with the details requested. I won't bother doing the standalone runs as we are going to do a build2

all failures in comment 1 look very similar

Comment 5

3 years ago
fwiw - I'm just reading and I noticed that my logs (comment 1) don't show a RETRY in the download getting the complete mar in question.

Comment 6

3 years ago
these failures look identical to the beta9 issues rail had:


log example of beta9 failure:

if it happens again for build2, I'll run standalone builders


3 years ago
Blocks: 1211313

Comment 7

3 years ago
similar things happened for build2. re-running the failed l10n builders worked so far.

Comment 8

3 years ago
I'll leave this bug open as a blocker for all things 43. Not sure if we have another bug currently open for needing to run standalone repacks against intermittently failing locales when unpacking their complete mar
Summary: 43.0b1 build1 locales vi, cy, be, bn-IN failing across various platforms → desktop firefox beta repacks failing to some locale's complete mar
Haven't seen this since november
Last Resolved: 2 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.