Problems with win32 en-ZA and eo for DevEdition 57.0b5



2 years ago
2 years ago


(Reporter: nthomas, Unassigned)


Firefox Tracking Flags

(Not tracked)


We had a lot of windows jobs retry for deved 57.0b5 because high prices in the AWS spot market caused instances to be terminated. This lead to 'devedition mozilla-beta win32 l10n repack 3/10' task [1] running twice, almost simultaneously [2,3]. Most probably we ended up with a mixture of artifacts attached to XvOSfX9CRKGDttkP3mkVQQ, and ended up with update verify errors [4-6] like

calling QuitProgressUI
Files source/bin/uninstall/helper.exe and target/bin/uninstall/helper.exe differ
FAIL: binary files found in diff
FAIL: [57.0 en-ZA complete] check_updates returned failure for WINNT_x86-msvc-x86 downloads/Firefox Setup 57.0b1.exe vs. downloads/Firefox Setup 57.0b5.exe: 1

helper.exe is localised, so it is signed in each l10n repack run. I think we ended up with the complete.mar and installer.exe from different runs for en-ZA and eo (South African English and Esperanto).

Nothing much to see at

The buildbot start times are only one second apart, so perhaps we hit a race claiming the buildrequest in the buildbot db. I don't recall this being a problem before but it's the sort of thing that could happen at the start of any job.
ftr - I removed en-ZA and eo locales under win32 in release blob:
Say this was a chemspill and windows german, what options would we have?
(In reply to Axel Hecht [:Pike] from comment #3)
> Say this was a chemspill and windows german, what options would we have?

We would be forced into delaying the release for every platform and locale and do a build2.
We shipped 57.0b5. I can't quite predict what's going to happen with subsequent betas, but if we get a lot of update verify failures we could consider a patch like this:

diff --git a/release/patcher-configs/mozDevedition-branch-patcher2.cfg b/release/patcher-configs/mozDevedition-branch-patcher2.cfg
--- a/release/patcher-configs/mozDevedition-branch-patcher2.cfg
+++ b/release/patcher-configs/mozDevedition-branch-patcher2.cfg
@@ -331,12 +331,14 @@
                     ja   linux-i686, linux-x86_64, win32, win64
                     ja-JP-mac   mac
+                    en-ZA  win32
+                    eo   win32
                 extension-version   57.0
                 locales   ach af an ar as ast az be bg bn-BD bn-IN br bs ca cak cs cy da de dsb el en-GB en-US en-ZA eo es-AR es-CL es-ES es-MX et eu fa ff fi fr fy-NL ga-IE gd gl gn gu-IN he hi-IN hr hsb hu hy-AM id is it ja ja-JP-mac ka kab kk km kn ko lij lt lv mai mk ml mr ms my nb-NO nl nn-NO or pa-IN pl pt-BR pt-PT rm ro ru si sk sl son sq sr sv-SE ta te th tr uk ur uz vi xh zh-CN zh-TW
                     linux-i686   20171002181526
                     linux-x86_64   20171002181526
Closed: 2 years ago
Resolution: --- → FIXED
See Also: → 1406403
Even though the patch in comment 5 landed[1], we still got failures[2]. That makes me wonder if we shouldn't have:

> en-ZA  mac, linux-i686, linux-x86_64, win64, 
> eo   mac, linux-i686, linux-x86_64, win64, 

instead. If I understand the the japanese lines correctly, ja-JP-mac is for mac. Then, "en-ZA  win32" may mean: this locale only exist for win32. Reopening the bug. 

Resolution: FIXED → ---
Closed: 2 years ago2 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1406403
You need to log in before you can comment on or make changes to this bug.