1223385 - Switch to use in-tree version of compare-locales

Status: RESOLVED FIXED

(Release Engineering :: Applications: MozharnessCore, defect)

Description

[Axel Hecht [:Pike]]

Let's use the in-tree version of compare-locales for Desktop, and Android.

This is the less-aggressive path than originally proposed in https://groups.google.com/d/msg/mozilla.dev.builds/0koWKXx7arg/izvBHzy8nmwJ

I don't know what it'd take for SeaMonkey, tbh. For Thunderbird, we'll want bug 1154448 in the tree, though.

Comment 1

[Axel Hecht [:Pike]]

Attachment: MozReview Request: bug 1223385, refactor mach helper into LocalesMixin, r?rail

Review commit: https://reviewboard.mozilla.org/r/30161/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/30161/

Comment 2

[Axel Hecht [:Pike]]

Attachment: MozReview Request: bug 1223385, use in-tree compare-locales in mozharness, r?rail

Review commit: https://reviewboard.mozilla.org/r/30163/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/30163/

Comment 3

[Axel Hecht [:Pike]]

Attachment: MozReview Request: bug 1223385, use in-tree compare-locales in Makefiles, r?gps

Also fix that the default merge dir in the mach command creates a directory
that's the merge make target, and thus keeps that make target from actually
running.

Review commit: https://reviewboard.mozilla.org/r/30165/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/30165/

Comment 4

[Rail Aliiev [:rail]]

Comment on attachment 8705715 [details]
MozReview Request: bug 1223385, refactor mach helper into LocalesMixin, r?rail

https://reviewboard.mozilla.org/r/30161/#review26861

Comment 5

[Rail Aliiev [:rail]]

Comment on attachment 8705716 [details]
MozReview Request: bug 1223385, use in-tree compare-locales in mozharness, r?rail

https://reviewboard.mozilla.org/r/30163/#review26865

::: testing/mozharness/mozharness/mozilla/l10n/locales.py:164
(Diff revision 1)
> -        status = self.run_command(command, error_list=compare_locales_error_list,
> +        status = self._mach(command, env, error_list=compare_locales_error_list,

Much easier to read! Thanks.

Comment 6

[Gregory Szorc [:gps]]

Comment on attachment 8705717 [details]
MozReview Request: bug 1223385, use in-tree compare-locales in Makefiles, r?gps

https://reviewboard.mozilla.org/r/30165/#review26903

Thank you for explaining the default merge dir change in the commit message, as I would have been scratching my head otherwise.

Comment 7

[Pulsebot]

https://hg.mozilla.org/integration/fx-team/rev/47e2dfac99d5
https://hg.mozilla.org/integration/fx-team/rev/a575f9a6f0d2
https://hg.mozilla.org/integration/fx-team/rev/a87a27864bb8

Comment 8

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/47e2dfac99d5
https://hg.mozilla.org/mozilla-central/rev/a575f9a6f0d2
https://hg.mozilla.org/mozilla-central/rev/a87a27864bb8

Comment 9

[Axel Hecht [:Pike]]

Attachment: adding back abs_compare_locales_dir to fix bustage

Nightlies are busted :-(

https://treeherder.mozilla.org/logviewer.html#?job_id=3079720&repo=mozilla-central is the log.

The removal of the dirs entry was premature, we should add it back so that clobber find the compare-locales checkout that mozharness still does.

We should remove those in a follow-up.

Comment 10

[Pulsebot]

https://hg.mozilla.org/mozilla-central/rev/27eb5e90eeee

Comment 11

[Pulsebot]

https://hg.mozilla.org/mozilla-central/rev/f59d1f7941b0

Comment 12

[Axel Hecht [:Pike]]

Android is still broken, it doesn't find the right merge dirs.

Asking the Sheriff to back all of these out:

https://hg.mozilla.org/mozilla-central/rev/47e2dfac99d5
https://hg.mozilla.org/mozilla-central/rev/a575f9a6f0d2
https://hg.mozilla.org/mozilla-central/rev/a87a27864bb8
https://hg.mozilla.org/mozilla-central/rev/27eb5e90eeee
https://hg.mozilla.org/mozilla-central/rev/f59d1f7941b0

Comment 13

[Pulsebot]

Backout:
https://hg.mozilla.org/mozilla-central/rev/34ff473cbd86
https://hg.mozilla.org/mozilla-central/rev/8fb4806df089
https://hg.mozilla.org/mozilla-central/rev/3e4f0117b9c4
https://hg.mozilla.org/mozilla-central/rev/43e0e5d33880
https://hg.mozilla.org/mozilla-central/rev/02398f2be72b

Comment 14

[Axel Hecht [:Pike]]

Backed this out because I've used mozilla-central as try enough.

This patch turned out to be mozharness more than anything else, so I'm moving this bug over.

I think the patches are an OK start, but this patch needs someone that can actually hack on mozharness, and I can't even get it to run locally. Thus I'm unassigning myself.

Maybe the last problem was that the merge dirs for chrome-% were wrong in the multilocale fennec build. Maybe that wasn't the last problem :-/

We do have things in the code today, notably https://dxr.mozilla.org/mozilla-central/source/dom/locales/en-US/chrome/netErrorApp.dtd, which is breaking the currently used compare-locales parser. That's riding with 45, https://hg.mozilla.org/mozilla-central/rev/b957faece566. So we're already late, sadly.

Comment 15

[Axel Hecht [:Pike]]

PS: desktop nightlies work without https://hg.mozilla.org/mozilla-central/rev/f59d1f7941b0, i.e., within the bootstrap environment, but not with the plain query_env().

Comment 16

[Chris AtLee [:catlee]]

Nick, can you take a look?

Comment 17

[Nick Thomas [:nthomas] (UTC+12)]

Not sure how far I'll get in mach land, but I'll have a look around.

Comment 18

[Justin Wood (:Callek)]

Attachment: Bug 1223385 - Remove unused extract-bookmarks.py.

Review commit: https://reviewboard.mozilla.org/r/63240/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/63240/

Comment 19

[Justin Wood (:Callek)]

Attachment: Bug 1223385 - remove b2g_desktop_multilocale script due to being unused (dougt-encouraged).

Review commit: https://reviewboard.mozilla.org/r/63242/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/63242/

Comment 20

[Justin Wood (:Callek)]

Attachment: Bug 1223385 - use in-tree compare-locales in Makefiles,

Also fix that the default merge dir in the mach command creates a directory
that's the merge make target, and thus keeps that make target from actually
running.

Review commit: https://reviewboard.mozilla.org/r/63244/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/63244/

Comment 21

[Justin Wood (:Callek)]

Attachment: Bug 1223385 - Make sure nothing in mozharness uses (or clones) compare-locales manually.

Review commit: https://reviewboard.mozilla.org/r/63246/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/63246/

Comment 22

[Justin Wood (:Callek)]

Attachment: Bug 1223385 - Don't clone the older copy of compare-locales anywhere.

Review commit: https://reviewboard.mozilla.org/r/63248/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/63248/

Comment 23

[Axel Hecht [:Pike]]

Comment on attachment 8769273 [details]
Bug 1223385 - Remove unused extract-bookmarks.py.

https://reviewboard.mozilla.org/r/63240/#review60046

Removal is cool, it was only useful when we converted bookmarks.html to bookmarks.ini back in the old old days.

Comment 24

[Rail Aliiev [:rail]]

Comment on attachment 8769277 [details]
Bug 1223385 - Don't clone the older copy of compare-locales anywhere.

https://reviewboard.mozilla.org/r/63248/#review60054

Comment 25

[Justin Wood (:Callek)]

Fwiw, I did a try push at: https://treeherder.mozilla.org/#/jobs?repo=try&revision=3d6c071826f9

Also of note though is the buildbot l10n stuff will likely die due to timeouts (it has that problem), but if its timeout-dead that is still a good sign I didn't break it.

Comment 26

[Jordan Lund (:jlund)]

Comment on attachment 8769274 [details]
Bug 1223385 - remove b2g_desktop_multilocale script due to being unused (dougt-encouraged).

https://reviewboard.mozilla.org/r/63242/#review60056

Comment 27

[Jordan Lund (:jlund)]

Comment on attachment 8769276 [details]
Bug 1223385 - Make sure nothing in mozharness uses (or clones) compare-locales manually.

https://reviewboard.mozilla.org/r/63246/#review60060

<p>what does this mean for these: https://dxr.mozilla.org/mozilla-central/search?q=path%3Amozharness%2Fconfigs+compare-locales&amp;redirect=true ?</p>

Comment 28

[Justin Wood (:Callek)]

(In reply to Jordan Lund (:jlund) from comment #27)
> <p>what does this mean for these:
> https://dxr.mozilla.org/mozilla-central/
> search?q=path%3Amozharness%2Fconfigs+compare-locales&amp;redirect=true ?</p>

See the last patch in the series (already reviewed by rail)

Comment 29

[Gregory Szorc [:gps]]

Comment on attachment 8769275 [details]
Bug 1223385 - use in-tree compare-locales in Makefiles,

https://reviewboard.mozilla.org/r/63244/#review60688

Comment 30

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/7f0116af2fd7
Remove unused extract-bookmarks.py. r=Pike
https://hg.mozilla.org/integration/mozilla-inbound/rev/fcb193036063
remove b2g_desktop_multilocale script due to being unused (dougt-encouraged). r=jlund
https://hg.mozilla.org/integration/mozilla-inbound/rev/5278700efd2d
use in-tree compare-locales in Makefiles, r=gps
https://hg.mozilla.org/integration/mozilla-inbound/rev/5e4044707462
Make sure nothing in mozharness uses (or clones) compare-locales manually. r=jlund
https://hg.mozilla.org/integration/mozilla-inbound/rev/c9964faafa19
Don't clone the older copy of compare-locales anywhere. r=rail

Comment 31

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/7f0116af2fd7
https://hg.mozilla.org/mozilla-central/rev/fcb193036063
https://hg.mozilla.org/mozilla-central/rev/5278700efd2d
https://hg.mozilla.org/mozilla-central/rev/5e4044707462
https://hg.mozilla.org/mozilla-central/rev/c9964faafa19