l10n-bumper for desktop repacks

RESOLVED FIXED

Status

Release Engineering
Release Automation
P2
normal
RESOLVED FIXED
a year ago
5 months ago

People

(Reporter: aki, Assigned: aki)

Tracking

unspecified
Dependency tree / graph

Firefox Tracking Flags

(firefox58 fixed)

Details

MozReview Requests

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(9 attachments, 1 obsolete attachment)

59 bytes, text/x-review-board-request
Callek
: review+
Details | Review
59 bytes, text/x-review-board-request
Callek
: review+
Details | Review
59 bytes, text/x-review-board-request
Callek
: review+
Details | Review
59 bytes, text/x-review-board-request
Callek
: review+
Details | Review
59 bytes, text/x-review-board-request
Callek
: review+
Details | Review
2.99 KB, patch
Callek
: review+
Details | Diff | Splinter Review
1.35 KB, patch
Callek
: review+
Details | Diff | Splinter Review
1.03 KB, patch
RyanVM
: review+
Details | Diff | Splinter Review
20.75 KB, patch
Pike
: review+
Details | Diff | Splinter Review
(Assignee)

Description

a year ago
As noted in bug 1345340, it would make sense to bump desktop l10n changesets as well.

We can do two approaches:

completeness / convergence
==========================
- bump l10n-changesets.json for desktop  (use all-locales + elmo revisions + a platform map for this)
- change all desktop repacks + release automation to use l10n-changesets.json
- change shipit to know about l10n-changesets.json

pros:
- same format as mobile
- same/similar logic as mobile
- one file to reference

cons:
- more work / things to break

less short-term work
====================
- bump shipped-locales for desktop (use all-locales + elmo revisions, only do thi s on m-b?)
- don't change logic in repacks or release automation

opposite pros/cons from above
(Assignee)

Comment 1

a year ago
It's probably not too much extra work to bump both shipped-locales and an l10n-changesets.json file; that way we'd have a window of time to migrate everything to point to l10n-changesets.json.
Priority: -- → P2
(Assignee)

Comment 2

5 months ago
... I think l10n bumper for desktop may Just Work in taskcluster.

For our desktop nightly l10n [1], we're already passing in `locale=pa-IN:default locale=gd:default` to desktop, from here [2]. But if the locales file ends in .json, we use this logic [3], which hardcodes the `android` string, but we can change it to use `build_platform`. We already have `android-api-15` in the mobile l10n changesets file; we can update this to -16 if needed [4].

We specify the locales file here [5].

I think we:

- add a browser/locales/l10n-changesets.json to each repo, then start bumping them
 - m-c will have `default` revisions; m-b will get the revisions from the dashboard like fennec
- fix the mobile bumper to specify android-api-16
- fix the l10n logic to unhardcode the `android` string, and check for `build_platform`
- change the nightly-l10n kind to point `default` at `browser/locales/l10n-changesets.json`

[1] https://tools.taskcluster.net/groups/UdvyXhWmQ9SWGOQOLBK2yg/tasks/NjaS2FSXT5KGmrEr59LH6g/details
[2] https://hg.mozilla.org/mozilla-central/file/tip/taskcluster/taskgraph/transforms/l10n.py#l153
[3] https://hg.mozilla.org/mozilla-central/file/tip/taskcluster/taskgraph/transforms/l10n.py#l142
[4] https://hg.mozilla.org/mozilla-central/file/tip/mobile/locales/l10n-changesets.json#l5
[5] https://hg.mozilla.org/mozilla-central/file/tip/taskcluster/ci/nightly-l10n/kind.yml#l31
(Assignee)

Comment 3

5 months ago
I suppose that will need to be `android-api-16-nightly` as build_platform. Also, for posterity, the `tip` in the urls should be `f8dd3f21e434`.

Comment 4

5 months ago
Pike, do you have any concerns on your end with c#2?
Flags: needinfo?(l10n)
(Assignee)

Comment 5

5 months ago
Ah, we also have to update https://dxr.mozilla.org/mozilla-central/source/build/sparse-profiles/taskgraph with browser/locales/l10n-changesets.json.
(Assignee)

Comment 6

5 months ago
I think this should do it: https://github.com/escapewindow/gecko/commits/bumper
When I diff the new task-graph against the clean task-graph (but with the sorting patch applied), the task graphs are identical. When I add revisions to browser/locales/l10n-changesets.json, the nightly task graph gets revisions applied.

I just merged central->date and kicked off some nightlies as a baseline. I can land these commits on date and trigger new nightlies.

Assuming we get this landed on central, we'll need a followup puppet patch to remove the aurora bumper and adjust the revisions of the central and beta bumpers.
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
(Assignee)

Comment 12

5 months ago
When these patches land, we need a puppet patch to update/remove the various l10n-bumpers.
Keywords: leave-open
(In reply to Justin Wood (:Callek) from comment #4)
> Pike, do you have any concerns on your end with c#2?

I'm waiting on this before I review at the moment. It's my understanding that we're not yet blocked on this bugs reviews.

Comment 14

5 months ago
Sorry for the lag, we were deep down in a workweek, and I didn't dare to page this into memory.

The only thing that I'd like to get a better understanding of is how we keep locales in Nightly or Beta?

Another question that we'll ask some time late this year or early next year is wether we can ship updates to localizations on release and ESR. We'll be able to do so technically with cross-channel, and my guess is that it'll be easier from a release automation POV with bumper, too, but I'd like to ask explicitly. Any concerns on that in the light of this patch, just from a technical POV? This would be good technical information to have when we open this conversation up with release management.
Flags: needinfo?(l10n)
(Assignee)

Comment 15

5 months ago
(In reply to Axel Hecht [:Pike] from comment #14)
> Sorry for the lag, we were deep down in a workweek, and I didn't dare to
> page this into memory.
> 
> The only thing that I'd like to get a better understanding of is how we keep
> locales in Nightly or Beta?

With the bumper, Nightly for desktop uses all-locales, and specifies the revision as `default`. Beta for desktop uses shipped-locales and the l10n dashboard for its locale list and revisions.

> Another question that we'll ask some time late this year or early next year
> is wether we can ship updates to localizations on release and ESR. We'll be
> able to do so technically with cross-channel, and my guess is that it'll be
> easier from a release automation POV with bumper, too, but I'd like to ask
> explicitly. Any concerns on that in the light of this patch, just from a
> technical POV? This would be good technical information to have when we open
> this conversation up with release management.

Once this patch lands, it'll ride the trains, so unless we uplift it, it'll be >=Fx58 only. Once 58 reaches m-r, it'll be static, but anyone can write a patch to update l10n-changesets.json. If we foresee regular locale updates on m-r, we can run bumper there as well. The same is true for ESR59.

Comment 16

5 months ago
mozreview-review
Comment on attachment 8911877 [details]
bug 1345619 - update l10n-bumper to include desktop.

https://reviewboard.mozilla.org/r/183254/#review190558
Attachment #8911877 - Flags: review?(bugspam.Callek) → review+

Comment 17

5 months ago
mozreview-review
Comment on attachment 8911879 [details]
bug 1345619 - land initial browser/locales/l10n-changesets.json.

https://reviewboard.mozilla.org/r/183256/#review190560

::: browser/locales/l10n-changesets.json:7
(Diff revision 1)
> +    "ach": {
> +        "platforms": [
> +            "linux", 
> +            "linux64", 
> +            "macosx64", 
> +            "win32", 

nit: line endings (whole file)?
Attachment #8911879 - Flags: review?(bugspam.Callek) → review+

Comment 18

5 months ago
mozreview-review
Comment on attachment 8911880 [details]
bug 1345619 - look for build_platform in l10n-changesets.json.

https://reviewboard.mozilla.org/r/183258/#review190564
Attachment #8911880 - Flags: review?(bugspam.Callek) → review+

Comment 19

5 months ago
mozreview-review
Comment on attachment 8911881 [details]
bug 1345619 - sorted locales.

https://reviewboard.mozilla.org/r/183260/#review190566
Attachment #8911881 - Flags: review?(bugspam.Callek) → review+

Comment 20

5 months ago
mozreview-review
Comment on attachment 8911882 [details]
bug 1345619 - point at browser/locales/l10n-changesets.json.

https://reviewboard.mozilla.org/r/183262/#review190568
Attachment #8911882 - Flags: review?(bugspam.Callek) → review+
(Assignee)

Comment 21

5 months ago
(In reply to Justin Wood (:Callek) from comment #17)
> Comment on attachment 8911879 [details]
> bug 1345619 - land initial browser/locales/l10n-changesets.json.
> 
> https://reviewboard.mozilla.org/r/183256/#review190560
> 
> ::: browser/locales/l10n-changesets.json:7
> (Diff revision 1)
> > +    "ach": {
> > +        "platforms": [
> > +            "linux", 
> > +            "linux64", 
> > +            "macosx64", 
> > +            "win32", 
> 
> nit: line endings (whole file)?

Yeah. I don't like it, but afaict that's what py2 json outputs; if I edit it, I think l10n-bumper will add them back in. I think py3 json doesn't have the trailing whitespace.
(Assignee)

Updated

5 months ago
Assignee: nobody → aki

Comment 22

5 months ago
Pushed by asasaki@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/13c7532c9df7
update l10n-bumper to include desktop. r=Callek
https://hg.mozilla.org/integration/autoland/rev/b34bd373582d
land initial browser/locales/l10n-changesets.json. r=Callek
https://hg.mozilla.org/integration/autoland/rev/3a9620622d8f
look for build_platform in l10n-changesets.json. r=Callek
https://hg.mozilla.org/integration/autoland/rev/82365523b591
sorted locales. r=Callek
https://hg.mozilla.org/integration/autoland/rev/56158165aeea
point at browser/locales/l10n-changesets.json. r=Callek
(Assignee)

Updated

5 months ago
Blocks: 1397774
(Assignee)

Comment 24

5 months ago
Created attachment 8914536 [details] [diff] [review]
[puppet] use latest bumper script+configs; stop bumping aurora
Attachment #8914536 - Flags: review?(bugspam.Callek)
Comment on attachment 8914536 [details] [diff] [review]
[puppet] use latest bumper script+configs; stop bumping aurora

>diff --git a/manifests/moco-config.pp b/manifests/moco-config.pp
>     $l10n_bumper_env_config = {
>         'jamun'                 => {
>             mozharness_repo     => 'https://hg.mozilla.org/projects/jamun',
>-            mozharness_revision => 'e1af5dd01c02',
>+            mozharness_revision => '8c6b4fd1d769',

No such rev on jamun

>             config_file         => 'l10n_bumper/jamun.py',
>         },
>         'mozilla-central' => {
>             mozharness_repo     => 'https://hg.mozilla.org/mozilla-central',
>-            mozharness_revision => 'e1af5dd01c02',
>+            mozharness_revision => '8c6b4fd1d769',
>             config_file         => 'l10n_bumper/mozilla-central.py',
>         },
>         'mozilla-beta'    => {
>             mozharness_repo     => 'https://hg.mozilla.org/mozilla-central',

Just a drive-by question, why m-c for beta?
Attachment #8914536 - Flags: review?(bugspam.Callek) → review-
(Assignee)

Comment 26

5 months ago
(In reply to Justin Wood (:Callek) from comment #25)
> Comment on attachment 8914536 [details] [diff] [review]
> [puppet] use latest bumper script+configs; stop bumping aurora
> 
> >diff --git a/manifests/moco-config.pp b/manifests/moco-config.pp
> >     $l10n_bumper_env_config = {
> >         'jamun'                 => {
> >             mozharness_repo     => 'https://hg.mozilla.org/projects/jamun',
> >-            mozharness_revision => 'e1af5dd01c02',
> >+            mozharness_revision => '8c6b4fd1d769',
> 
> No such rev on jamun

Ah, good catch. It's not scheduled anyway; maybe I should just remove that.

> >             config_file         => 'l10n_bumper/jamun.py',
> >         },
> >         'mozilla-central' => {
> >             mozharness_repo     => 'https://hg.mozilla.org/mozilla-central',
> >-            mozharness_revision => 'e1af5dd01c02',
> >+            mozharness_revision => '8c6b4fd1d769',
> >             config_file         => 'l10n_bumper/mozilla-central.py',
> >         },
> >         'mozilla-beta'    => {
> >             mozharness_repo     => 'https://hg.mozilla.org/mozilla-central',
> 
> Just a drive-by question, why m-c for beta?

They all use the same revision on m-c. Otherwise I'd have to uplift everything to m-b.
(Assignee)

Comment 27

5 months ago
Created attachment 8914571 [details] [diff] [review]
[puppet] use latest bumper script+configs; stop bumping aurora; remove jamun
Attachment #8914536 - Attachment is obsolete: true
Attachment #8914571 - Flags: review?(bugspam.Callek)

Updated

5 months ago
Attachment #8914571 - Flags: review?(bugspam.Callek) → review+
(Assignee)

Comment 28

5 months ago
Created attachment 8914847 [details] [diff] [review]
fix-central-bumper.diff

Oops. central-changesets.json was my jamun testing config, so I could test bumping both beta-style and central-style. On m-c we want it to be browser/locales/l10n-changesets.json.

Once this merges to m-c, I'll need to bump the revision in puppet, and then we're good.
Attachment #8914847 - Flags: review?(bugspam.Callek)
(Assignee)

Comment 29

5 months ago
Created attachment 8914879 [details] [diff] [review]
dontbuild_bumper.diff
Attachment #8914879 - Flags: review?(ryanvm)
Attachment #8914879 - Flags: review?(ryanvm) → review+

Updated

5 months ago
Attachment #8914847 - Flags: review?(bugspam.Callek) → review+

Comment 30

5 months ago
Pushed by asasaki@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/e4b70f1fc8d6
fix central l10n-changesets.json path. r=pike
https://hg.mozilla.org/integration/mozilla-inbound/rev/1f2bd44fa069
dontbuild on l10n bumper. r=ryanvm
(Assignee)

Comment 31

5 months ago
Once the above changesets make it to central, I'll create another puppet patch and a remove-central-changesets.json patch.
(Assignee)

Comment 33

5 months ago
Created attachment 8915395 [details] [diff] [review]
remove-central-changesets
Attachment #8915395 - Flags: review?(l10n)

Updated

5 months ago
Attachment #8915395 - Flags: review?(l10n) → review+
(Assignee)

Updated

5 months ago
Keywords: leave-open

Comment 34

5 months ago
Pushed by asasaki@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/445c25bcc6fa
remove central-changesets.json. DONTBUILD r=pike

Comment 35

5 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/445c25bcc6fa
Status: NEW → RESOLVED
Last Resolved: 5 months ago
status-firefox58: --- → fixed
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.