[META] Strings l10n-central

RESOLVED FIXED in Firefox 62

Status

()

RESOLVED FIXED
11 months ago
5 months ago

People

(Reporter: Mardak, Unassigned)

Tracking

(Blocks: 1 bug, {meta})

unspecified
Firefox 62
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox60 wontfix, firefox61 wontfix)

Details

User Story

Move A-S localization into mozilla-central/gecko-strings/l10n-central.

File location: browser/extensions/activity-stream/locales/en-US/activity-stream.properties
L10n/GS location: browser/extensions/activity-stream/activity-stream.properties
(following what formautofill does)


Add file to m-c with l10n.toml/l10n.ini changes, r?Pike

Ride that to quarantine, stop pontoon sync.

Create l10n files from https://github.com/mozilla/activity-stream-l10n via hg convert
a) new root and merge
b) rebase onto current default via
    hg log -r default -T'f9f7148c720362366d60d3e056f08bc76e7c3faa {node}\n' > .hg/shamap

Forward from quarantine to gecko-strings and then to pontoon. Switch sync back on.

Life goes on: new strings for A-S land in m-c, and get localized as part of Firefox. Picked up by A-S as part of pre-render dumps on the next landing until we have pre-rendering in m-c.
(Reporter)

Description

11 months ago
This will track the work for moving the strings process that uses currently uses https://github.com/mozilla/activity-stream-l10n to https://hg.mozilla.org/l10n-central/ so localization can be done more like the rest of Firefox instead of our special experimental-feature/go-faster process.

At minimum, activity stream checks in a .properties file to mozilla-central as part of export (npm run buildmc) as well as a replacement for getting the localized strings (npm run strings-import).

This removes the separate step for landing strings (npm run strings-export):
https://github.com/mozilla/activity-stream-l10n/pulls?q=is%3Aclosed

Pike says he will think more about how to get the existing strings moved, e.g.,
from https://github.com/mozilla/activity-stream-l10n/blob/master/l10n/de/strings.properties
to https://hg.mozilla.org/l10n-central/de/file/tip/browser/extensions /… ?

The work here is separate from fluent work bug 1438748 and moving to components/newtab bug 1329955.

Also not covered here is increasing the frequency of getting localized strings into nightly/beta as that would need to be able to generate prerendered html/js files as part of builds that depend on node bug 1187958.
(Reporter)

Comment 1

11 months ago
Pike, any suggested reasonable ordering of what needs to be done? Will things break if we somehow checked in a .. activity-stream.properties (?) to each l10n-central/<locale> repository? Then we can test a new strings-import implementation. Then finally actually export / land the .properties to m-c?

We happen to be in a self-imposed string freeze for the current 61.4 iteration ending may 7, and we could hold off any new strings early in 62.1. Although, we do have bug 1445090 to remove unused strings.. which could land during this current iteration or we could defer it assuming that simplifies the work here. ?
Iteration: 62.1 - May 21 → ---
Flags: needinfo?(l10n)
See Also: → bug 1445090
(Reporter)

Updated

11 months ago
Depends on: 1457578
(Reporter)

Updated

11 months ago
Depends on: 1457580
(Reporter)

Updated

11 months ago
Depends on: 1457582

Comment 2

11 months ago
Sorry for the lag.

I'm putting my thoughts into the user-story, that's a draft for now. Putting them there 'cause then we can edit that as we get to a plan.

flod, Matjaz: I'd like to have your input on when we should do the l10n part, and when en-US. I think what I wrote down mimicks what we do for fluent conversions?

flod: Any preference on how to do the git->hg history migration? I tested both with German so far.
flod: Also, preference on when to land the strings cleanup?

Pros of "rebase": linear history, no second root
Pros of "merge": the linear history is fake, and only the merge commit has a race condition.

As for timing, I'm busy with other stuff Mon-Wed, Thursday is a public holiday. Is Friday 11th any good?
User Story: (updated)
Flags: needinfo?(m)
Flags: needinfo?(l10n)
Flags: needinfo?(francesco.lodolo)
We don't necessarily need activity-stream.properties in gecko-strings to add the files to l10n repositories, even if it would be good to do it within a short timespan. Rationale: the file is brand new, Pontoon won't know about it, which means it won't remove it, nor change it.

There's one big concern: how are we going to automate this?

> Create l10n files from https://github.com/mozilla/activity-stream-l10n via hg convert

So far so good

> a) new root and merge
> b) rebase onto current default via
>    hg log -r default -T'f9f7148c720362366d60d3e056f08bc76e7c3faa {node}\n' > .hg/shamap

Can you detail with an example how this would work?

Secondary concern: there are a bunch of locales with pending suggestions, that would be lost (~400 at the moment). We can communicate what's going to happen to those suggestions, and ask teams to review content, but that's pretty much it. Also considering that we have to disable that project before starting to move things.

I can't really judge the May 11th timeframe without a better idea of what's needed to make this scale for over 100 languages.
Flags: needinfo?(francesco.lodolo)
LGTM.

While Pontoon sync is stopped, I can try to move Activity Stream "files" in Pontoon DB to Firefox. That way we'll keep data that's not stored in the VCS (e.g. we have >400 suggestions ATM).
Flags: needinfo?(m)

Comment 5

11 months ago
The way to convert is to use the file map like this:

include "l10n/de/strings.properties"
rename "l10n/de/strings.properties" "browser/extensions/activity-stream/activity-stream.properties"

(modified per locale)

and then run

    hg convert --filemap fm activity-stream-l10n l10n-central/$loc

Depending on which way we go, we need to either 

    hg log --cwd l10n-central/$loc -r default -T'f9f7148c720362366d60d3e056f08bc76e7c3faa {node}\n' > l10n-central/$loc/.hg/shamap

before or 

    hg merge -r tip; hg commit -m'Merge activity stream history into l10n-central'

after the migration, and then push. The actual `hg convert` took some 10 seconds or so.

Technical detail about the `hg log` thingie: This tells convert that there's a revision in hg that maps to f9f7148c720362366d60d3e056f08bc76e7c3faa. That is the first commit in git, and doesn't contain any localizations yet. So at the point it reaches a locale change, it says "oh, i know which hg revision to use as parent of this", and that's the default head ;-)
After getting some clarifications over IRC, and trying locally, I think May 11th works. 

Happy to review the script to run this, noting that we need to include locales not on hg.mozilla.org, but also the ones on BitBucket. Something from this script would be reusable to get the fill list
https://github.com/flodolo/scripts/blob/master/mozilla_l10n/update_locales.py
(Reporter)

Comment 7

11 months ago
With the May 11th date, should we go ahead with merging string cleanup right now or wait until after migration?

string cleanup: https://github.com/mozilla/activity-stream/pull/4140
(In reply to Ed Lee :Mardak from comment #7)
> With the May 11th date, should we go ahead with merging string cleanup right
> now or wait until after migration?
> 
> string cleanup: https://github.com/mozilla/activity-stream/pull/4140

Missed also Pike's question.

(In reply to Axel Hecht [:Pike] from comment #2)
> flod: Also, preference on when to land the strings cleanup?

I think it would be good to clean up before we move the files to central, and get that out of the way.
(Reporter)

Comment 9

11 months ago
Okay, and only locales that modify strings will result in the .properties actually cleaning up the strings, so I suppose actually it's not really any different in that localizers who would have touched strings would result in some migration updates and those who don't touch strings will not cause .properties changes and will get removed after migration + new strings.
(In reply to Ed Lee :Mardak from comment #9)
> Okay, and only locales that modify strings will result in the .properties
> actually cleaning up the strings, so I suppose actually it's not really any
> different in that localizers who would have touched strings would result in
> some migration updates and those who don't touch strings will not cause
> .properties changes and will get removed after migration + new strings.

Note: if we really care, we can force Pontoon to export all locales, and remove empty files (with only comments) generated in the process.
(Reporter)

Comment 11

10 months ago
Pike, does these look right?

import: https://github.com/mozilla/activity-stream/pull/4148/files
export: https://github.com/mozilla/activity-stream/pull/4149/files

For import, it checks l10n-central to find locales then fetches each locale's raw tip file for l-c/browser/extensions/activity-stream/activity-stream.properties to a-s/locales/<locale>/strings.properties

For export, it just copies a-s/locales/en-US/strings.properties to m-c/browser/extensions/activity-stream/locales/en-US/activity-stream.properties

Does anything else need to happen in m-c for the unbuilt m-c activity-stream.properties (i.e., no need for moz.build or jar.mn?)
Flags: needinfo?(l10n)

Comment 12

10 months ago
flod and I have some changes requested on the patches.

We're also looking at bug 1329955, though. Do you intend to move your code into browser/components soon? 'Cause then we'd move the l10n file out of browser/extensions into browser/chrome/browser, where the rest of the components are.

It'd be advantageous to only do the migration once.
Flags: needinfo?(l10n) → needinfo?(edilee)
(Reporter)

Comment 13

10 months ago
If it's just a directory to put strings in mozilla-central and to get from l10n-central, I think it'll be fine to put them into browser/components now. I don't know when that will happen yet, but that's where we're headed… (pretty sure…) :)

I'm guessing it would be mozilla-central/browser/locales/en-US/newtab/activity-stream.properties ??

I suppose planning a little bit more ahead -- at least right now, all the importing / "building" / packaging happens in activity stream repository and gets exported. If that were to happen as part of mozbuild, does the file location of activity-stream.properties matter?
Flags: needinfo?(edilee) → needinfo?(l10n)
(Reporter)

Comment 14

10 months ago
Oh, oops. I see you suggested browser/chrome/browser, which I would guess could be mozilla-central/browser/locales/en-US/chrome/browser/activity-stream.properties and l10n-central/<locale>/browser/chrome/browser/activity-stream.properties
(Reporter)

Comment 15

10 months ago
Oh, I suppose complicating the location / name / directory structure is all the activity-stream related strings, e.g., about:library, about:welcome, AS router, etc. And unclear if they'll live close in browser/components/newtab or if they'll share strings…
(Reporter)

Comment 16

10 months ago
Also, I see that the new preferences ftl are now in browser/locales/en-US/browser/preferences where previously in browser/locales/en-US/chrome/browser/preferences

Should activity-stream go to just browser instead of chrome/browser ? naming.. so tricky ;)

Updated

10 months ago
Flags: needinfo?(l10n)
(Reporter)

Comment 17

10 months ago
Looks like we're all done here! Thanks all. (I guess we'll have a separate META for fluent, etc.)
Status: NEW → RESOLVED
Last Resolved: 10 months ago
Resolution: --- → FIXED
(Reporter)

Updated

10 months ago
Target Milestone: --- → Firefox 62
(Reporter)

Updated

5 months ago
Depends on: 1499901
(Reporter)

Updated

5 months ago
Depends on: 1497565
You need to log in before you can comment on or make changes to this bug.