Closed Bug 1235419 Opened 9 years ago Closed 9 years ago

Migrate fennec locales to input

Categories

(Input :: General, defect)

Product:
Input
Component:
General
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: hoosteeno, Unassigned)

References

Details

We need to migrate fennec's existing localized strings into input. Those strings are available in transifex. Specifically... 1) each string has an ID here: http://mxr.mozilla.org/mozilla-central/source/mobile/android/locales/en-US/chrome/aboutFeedback.dtd 2) each ID can be used to grab translated strings like so: https://transvision.mozfr.org/api/v1/entity/aurora/?id=mobile/android/chrome/aboutFeedback.dtd:pageTitle
I have a script that seems to work: https://gist.github.com/hoosteeno/eced76e00e3a5e3a409b I could use a r? on this (for which I'm needinfoing :willkg); if it passes that, I think I need someone with SVN commit privileges to run it and commit. :flod, feel free to comment on the code/approach.
Flags: needinfo?(willkg)
I'm not sure I understand this script. How do you run it? To answer the svn questions, I can commit to svn, so I can handle that aspect of things. I usually coordinate things like this with mathjazz, but happy to coordinate with flod instead if that's more appropriate. My biggest concern is that this creates new strings that don't exist in the current fjord code-base, yet. So if we run this script now and then later do a strings extraction, all these strings go fuzzy and we could end up in a complex unhappy state. I think the better order of operations is something like: 1. finish up work on bug #1225676 which adds the strings to the codebase 2. land that, but don't push to production 3. do a strings extraction which updates the django.pot file with all the new strings 4. run this script which updates all the locale django.po files with the translations from fennec 5. run a strings merge to make sure everything is correct and that the msgids line up 6. go through all the changes to make sure everything is correct and fix any issues 7. sync with svn 8. push to production If we do it in that order, then: 1. we won't be in danger of accidentally making strings go fuzzy 2. we don't waste translators' time 3. we get a chance to make sure it's all correct so we're guaranteed to have the Firefox for Android feedback form be translated from the get go Input doesn't have a lot of strings, so I think steps 3 through 8 are doable in an afternoon. I've done vaguely similar things before. I can do this work. Does that make sense? Are there complexities that aren't addressed, yet?
Flags: needinfo?(willkg)
(In reply to Will Kahn-Greene [:willkg] from comment #2) > To answer the svn questions, I can commit to svn, so I can handle that > aspect of things. I usually coordinate things like this with mathjazz, but > happy to coordinate with flod instead if that's more appropriate. :mathjazz is the right person for tools (CCing him so he's aware of the discussion). Right now the only tool bound to Input is Pontoon (we got rid of Verbatim), and Verbatim was the only reason we needed to sync in advance (it didn't commit automatically to SVN, so the risk was to lose work stored in Verbatim but never exported to the repository). > I think the better order of operations is something like: > 1. finish up work on bug #1225676 which adds the strings to the codebase > 2. land that, but don't push to production > 3. do a strings extraction which updates the django.pot file with all the > new strings > 4. run this script which updates all the locale django.po files with the > translations from fennec > 5. run a strings merge to make sure everything is correct and that the > msgids line up > 6. go through all the changes to make sure everything is correct and fix any > issues > 7. sync with svn > 8. push to production The plan looks good, I would add: * We want to fix bug 1235416 before that. * For #6 I can do a full check on the Italian file.
(In reply to Justin Crawford [:hoosteeno] [:jcrawford] from comment #1) > :flod, feel free to comment on the code/approach. The approach seems reasonable: * I'm not sure why we have locales with dashes and underscores in there. Might worth a ping to Matjaz to see if we can fix that. * Make sure you ignore symlinks. Currently I'm not sure where po_files is updated (is the code complete? I'm a bit lost too on how you'd run this). There is one tricky group of strings <!ENTITY support.pre3 "Visit our "> <!ENTITY support.link2 "Support Site"> <!ENTITY support.post3 " to get answers for common issues."> You don't need this is a po file, you can have a single string: 'Visit our <a href="%(url)s">Support Site</a> to get answers for common issues.' Ideally you should concatenate these three entities and add a single string to the .po, or add a second run to just merge the 3 strings directly in the .po (might be easier).
> I'm not sure I understand this script. How do you run it? > is the code complete? Sorry about that. The code is fairly complete. But it's not beautiful or documented. I created a git repo with a README to help answer the "how do I run it" question (and create the possibility of fixes): https://github.com/hoosteeno/tounguescraper > I think the better order of operations is something like: <snip> This sounds great. And it highlights the one gap in the code, which is, "how do you use polib to replace entries in a pofile?" I will try to discuss with :willkg today. I may not be able to dig back into this project deeply until next week. Thanks all!
> And it highlights the one gap in the code, which is, "how do you use polib to replace entries in a pofile?" I will try to discuss with :willkg today. Scratch that, figured it out. https://github.com/hoosteeno/tounguescraper/commit/55efc266ce572c662766767825e6c6999d9765a7 Next I'll work on bug #1225676.
No longer blocks: 1225676
Depends on: 1225676
(In reply to Francesco Lodolo [:flod] from comment #4) > * I'm not sure why we have locales with dashes and underscores in there. > Might worth a ping to Matjaz to see if we can fix that. That must be a relic from the past - Verbatim only supported underscores, and I added locales with dashes in bug 1235416.
The steps in Comment 2 have been done, except for "sync with svn" and "push to production". We have a diff file for the svn sync. :matjaz or :flod, would one of you take a look at the diff and shout if you see glaring flaws? https://gist.githubusercontent.com/Osmose/f745a5d219eef0986f69/raw/4db54825bd0e54b9c108f2bbcfa31ee5244ed360/string_changes.diff The specific content that was added for this bug is all attached to files called "android_feedback" and "android_thanks". There are lots of other changes in this diff, though -- :osmose ran a msguniq command that might have made those.
Flags: needinfo?(francesco.lodolo)
(In reply to Justin Crawford [:hoosteeno] [:jcrawford] from comment #8) > :osmose ran a msguniq command that might have made those. To clarify; the first time I ran hoosteeno's import script and then ran the merge command, the merge failed due to duplicate strings, presumably introduced by the import script. A bit of searching informed me about msguniq, and I went through the process again, this time running msguniq against all of our po files before running the merge command, and things went smoothly. One thing I'm not sure of is how msguniq handles the combining of strings. I ran it without any extra options beyond an output file. I can re-run it with whatever options are necessary if the diff is wrong anywhere.
I spot checked the full file, and checked the resulting Italian .po, everything looks good as far as I can tell. Only two strings are missing in the Italian file, one is the concatenation from comment 4, another one is where "Firefox for Android" is replaced by "%(product)s". > Tell us what you think about %(product)s so far. Talking about that: we don't have a "Firefox for Android" string in the .po? That seems an error.
Flags: needinfo?(francesco.lodolo)
(In reply to Francesco Lodolo [:flod] from comment #10) > I spot checked the full file, and checked the resulting Italian .po, > everything looks good as far as I can tell. Thanks :flod! > > Only two strings are missing in the Italian file, one is the concatenation > from comment 4, I see those strings -- they're still not concatenated. This is the case only in some .po files (including Italian); I'm not sure why, but it pre-dates the work in this bug. Do you think it is a blocker for this bug? Or would a separate bug be OK for it? +#~ msgid "Visit our" +#~ msgstr "Visita il nostro" +#~ msgid "Support Site" +#~ msgstr "sito di supporto" +#~ msgid "to get answers for common issues." +#~ msgstr "per trovare la soluzione ai problemi più frequenti." > another one is where "Firefox for Android" is replaced by > "%(product)s". > > > Tell us what you think about %(product)s so far. > > Talking about that: we don't have a "Firefox for Android" string in the .po? > That seems an error. This seems to be very common across all the strings -- it doesn't seem to be introduced by this specific migration. If you agree, I can file a separate bug for it.
(In reply to Justin Crawford [:hoosteeno] [:jcrawford] from comment #11) > I see those strings -- they're still not concatenated. This is the case only > in some .po files (including Italian); I'm not sure why, but it pre-dates > the work in this bug. Do you think it is a blocker for this bug? Or would a > separate bug be OK for it? > > +#~ msgid "Visit our" > +#~ msgstr "Visita il nostro" > > +#~ msgid "Support Site" > +#~ msgstr "sito di supporto" > > +#~ msgid "to get answers for common issues." > +#~ msgstr "per trovare la soluzione ai problemi più frequenti." They're commented out (obsolete). Definitely not a blocker, localizers will see the new string to localize. > This seems to be very common across all the strings -- it doesn't seem to be > introduced by this specific migration. If you agree, I can file a separate > bug for it. Uhm, never mind. There is a "Firefox for Android" string in the .po, so you can ignore this comment. For me this is good to merge.
The strings have been migrated successfully and can be seen in action on prod: https://input.mozilla.org/en-US/feedback/android/
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.