Remove ar, fr and zh-TW files in Gaia

RESOLVED FIXED

Status

Firefox OS
Gaia
RESOLVED FIXED
4 years ago
3 years ago

People

(Reporter: stas, Assigned: stas)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Assignee)

Description

4 years ago
Bug 1011519 tracks the work required to move all tests to use pseudolocales and disable ar, fr, and zh-TW in languages.json.  Once this work is done, it will be possible to remove all unused *.properties files.
Seems it can still be useful to have *real* locale files in the codebase to test with. Sometimes bugs do not reproduce with psuedolocales, so it might be useful to have a second *real* locale in the codebase.
I don't think that keeping a real locale in source makes any sense. On the other hand, I believe that generatic a pseudo-locale at build time adds value and covers cases which are not covered by runtime pseudolocale.
If we move to build-time pseudo-locales (which I think we should probably do), then my concerns would be addressed, and I would support removing other locale folders. The biggest problem right now is that they behave differently, so sometimes developers need to go into a different language property file to hack it and test.
(Assignee)

Comment 4

4 years ago
Even if we generate pseudolocales on buildtime (which would be good for performance, but a bit contradicting the fact that they're a runtime setting right now), the files would be generated automatically right before they're put into application.zip, so it would be difficult to change something in them and test. To address the specific use case Kevin mentioned in comment 3, I think we'd need to introduce a new make target, say `make pseudolocalizations` which would generate the files before build time.  Note that right now the qps logic only operates on ASTs, while `make pseudolocalizations` would need to also serialize those ASTs to .properties.

An alternative solution might come from an admittedly unexpected angle.  I hear we might want to keep all localization in the Gaia git repo after all.  I'm not sure how developed this idea is, but if we were to keep the 'master' versions of each localization in Gaia repo then the synchronization issue would be be a problem, mostly.  Pseudolocalizations could still be used for RTL testing and when a new feature is developed for which localization don't exist yet.  Pike, what are your thoughts on this?
Flags: needinfo?(l10n)

Comment 5

4 years ago
I'd run on the assumption that the l10n files stay outside of the gaia repo for the time being. Ad-hoc, in a post-Aisle world, we could consider that, but even then there's a bunch of technical and political stuff to clear out. Like trying to contract LSPs to commit to our github repo sounds challenging. Tree rules on frozen branches, etc etc etc.
Flags: needinfo?(l10n)
(Assignee)

Comment 6

4 years ago
Thanks, Axel.  Looks like we won't be able to do what I had in mind in comment 4.

Going back to Kevin's comment 1:  I'm still not sure I like the idea of having even one additional language in the repo.  It would need to be synchronized periodically or would fall out of sync quickly and become a source of additional confusion when things don't work as expected.

I wonder if we could instead have a special make flag, INCLUDE_TEST_LOCALES, which when set would create a new temp dir, pull from https://hg.mozilla.org/gaia-l10n/fr and set LOCALES_FILE and LOCALE_BASEDIR such that the resulting build would include the freshly fetched localization files and source en-US.  Kevin, do you think this would be a good idea?

One challenge of this approach is how to make sure the files are fetched from the correct repo.  It's https://hg.mozilla.org/gaia-l10n for Gaia master, but https://hg.mozilla.org/releases/gaia-l10n/v2_1 for v2.1 etc.  I think we can start with master for now and figure this out later when we're getting ready to branch.
Flags: needinfo?(kgrandon)
My concerns are actually addressed if we move the pseudo-locales to use a build-time thing. The reason being is that generally we just need to "make sure things work", and today pseudo locales almost provide that, but the fall short. The last problem being when we ran into the issue with private browsing, where we weren't referencing the manifest.webapp. This worked fine for pseudo-locales, but did not work when applied to a real locale. Yes - there are ways we could patch this, and provide warnings to the user, but I feel they will always fall short of working like "real locales" in some way.

Basically if we have one additional real locale (or build-time pseudo-locales) then I'm happy. Using a build flag to fetch external locales might also work, but it seems overly complex to me.
Flags: needinfo?(kgrandon)
(Assignee)

Updated

4 years ago
Depends on: 1101632
(Assignee)

Comment 8

4 years ago
Thanks, Kevin.  This makes sense.  I've adjusted the strategy as follows:

 1. First, let's migrate all tests to use runtime pseudolocales (bug 1011519).
 2. Then, let's make it possible to build pseudolocales on build-time (bug 1101632).
 3. Finally, let's disable and remove ar, fr and zh-TW from the source repo (this bug).
Summary: Remove *.{ar,fr,zh-TW}.properties files from Gaia source repo → Disable and remove ar, fr and zh-TW in Gaia dev builds
(Assignee)

Comment 9

3 years ago
Bug 1101632 disabled ar, fr and zh-TW in dev builds.  Let's remove those outdated *.properties files.
Summary: Disable and remove ar, fr and zh-TW in Gaia dev builds → Remove ar, fr and zh-TW files in Gaia
Created attachment 8568491 [details] [review]
[gaia] stasm:1099321-remove-ar-fr-zh-TW > mozilla-b2g:master
(Assignee)

Comment 11

3 years ago
Comment on attachment 8568491 [details] [review]
[gaia] stasm:1099321-remove-ar-fr-zh-TW > mozilla-b2g:master

Bye bye, *.{ar,fr,zh-TW}.properties :)
Attachment #8568491 - Flags: review?(gandalf)
Comment on attachment 8568491 [details] [review]
[gaia] stasm:1099321-remove-ar-fr-zh-TW > mozilla-b2g:master

r++
Attachment #8568491 - Flags: review?(gandalf) → review+
(Assignee)

Comment 13

3 years ago
Comment on attachment 8568491 [details] [review]
[gaia] stasm:1099321-remove-ar-fr-zh-TW > mozilla-b2g:master

Kevin, I'd like to run this by you as well.  The change only removes *.{ar,fr,zh-TW}.properties files in the repo.  I fixed the tests earlier in bug 1011519.

Treeherder: https://treeherder.mozilla.org/#/jobs?repo=gaia-try&revision=93c4078a670b
Attachment #8568491 - Flags: review?(kgrandon)
Comment on attachment 8568491 [details] [review]
[gaia] stasm:1099321-remove-ar-fr-zh-TW > mozilla-b2g:master

Nice work, looks good to me. Please make sure the try run looks ok before landing.
Attachment #8568491 - Flags: review?(kgrandon) → review+
(Assignee)

Comment 15

3 years ago
Thanks.  I rebased one more time because I was getting an orange Gij.  This one's all green:

https://treeherder.mozilla.org/#/jobs?repo=gaia-try&revision=765c3bbcc5ba
(Assignee)

Comment 16

3 years ago
https://github.com/mozilla-b2g/gaia/commit/1cb43e424af904ce45bcf2bd312dd2c1ca03af2c

¯\_(ツ)_/¯_/¯(ツ)¯\_ ¯\_(ツ)¯\_ _/¯(ツ)_/¯
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
:stas,

I have noticed that HTML files still contains the following <meta> tag:

  <meta name="availableLanguages" content="ar, en-US, fr, zh-TW">

should they be removed too?
Assignee: nobody → stas
Flags: needinfo?(stas)
(Assignee)

Comment 18

3 years ago
You're right, Tim, thanks for pointing that out.  I filed bug 1139293.
Flags: needinfo?(stas)
You need to log in before you can comment on or make changes to this bug.