1325870 - [meta] Unify locale selection across components and languages

Status: NEW

(Core :: Internationalization, defect, P3)

Description

[Zibi Braniecki [:zbraniecki][:gandalf]]

My understanding is that the end goal, where we would like to end up with browser UI locale and web content locale selection is a scenario that looks something like this: 

https://labs.braniecki.net/fx-pref-l10n/ 

That mock is a brainstorming result of the vision discussed with :stas and :axel.
It's not set in stone, but covers the major features that circulate.

In order to get there we will want to have four prefs:

1) whether the UI should match OS selection (bool)
2) the list of locales selected for the UI (list)
3) whether the web content should match browser locales (bool)
4) the list of locales selected for the web content (list)

===================

Before we get there, it would be useful to clean up the current paths and preferences used to represent locales.

It seems that in Gecko we have multiple ways various code pieces retrieve the locale:

 - http://searchfox.org/mozilla-central/search?q=general.useragent.locale&case=false&regexp=false&path=
 - http://searchfox.org/mozilla-central/search?q=getSelectedLocale&path=
 - http://searchfox.org/mozilla-central/search?q=getApplicationLocale&path=
 - others?

and I'd like to figure out which ones match to which of the four desired bits of information and attempt to unify callsites.

:pike, :mkato, :stas, :jfktheme - does the proposal look good?

Comment 1

[Axel Hecht [:Pike]]

Clearing my needinfo in favor of one on Philipp. Do you have something in your head for selecting locale-related preferences and defaults? Any other head to redirect to or involve as well?

Comment 2

[Makoto Kato [:m_kato]]

For Chrome UX locale, we shouldn't use web content locale.

> 2) the list of locales selected for the UI (list)

Windows has application locale and system locale.  Android N or later can have multiple locale settings (bug 1256374).  So, we should decide the order of this (ex. first is application locale, 2nd is system locale.) too.

For web content locale (and font selection), the locale order should be different of UX locale.  Even if application locale is Chinese, we should select Japanese locale and font when web content is ja-JP.

Comment 3

[Zibi Braniecki [:zbraniecki][:gandalf]]

> For Chrome UX locale, we shouldn't use web content locale.

Can you elaborate?

When we were working on it for Firefox OS we found out that most users what their apps and web content to be in the same language.
We also found cases where users will want to keep those two lists separate and the proposed UI reflects that.

Is that enough?

> Windows has application locale and system locale.  Android N or later can have multiple locale settings (bug 1256374).  So, we should decide the order of this (ex. first is application locale, 2nd is system locale.) too.

The proposal takes the system locale (list) unless the user unchecks the mark and selects the list manually.
In that scenario we will not follow system locales, but only application locales selected and sorted by the user.

Is that what you mean?

> Even if application locale is Chinese, we should select Japanese locale and font when web content is ja-JP.

I'm not sure if I follow.
If the user will preserve the defaults (and thus follow the OS locale list) and their OS locale is Chinese, then the app will be using Chinese, yes.

If the user will not uncheck the mark in the UI to follow the app locale for the web, Firefox will submit the same locale list  in the http request header.
If the response will be in ja-JP, then of course we will use ja-JP fonts.

Does it sound right or did I miss something?

Comment 4

[(Currently slow to respond) Philipp Sackl [:phlsa] (Firefox UX) please use needinfo]

I feel like I'm missing a lot of context here.
Isn't the language of the browser chrome determined by the build that a user has downloaded?
Also, it sounds to me like this is supposed to go into Firefox preferences? Currently I can select multiple languages there. How does this change interact here?

Comment 5

[Zibi Braniecki [:zbraniecki][:gandalf]]

> I feel like I'm missing a lot of context here.

My fault.

Basically the current way Firefox UI code calls to get the answer to "what's the current language" is very confusing, there are many APIs for that that return different languages (OS language, browser chrome language, and a few others).
Also, with the upcoming switch to l20n, we will gain completely new way to approach language selection in the UI and part of my plan for upcoming months is to prepare Gecko for that transition.

> Isn't the language of the browser chrome determined by the build that a user has downloaded?

We want to move away from that model and make changing languages easier. L20n will enable us to do runtime language changing and L10nRegistry (bug 1280671) will enable us to download and update locales at any time.
This is important for our ability to decouple releases from localization.

So, creating a unified API and migrating current Gecko callsites to it is the first step.

> Also, it sounds to me like this is supposed to go into Firefox preferences? Currently I can select multiple languages there. How does this change interact here?

The way we'd like it to work is:

1) user is downloading a bundle - Firefox+locale - optimally, it's the locale for his OS, but it can also be a different one.
2) user can either keep the locale selection matching his OS locales (the default) in which case we will keep downloading latest versions of language packs for the locales selected in the OS (say, ["de", "en-US"])
3) or he can uncheck the box, and select his locales manually (say, he switches to ["de-AT", "de", "en-US"]. In this case, we will maintain the latest versions of those locales downloaded for the user.
4) user can also decide that he wants to submit different language list for websites by unchecking the second box in the Languages Preferences
5) Then, he can select the locales that will be submitted in the http-request header for websites.

What I'd like to get out of this bug is:

1st half of 2017:
a) Agree on the UX between L10n-Drivers team, Intl team and UX team.
b) Design the APIs that Gecko will use to retrieve the current locale selection
c) File bugs to implement the APIs and switch current calls to the new APIs
d) File a bug to design the new Language selection preferences UI/UX

2nd half of 2017:
e) Once we transition to l20n we will be able to switch to new preferences

Does it make sense?

Comment 6

[Makoto Kato [:m_kato]]

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #3)
> > For Chrome UX locale, we shouldn't use web content locale.
> 
> Can you elaborate?

When browsing Chinese content, should Chrome UI language be change from current Chrome UI language to Chinese when Chinese UI language pack of Firefox is installed?

Even if installing a language pack to Firefox, user may not be able to recognize the language. If it browser web content of not recognized language, user can easily back and/or navigate to another page via address bar since user can recognize the language of browser UI.
 
> When we were working on it for Firefox OS we found out that most users what
> their apps and web content to be in the same language.
> We also found cases where users will want to keep those two lists separate
> and the proposed UI reflects that.
> 
> Is that enough?

Chrome UI language should be package locale at first.  Because user choose the language when downloading it.  If stub installer can use multiple language UI, user will select their language.

If multiple-locale build such as current Android package, we should use system default at first for Chrome UI.

> > Windows has application locale and system locale.  Android N or later can have multiple locale settings (bug 1256374).  So, we should decide the order of this (ex. first is application locale, 2nd is system locale.) too.
> 
> The proposal takes the system locale (list) unless the user unchecks the
> mark and selects the list manually.
> In that scenario we will not follow system locales, but only application
> locales selected and sorted by the user.
> 
> Is that what you mean?

Yes.

> > Even if application locale is Chinese, we should select Japanese locale and font when web content is ja-JP.
> 
> I'm not sure if I follow.
> If the user will preserve the defaults (and thus follow the OS locale list)
> and their OS locale is Chinese, then the app will be using Chinese, yes.
> 
> If the user will not uncheck the mark in the UI to follow the app locale for
> the web, Firefox will submit the same locale list  in the http request
> header.
> If the response will be in ja-JP, then of course we will use ja-JP fonts.
> 
> Does it sound right or did I miss something?

Yes, right.

Comment 7

[Axel Hecht [:Pike]]

Can we look at this from the perspective of users instead of bugs and code?

I've tried to find out some stats about multi-lingualism, and it's hard, because it's hard to define what "can do multiple languages" means precisely, and then measure just that.

One data point that seems to be agreed upon is that some 50% of humanity are mono-lingual. For these users, we need install paths that are in their language, and make it hard to shoot themselves in the foot. Or at least easy to undo damage.
For Desktop, we have mono-lingual installs right now, with localized installers on Windows. On linux, distributions use multi-locale builds with language selection being tied to the OS. For pre-installed versions of Firefox, we might want to do this for other OSes, too. These don't come with UI locale selection code right now.
On iOS, we ship multi-locale without UI locale selection code.
On Android, we ship multi-locale with locale selection code in the Firefox settings, https://support.mozilla.org/en-US/kb/change-default-language-firefox-android. (I think the telemetry on that data is buggy right now, in case you're curious about usage)

The other data point isn't really data, but anecdotal. People might not want to reveal their language preference publicly. Say, you're preferred language is Kurdish and you're living in Turkey. Should each unencrypted request tell every machine on the way say that? There are more combinations of languages and governments that might make users prefer to keep their language preference private.

As for language change, it'd be good to get some use-cases up, so that we know what to design for. Here are some ideas:

- bad UI localization leads to choosing a different UI locale. We've done that in the past when we had to unsupport an unmaintained localization effort. In that scenario, it'd be nice to keep track that the person might actually prefer something else than they have right now.
- bad install process. Not too many people are in that bucket, according to telemetry. There might be folks that picked the wrong version of Spanish. But being able to fix this might be good
- language learners. Users trying to immerse themselves into another language to learn better/faster.

We also have a few cases where we are multi-lingual on the product front. Pocket was only available in a few languages. Devtools are in English for quite a few locales. Cliqz is German only. I also saw add-ons that implemented their own language chooser for their own strings.

Comment 8

[Zibi Braniecki [:zbraniecki][:gandalf]]

> When browsing Chinese content, should Chrome UI language be change from current Chrome UI language to Chinese when Chinese UI language pack of Firefox is installed?

Oh, no. Chrome UI languages never follow content languages in the scenario I proposed.

So, if your UI locales are ["zh-CN", "zh-TW"], then we will keep your UI in "zh-CN" and keep updating the langpack for this locale.

If you keep the content locales in sync, the browser will also submit the same two locales to websites in headers.

> Chrome UI language should be package locale at first.  Because user choose the language when downloading it.  If stub installer can use multiple language UI, user will select their language.
> If multiple-locale build such as current Android package, we should use system default at first for Chrome UI.

That's a good point. And as :pike said, I believe it's worth making it hard for the user to remove the OS locale.

I think one thing we can alter in my proposal is to make it impossible to remove the OS locale even if you uncheck the tick and manually select UI locales.

That would help user get back to a locale he understands (and maybe use it in recovery mode or sth).

But I doubt we can hard-counter a scenario where the user manually selects a locale that he doesn't understand.

Comment 9

[Zibi Braniecki [:zbraniecki][:gandalf]]

Ok, next questions,

Where should the API live?

Let's assume that the API would look something like this:

x.getChromeLocales([context: String]); // ['de', 'pl', 'en-US']
x.getContentLocales([context: String]); // ['fr', 'de', 'pl', 'en-US']

where context is an optional parameter that we can use to allow for different contexts to use different languages.

For example, I can imagine browser in 'pl', but devtools in 'en-US'.
And for content, I can imagine user wanting to use 'pl' in one browsing context (like, maybe container?) and 'en-US' in the other (private browsing?)

I'd suggest we use new API mostly because I'd like to make sure that we don't regress anything, and we can slowly migrate toward the new API.

Should it live in intl/locales? toolkit/components/mozIntl?

On top of that we'll need private APIs to:

1) retrieve OS locale list
2) get/set the "link UI locales to OS locales" and "link content locales to UI locales" (I assume we'll store it in prefs)
3) get/set the lists of locales (I assume we'll store it in prefs)

And later we'll hook it into L10nRegistry and start informing `getChromeLocales` with the available locales (so it will return only the list of locales that users wants and have loaded langpacks).

:pike, does it sound good?
:mkato, :jfkthame - does it sound good? Where should the API live?

Comment 10

[Axel Hecht [:Pike]]

I wonder if there's synergies with content prefs, https://dxr.mozilla.org/mozilla-central/source/dom/interfaces/base/nsIContentPrefService2.idl and https://dxr.mozilla.org/mozilla-central/source/dom/interfaces/base/nsIContentPrefService.idl.

No idea how that maps to containers.

Comment 11

[Zibi Braniecki [:zbraniecki][:gandalf]]

We'll also need an event listener on it:

x.addEventListener('localeschanged', [context, ] callback);

The idea of context here is purely optional, we can skip it in the first iteration, but I think that we have use-cases for potential UX features around it.

Comment 12

[Zibi Braniecki [:zbraniecki][:gandalf]]

Additional question is how will we handle addons.

Two ideas would be

1) add x.getAddonsLocales
2) treat addon-id as a context and use x.getChromeLocales

My gut feeling is to go for (2).

Comment 13

[Jason Duell]

Gandalf asked me to comment on this API from a necko point of view.

It sounds like the main thing for necko here is that we need some way to tell necko (or have it ask) for the list of locales that should be sent in the Accept-Language HTTP header (along with q-values?).  At first glance at least, I don't have a preference as to whether we:

1) store the info as a ready-to-use string somewhere ("da, en-gb;q=0.8, en;q=0.7", etc), possibly as a field in nsILoadinfo?

2) Store some token in nsILoadInfo (or elsewhere) that gives us a context ("devtools", "private-browsing"), etc, that necko can then use to get the string (getContentLocales("devtools") assuming the comments above).

It doesn't make much of a difference at the necko level.

I'm less thrilled about an API that gives us an array and makes necko construct a single header value string out of it for each channel--seems like we should cache the common value, which is unlikely to change.  But we can probably work with whatever API you throw at us.

As far as addons go (vis a vis the Accept-Language header), I'd suggest we simply guarantee that we set the Accept-Language header for a channel during channel creation (if possible--it would require the relevant information to be available at construction time, obviously--when would we mark a channel as 'devtools', for instance?), so that at any point between then and AsyncOpen (or, actually, on-modify-request) an addon could read and modify the Accept-Language header.

Asking Pat for a glance over this in case I'm missing something about l20n in HTTP.

Comment 14

[Zibi Braniecki [:zbraniecki][:gandalf]]

Started drafting code for the LocaleService in bug 1332207 - let's keep the conversation about the UX and API here, and implementation details there.

Comment 15

[Patrick McManus [:mcmanus]]

I doubt the string production per channel is going to add up to much no matter how we do it.. obviously different approaches just shift the complexity around to the last person to yell not it.

and it should be easy enough as jason says to let the addons overwrite.

Comment 16

[Zibi Braniecki [:zbraniecki][:gandalf]]

I've updated the UX prototype: https://rawgit.com/zbraniecki/fx-pref-l10n/master/

New features:

 - OS locale selection scenarios
 - by default the UI and content locale selection is not even displayed to reduce clutter
 - OS locale is selected in the UI and cannot be removed
 - Downloadable locales are marked and downloaded when selected
 - When new locale is picked the confirm request in both, old and new languages is prompted

It's still a mock, so of course I expect it to be buggy and limited, but I think it conveys most of what we've been talking about here.
I'd like to get an opinion from the :phlsa, :pike and :stas on the general direction and what's missing.

Comment 17

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

(In reply to Makoto Kato [:m_kato] from comment #6)
> (In reply to Zibi Braniecki [:gandalf][:zibi] from comment #3)
> > If the user will not uncheck the mark in the UI to follow the app locale for
> > the web, Firefox will submit the same locale list  in the http request
> > header.
> > If the response will be in ja-JP, then of course we will use ja-JP fonts.
> > 
> > Does it sound right or did I miss something?
> 
> Yes, right.

Please keep this issue in mind, that is, web content local may be unclear. Especially, some websites lied to browsers (I don't know whether there are still such website in these days). For example, an element, <foo lang="en-US">, may have non-Western characters (e.g., Chinese characters).

In such case, especially when we meet Chinese characters, we need to guess most preferred language with some hints to use better glyph because glyph of Chinese characters is different in each language. Currently we're using "accept-language" header setting and application's locale:
https://dxr.mozilla.org/mozilla-central/rev/a3978751f45108ff1ae002ecebdc0fa23fc52b84/gfx/thebes/gfxPlatformFontList.cpp#1058,1101,1128

So, be careful if Firefox will change chrome locale dynamically.

Comment 18

[Zibi Braniecki [:zbraniecki][:gandalf]]

I don't think I understand the concern.

My plan is not supposed to affect the algo that chooses which language the website is displayed in.

What we are changing is how we select languages for our UI and how we select the languages we submit in HTTP requests.

So, if currently you download your browser in zh-CH, and in Settings->Content->Languages you have ["zh-CN", "zh-TW", "en-US"], it will work basically the same (except that for browser UI you will also have a fallback list starting with "zh-CH"). If you used that to guess the websites language, you'll still be able to use the same guesses.

All I'll do for the gfxPlatformFontList is replace it with mozILocaleService.getContentLocales() and mozILocaleService.getAppLocales()[0] and it should stay exactly the same.

Does it sound like it answers your concern?

Comment 19

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #18)
> I don't think I understand the concern.
> 
> My plan is not supposed to affect the algo that chooses which language the
> website is displayed in.
> 
> What we are changing is how we select languages for our UI and how we select
> the languages we submit in HTTP requests.
> 
> So, if currently you download your browser in zh-CH, and in
> Settings->Content->Languages you have ["zh-CN", "zh-TW", "en-US"], it will
> work basically the same (except that for browser UI you will also have a
> fallback list starting with "zh-CH"). If you used that to guess the websites
> language, you'll still be able to use the same guesses.

My concern is, if accept-lang setting would be changed dynamically, and that would cause breaking the order to guess the priority of CJKT fonts.

> All I'll do for the gfxPlatformFontList is replace it with
> mozILocaleService.getContentLocales() and
> mozILocaleService.getAppLocales()[0] and it should stay exactly the same.
> 
> Does it sound like it answers your concern?

Although, I'm not sure what sets the getContentLocales() result, the result is same (or similar to) accept-language setting, I think it must be safe.

I'd like to just notifying you guys of CJKT specific issue which isn't listed up at comment 0. So, I've not found actual problem of current idea yet.

Comment 20

[Zibi Braniecki [:zbraniecki][:gandalf]]

> My concern is, if accept-lang setting would be changed dynamically, and that would cause breaking the order to guess the priority of CJKT fonts.

The change is going to be triggered by the user changing locale. Either OS locale, UI locale or content locale.
I don't see it being different from user going currently to Settings->Content->Languages and changing his locales.

What do you see that doesn't happen now, but would happen in the new UX that is breaking for the guessing algorithm?

> Although, I'm not sure what sets the getContentLocales() result, the result is same (or similar to) accept-language setting, I think it must be safe.

It's exactly the same. The only difference is that unless you change it manually, we'll link it to your UI locale selection.

> I'd like to just notifying you guys of CJKT specific issue which isn't listed up at comment 0. So, I've not found actual problem of current idea yet.

Yeah, that makes sense. Thank you. So far it doesn't sound to me like anything that cannot be handled in the UX proposal and once we get first patches to test I'll make sure to run them through the guessing algorithm.

Comment 21

[Axel Hecht [:Pike]]

Let me see if I understand what layout needs preferred languages for:

In particular for CJKT, layout wants to know how likely it is that we'll see those languages, and in which order?
If so, if it hits one of the https://en.wikipedia.org/wiki/CJK_Unified_Ideographs characters, it assigns fonts based on the preferences among CJKT languages?

That triggered me to think: We do have the cld2 language detection code in Firefox. We could keep track of the languages a user actually sees, and based on that give hints both to layout, but also perhaps to prompt a user to change the languages they expose in http headers?

We'd still need good defaults for layout and accept-lang, of course.

Also, if we're in the situation where a user adds a CJKT language to their preferred languages, we might actually improve rendering of a page if we adjusted the fontlist. We do change fonts already when webfonts are loaded, right? Would it be bad if we rerendered a page if our font selection became better?

Masayuki, do you happen to know how good cld2 is to distinguish between CJKT languages?

(Yeah, sorry, my train of thought derailed quite a bit here.)

Comment 22

[Axel Hecht [:Pike]]

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #16)
> I've updated the UX prototype:
> https://rawgit.com/zbraniecki/fx-pref-l10n/master/

Generally speaking, I find it hard to comment on these knowing that we can't implement them anytime soon.

Given different constraints on different versions of Firefox makes that worse. I'm torn on whether I'd give more value to platform-integration here or more value to pan-Firefox consistency.

It'd help me to dissect what platform features map to which parts of the UI.

> New features:
> 
>  - OS locale selection scenarios

This is only available in some of our Firefox products. How would the UX look in versions of Firefox where this is not available?

>  - by default the UI and content locale selection is not even displayed to
> reduce clutter
>  - OS locale is selected in the UI and cannot be removed

Choosing the OS locale here seems weird. In particular for versions of Firefox that are single-locale.

>  - Downloadable locales are marked and downloaded when selected
>  - When new locale is picked the confirm request in both, old and new
> languages is prompted
> 
> It's still a mock, so of course I expect it to be buggy and limited, but I
> think it conveys most of what we've been talking about here.

TBH, I'm not sure about the trust in OS settings your mock-up shows. I'd rather step up in our role as user-agent, and offer good and safe defaults that transfer the Firefox and OS settings to the world wide web and it's intelligence agencies.

Comment 23

[Zibi Braniecki [:zbraniecki][:gandalf]]

(In reply to Axel Hecht [:Pike] from comment #22)
> (In reply to Zibi Braniecki [:gandalf][:zibi] from comment #16)
> > I've updated the UX prototype:
> > https://rawgit.com/zbraniecki/fx-pref-l10n/master/
> 
> Generally speaking, I find it hard to comment on these knowing that we can't
> implement them anytime soon.

What would block us in your opinion?

> Given different constraints on different versions of Firefox makes that
> worse. I'm torn on whether I'd give more value to platform-integration here
> or more value to pan-Firefox consistency.

I don't think I understand the trade-off you're describing.
 
> >  - OS locale selection scenarios
> 
> This is only available in some of our Firefox products. How would the UX
> look in versions of Firefox where this is not available?

I don't understand the question. What OS doesn't allow user to select a locale?

The mockup shows two windows - one is an OS preferences window, which mocks how user can select a locale of their whole OS, the other is a Firefox window which can follow the OS (basically what we do now with intl.locale.matchOS) or not.
 
> >  - by default the UI and content locale selection is not even displayed to
> > reduce clutter
> >  - OS locale is selected in the UI and cannot be removed
> 
> Choosing the OS locale here seems weird. In particular for versions of
> Firefox that are single-locale.

When I say "OS locale is selected in the UI" I mean - when you select Firefox UI locales, the locale that is currently used by the OS is marked as "[OS]" which may help you revert to it in case you mess up and end up with Firefox in a locale you don't understand.

I also made it impossible to remove that locale from selected list (in the worst case, it'll be at the bottom).
 
> TBH, I'm not sure about the trust in OS settings your mock-up shows. I'd
> rather step up in our role as user-agent, and offer good and safe defaults
> that transfer the Firefox and OS settings to the world wide web and it's
> intelligence agencies.

What do you mean by this?

I follow the assumption that most users use OS in their preferred language and when they download an app, they want it to be in the same language, and when they open a web page they want it to be in the same language (user case 0).

In that scenario, how would you like for Firefox to step up in a user-agent role?

Next most common case I believe is for regions which are multi-lingual in which case you're still likely to just add one more locale (because for example user lives in Quebec and ["fr-CA", "en-CA"] makes sense) and the same applies to OS, browser and content.

The rest is for power users who want to chose manually their browser locales or content locales, and we allow them to do that.

Can you provide examples of what Firefox does better *now*, that will not work in my mock?

Comment 24

[Zibi Braniecki [:zbraniecki][:gandalf]]

current list of accept_languages selections by localizers: https://transvision.mozfr.org/string/?entity=toolkit/chrome/global/intl.properties:intl.accept_languages&repo=aurora

Comment 25

[(Currently slow to respond) Philipp Sackl [:phlsa] (Firefox UX) please use needinfo]

Thanks for adding user stories, those help a lot!

> 1) Default
> 
> User has OS in 'de' and downloads Firefox in 'de'. User never changes languages, and wants his browser to be in 'de and his websites to also be displayed in 'de'.
> 
> 2) Default with missing locale
> 
> User has OS in 'de-AT'. Unfortunately, we do not have Firefox in 'de-AT'. Users wants to download the browser in the closest locale ('de' in this case), but when `de-AT` locale becomes available, he'd like Firefox to switch to it. He'd like websites to be displayed in 'de-AT' when possible, and in 'de' as a fallback.
> 
> 3) User changes OS language after downloading Firefox
> 
> User has OS in `en-US`, downloads Firefox in `en-US` and after some time he decides to switch locale to `de`. He expects that the web content will be displayed in `de` as well.

In these cases everything should happen behind the scenes, no UI in Firefox required.

> 4) User downloads Firefox in his locale, but we deprecate support for the locale
> 
> User has OS in `de`, and downloads Firefox in `de`. After several releases, we decide not to support `de` anymore and need to switch him to another locale.

Since there really isn't anything the user can _do_ in such a case, I think this should be mostly silent as well. Maybe open a support article on launch when the locale is removed.

> 5) User downloads Firefox in his OS locale, but decides to switch browser chrome locale.
> 
> User wants to keep OS in `de`, but decides to use Firefox in `en-US`.
> 
> 6) User downloads Firefox in his OS locale, want to keep the browser in the same locale, but websites in different.
> 
> In this case, user wants to see the browser in locale `de`, but websites in `en-US`.

These cases are the ones that need to be handled in Firefox preferences. More thoughts about the UI itself at the bottom...

> 7) User uses Firefox in `de`, but wants to use devtools in `en-US`.

> 8) User has OS in `en-US`, downloads Firefox in `en-US`, and then changes OS locale to `de`
> User expects that we will follow locale selection, download `de` and switch to it.

Isn't this the same as 3?

> 9) Users wants to browse the web in `de`, but when he's in a private mode, he'd like to only use `en-US` to minimize the fingerprinting.

It would be best to make that part of a privacy feature rather than a language feature.

> 10) User browses the web in `de`, but wants to use `en-US` for his `software engineering` container.

There are many, many open questions around containers and about if and how we will expose them to the user. So let's maybe stash this one until we have more clarity here.

> 11) User installs an addon that allows him to select domains for which different language list is submitted

Yeah, by-domain language preferences definitely sound like they should be handled by add-ons rather than core Firefox. Does WebExtensions have the right APIs to do this?

> 12) User has OS in 'en-US' but manually selects 'it' Firefox on the download page.
> 
> The downloaded Firefox has 'follow OS locales' checkbox unchecked, and UI locales selection is ['it', 'en-US'].
> 
> 13) User downloads silesian Firefox and silesian localizer decided that it's best not to tell websites that the user is using this language for political reasons.
> The localization has a curated list of languages to submit to web content that doesn't contain silesian.
> 
> We follow that list unless user unchecks "Follow UI locales" and manually selects different ones.

Makes sense to me!


About the UI itself:
It's good that the sensible default (follow OS) is removing most of the options that aren't relevant for most users. What I am a little concerned about is that the user potentially has to maintain two different lists of languages.
I think the UI that Chrome uses is actually a pretty good example of how this can be done: https://cl.ly/021g2A3V423o

It only uses one list, which works well with a language-centric mental model (rather than an application-centric model). Given that it is probably safe to assume that most users think about languages more in terms of »what languages do I speak«, I think this makes a lot of sense.

It also comes with the benefit of already having been tested in the wild and being roughly consistent with how Windows and macOS handle language selection.

Comment 26

[Zibi Braniecki [:zbraniecki][:gandalf]]

Here's the current plan by Gnome for a similar UI:
 - https://raw.githubusercontent.com/gnome-design-team/gnome-mockups/e6481b793baaad8fae023cc838d2fa1d05b7061d/system-settings/region-and-language/language.png
 - https://raw.githubusercontent.com/gnome-design-team/gnome-mockups/e6481b793baaad8fae023cc838d2fa1d05b7061d/system-settings/region-and-language/region-and-language-panel.png

Comment 27

[Nick Heim]

Hi experts,
i can give you a real user story (maybe you could call it a systems engineer story too).
Here in Switzerland we have 4 official languages (de, fr, it, rm). And even more @ ETHZ (tech. University) with en and es.
Therefore, our FF deployment for Windows is based on en-US with the XPIs globally installed for all those languages.
Until now, we let FF follow the OS with the help of an extension (Locale2MUI).
Starting with FF57, we can stop that thanks, to the now working implementation of "intl.locale.MatchOS" (thanks again Zibi).
But this is only part of the game.
For example, there is no Windows GUI-Language for Romansh, the fourth official language here.
So, if someone likes to use the Romansh FF-UI, the only possibility to switch to it, was the "Locale Switcher" extension.
But you know it: With the change to WE, there is no such thing as locale switcher extension (at least until now).
How will the Romansh user switch to its native interface in FF57?

If i wander trough our computer rooms and watch the students over their shoulders and hear them talk, i see and hear a lot of different languages. And the people are used to use the language that fit the moment. And they estimate, that they can seamless switch to what they need.

Thanks,
Nick

Comment 28

[Zibi Braniecki [:zbraniecki][:gandalf]]

Jorge, Stuart - Andrew pointed you as the right people to NI here.

Basically, seems like the API in Gecko will be relatively straightforward and the biggest task is adding endpoints from AMO for two things:

1) Retrieving a list of locales AMO has langpacks for

Sth that will feed data into the following API POC:

```
let locales = AddonsManager.getAvailableLocales(); // ["fr-CA", "fr-FR", "fr-CH", "de-AT", "de-DE", ...]
// Build a list of available locales to download
for (loc of locales) {
  let option = document.createElement('options');
  option.value = loc;
  option.textContent = Services.intl.getDisplayName(loc);
  select.appendChild(option);
}
```

2) Retrieving a langpack based on a locale code provided

```
async installLangpackAndSwitchToIt(locale) {
  return AddonsManager.installLangpackForLocale(locale).then(
    () => Services.locale.setRequestedLocales([locale]), // switch to locale only after installation succeeded
    (err) => ShowErrorDialog(`Failed to install a language resource: ${err}`),
  );
}
```

This is going to be important for several initiatives around Multilingual Firefox - https://docs.google.com/document/d/1hXi9NF_whnEoUiFvPOMz0eWRBd3cbVWKwk7ZvQy11k8/edit#
including Language Switcher in Preferences, and enterprise policies.

Can I get your help in evaluating the right bugs/issues to be filed for this and potential complexity and timeline for the AMO team?

Comment 29

[Zibi Braniecki [:zbraniecki][:gandalf]]

Wrong bug, sorry!

Comment 30

[Zibi Braniecki [:zbraniecki][:gandalf]]

I think it's time for an update here.

tl;dr -  Lots of progress over the last year!

I blogged about the changes in 2017 [0] and in the first quarter of 2018 [1].

We mostly focused on the lowest level, cleaning up the locale selection and related mechanisms, unifying everything around BCP47 and CLDR and getting us migrated from single-locale to a multi-locale APIs.

We now have restartless langpacks, the platform has APIs for negotiating languages and runtime management of available/requested and everything around it. We also look into OS for some regional prefs.

This year the focus shifts to more user-facing changes and if you look at the current dependencies you'll see UI language selector there, which we hope will come with a dynamic langpack installation.

There are still a lot of areas we didn't touch yet esp. on the build system side and also things like spellchecks/dictionaries/autocomplete, but we're making progress and I'll keep documenting it here.


[0] https://diary.braniecki.net/2018/01/08/multilingual-gecko-in-2017/
[1] https://diary.braniecki.net/2018/03/20/multilingual-gecko-status-update-2018-1/

Comment 31

[Ranjith Raj]

In which release we can expect this feature, waiting from months. excited :)

Comment 32

[Zibi Braniecki [:zbraniecki][:gandalf]]

The backend for this feature landed across many Firefox releases between Firefox 51 and Firefox 57. You can read more about it in my blog post - https://diary.braniecki.net/2018/01/08/multilingual-gecko-in-2017/

The first big piece of front end - in-product locale switcher - is now aiming to land in Firefox 65. You can track bug 1425941 for its progress.

Comment 33

[Ranjith Raj]

Thanks, I'm following the other bug as well.

Once this feature is enable in Firefox 65, will there be no need of https://www.mozilla.org/en-US/firefox/all/ ? 

btw, Firefox roadmap says it was supposed to be launched in Firefox 63. Is it postponed to 65?

Comment 34

[Francesco Lodolo [:flod]]

(In reply to Ranjith Raj from comment #33)
> Thanks, I'm following the other bug as well.
> 
> Once this feature is enable in Firefox 65, will there be no need of
> https://www.mozilla.org/en-US/firefox/all/ ? 
> 
> btw, Firefox roadmap says it was supposed to be launched in Firefox 63. Is
> it postponed to 65?

Please let's move this discussion outside of the tracking bug. The dev-l10n mailing is likely a good place to have this conversation
https://lists.mozilla.org/listinfo/dev-l10n

Comment 35

[Zibi Braniecki [:zbraniecki][:gandalf]]

> Once this feature is enable in Firefox 65, will there be no need of https://www.mozilla.org/en-US/firefox/all/ ? 

There'll be less need for it, but we'll want to carefully asses how people use the new feature and if it doesn't regress the experience before we make any changes to our offered packages.

> btw, Firefox roadmap says it was supposed to be launched in Firefox 63. Is it postponed to 65?

yes