1765375 - GeckoView handling of secondary OS languages leads to sites ignoring primary language

Status: NEW

(Core :: Internationalization, defect)

Description

[Ted Campbell [:tcampbell]]

STR

1. Set OS languages to English (Canada) followed by Simplified Chinese (China)/简体中文(中国) and no others. Note: You can search for "chinese" to get to right spot in ui list.
2. Visit youtube.com
3. Visit a test site to check accept-languages (such as https://manytools.org/http-html-text/browser-language/)

Expected:

• YouTube uses primary languages and renders in English

Actual:

• YouTube comes back in Chinese

A few notes

• Using en-US instead of en-CA does expected thing
• Using both en-CA and en-US leads to youtube in chinese due to how Gecko forms the accept-language priorities.
• The intl.accept_languages pref in about:config can be used to get correct behaviour, but restarting fenix resets it using computeAcceptLanguages https://searchfox.org/mozilla-central/rev/4b3039b48c3cb67774270ebcc2a7d8624d888092/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java#780
• Updating OS language does correctly trigger GV's hook, but the computed value is not ideal

Comment 1

[Ted Campbell [:tcampbell]]

The lack of UI/pref control over this in Fenix seems to mean OS languages currently need to be overridden. Having computeAcceptLanguages only use the top language would solve my case of a regional-variant-of-otherwise-well-supported-language, but wouldn't be ideal for folks who rely on fallback chain regularly.

Comment 2

[Ted Campbell [:tcampbell]]

I should add that in this scenario I am simply trying to add the keyboard IME, but on Android that is done by adding language to system language list.

Comment 3

[Makoto Kato [:m_kato]]

Chrome for Android seems to add language only tag after language Tag. Firefox desktop depends on localized language pack.

Comment 4

[Chris Peterson [:cpeterson]]

Moving this bug to the Internationalization component.

What behavior do we want on Fenix when the user adds secondary languages or keyboards?

In my Android phone's Settings > General management > Language, my default language is "English (United States)". If I add "Deutsch (Deutschland)" as a second option, then Chrome sends Accept-Language en-US,en;q=0.9,de-DE;q=0.8,de;q=0.7. Firefox 99 sends en-US,de-DE;q=0.5.

I tried adding a Chinese input language to my phone's Samsung keyboard, but that didn't affect Chrome's or Firefox's Accept-Language header.

Why doesn't Firefox Android include en as a fallback like desktop Firefox does (en-US,en;q=0.5)?

Comment 5

[Dan Minor [:dminor]]

(In reply to Makoto Kato [:m_kato] from comment #3)

Chrome for Android seems to add language only tag after language Tag. Firefox desktop depends on localized language pack.

Yes, it seems like we should try en if en-CA fails before moving to a different language.

Comment 6

[Dan Minor [:dminor]]

There's a bit of design discussion about vertical fallbacking from the ICU4X project in [1]... in general, we should fallback from language+region to language before trying a different language. It seems like we could introduce this here [2] without too much trouble...

Flod, does this approach make sense to you?

[1] https://github.com/unicode-org/icu4x/blob/main/docs/design/locale_fallback_and_negotiation.md#vertical-fallback
[2] https://searchfox.org/mozilla-central/rev/0ffae75b690219858e5a45a39f8759a8aee7b9a2/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java#789

Comment 7

[Francesco Lodolo [:flod]]

(In reply to Dan Minor [:dminor] from comment #6)

There's a bit of design discussion about vertical fallbacking from the ICU4X project in [1]... in general, we should fallback from language+region to language before trying a different language. It seems like we could introduce this here [2] without too much trouble...

Flod, does this approach make sense to you?

Yes, I think it makes sense. Ideally, I think it should be possible to customize that through the UI (like it happens in desktop), but this is at least an improvement over the current status.

Comment 8

[Dan Minor [:dminor]]

Attachment: Bug 1765375 - Add fallbacks to computeAcceptLanguages; r=#geckoview-reviewers!,#platform-i18n-reviewers!

Comment 9

[Chris Peterson [:cpeterson]]

Dan, what are the next steps for this bug? In the review comments [1], you wondered whether this approach should be abandoned. Agi is no longer at Mozilla, so if you have questions, you can follow up with calu or owlish in the review.

[1] https://phabricator.services.mozilla.com/D144997#5025119

Comment 10

[Dan Minor [:dminor]]

I've been busy with other projects, but I would like to get back to this. I think the next step is to write some unit tests to verify the new behaviour.

Comment 11

[Walter K.]

See also Bug 1813236 for implementing proper page content language UI.