1994642 - Language switcher is showing all languages from New Tab add-ons, breaking the functionality

Status: VERIFIED FIXED

(Firefox :: New Tab Page, defect)

Description

[Alex Cornestean]

Affected versions:

• Firefox Release: 144.0/20251009125714

Description:
After setting a language for displaying menus, messages and notifications in about:preferences, the corresponding language pack is not downloaded and applied, resulting in the UI not changing to the set language.
Checking about:addons shows that no language pack has been downloaded as the “Languages” section on the left sidebar is not displayed.
The language pack can be manually installed from AMO and then the language set from about:preferences. This will properly apply the language change to the browser UI.

Firefox Release 143.0.4, Beta 144.0b9 and Beta 145.0b2 are not affected.

Steps to reproduce:

1. Go to about:preferences and scroll down to the “Language” section
2. Click on “Set alternatives”
3. On the popup window that is displayed, click on “Select a language to add…” and select a language of your choosing
4. Click on “Add”
5. Optionally, select the chosen language and move it to the top of the list via the “Move Up” button
6. Click “OK”
7. Notice the UI language does not change
8. Go to about:addons and notice the “Languages” section is not displayed in the sidebar

Actual results:
When setting a language in about:preferences, corresponding language pack is not downloaded and applied.

Expected results:
When setting a language in about:preferences, corresponding language pack is downloaded and applied.

Comment 1

[Francesco Lodolo [:flod]]

I just tested on macOS with Firefox 144 and I cannot reproduced.

Italian build, added Catalan, the language pack was downloaded and applied.

What's the OS?

Comment 2

[Alex Cornestean]

Hello :flod

I cannot reproduce the issue anymore today, except on Release 144.0 snap build on Ubuntu 24.04 LTS but only when using an existing profile. With a new profile the issue no longer happens.

Breakdown:

• Release 144.0 from https://ftp.mozilla.org/pub/firefox/releases/ no longer exhibits the issue on Ubuntu 24.04 LTS even though yesterday it did.

• Release 144.0 from https://ftp.mozilla.org/pub/firefox/releases/ on macOS does not exhibit the issue either today.

• Release 144.0 from https://ftp.mozilla.org/pub/firefox/releases/ on Windows 11, no longer exhibits the issue either.

Note that today, as yesterday, I tested on new profiles.

Comment 3

[Francesco Lodolo [:flod]]

Thanks. Snap should be bug 1994920.

Comment 4

[Alex Cornestean]

@ :flod

I did more digging around and did find a way to still reproduce. Sorry that I did not catch this before you closed the report.
Even on new profiles if it works to set the language when the profile was just created, restarting the browser and trying to set another language as per the STR in Comment 0 will now fail.

So new STR would be:

1. Create a new profile
2. Go to about:preferences and scroll down to the “Language” section
3. Click on “Set alternatives”
4. On the popup window that is displayed, click on “Select a language to add…” and select a language of your choosing
5. Click on “Add”
6. Optionally, select the chosen language and move it to the top of the list via the “Move Up” button
7. Click “OK”
8. Notice language is applied to UI
9. Restart the browser
10. Repeat Steps 2->7
11. Notice language is no longer applied to UI

Happening on all OSes and snap build on Ubuntu 24.04 LTS

Comment 5

[Francesco Lodolo [:flod]]

I can confirm the issue.

The langpack is downloaded for the first language, but after that Firefox will stop downloading the language pack.
Settings are updated correctly (see about:support, Internationalization & Localization table).

Comment 6

[Francesco Lodolo [:flod]]

I think this is a problem with the code in Settings.

Another thing that doesn't look right to me: after installing a 2nd language, clicking the dropdown will display a full list of languages instead of only the ones available.

Comment 7

[Francesco Lodolo [:flod]]

Just to confirm: tested with ESR and it works as expected.

After installing, in General I can only see the 3–4 languages available in the dropdown. Also, when searching for new languages, I see a lot of languages that should not be listed (Japanese macOS, Mixteco).

Comment 8

[Francesco Lodolo [:flod]]

This is what the available locales list looks like after installing a 2nd language and restarting. No idea what's the cause, but that should be the problem (Firefox thinks these languages are installed).

["zh-CN","ta","ca","cs","ach","es-MX","fa","en-US","my","uk","bn","sq","mai","en-CA","frp","cy","sat","sc","id","skr","ga-IE","ixl","fy-NL","ro","szl","nn-NO","scn","en-GB","fur","hr","bg","wo","brx","ja-JP-macos","ru","uz","et","son","pl","oc","sk","rm","es-AR","mr","he","mix","es-ES","ka","nb-NO","lv","ast","gd","cak","gv","ml","es-CL","ms","pai","pt-PT","trs","gn","hu","crh","si","eu","el","sv-SE","be","meh","ckb","kk","hi-IN","bn-IN","kab","sr","ar","or","hsb","ko","az","de","sw","da","xh","tl","vi","nl","fr","sco","ur","gu-IN","hye","ks","th","kn","gl","xcl","dsb","is","mk","ace","bs","ia","km","lg","eo","lij","ltg","pa-IN","ca-valencia","an","hy-AM","ja","af","br","it","sl","lo","zh-TW","pt-BR","tr","tg","fi","bn-BD","te","ff","lt","as","ne-NP"]

Comment 9

[Francesco Lodolo [:flod]]

Tentatively asking Eemeli, in case the change is coming from the intl layer.

Comment 10

[Francesco Lodolo [:flod]]

I could reproduce on 143, but not on 142.

Finding a regression for this is going to be a challenge, since it requires release/beta to have a functional language switching UI.

Comment 11

[BugBot [:suhaib / :marco/ :calixte]]

The bug is marked as tracked for firefox144 (release). However, the bug still isn't assigned.

:pluk, could you please find an assignee for this tracked bug? If you disagree with the tracking decision, please talk with the release managers.

For more information, please visit BugBot documentation.

Comment 12

[Francesco Lodolo [:flod]]

Turns out there is no need to install a language.

1. Start 144 with a new profile. Check about:support#addons, you should see 144.1.0.
2. Wait a bit, restart Firefox. The New Tab will go to 145.0.20250919.173227. At this point the language switcher will show all languages in New Tab (some of them Nightly locales).

Comment 13

[Donal Meehan [:dmeehan]]

:mconley, adding a new info here for your investigation.
Repro steps from :lgreco

• from a browser console with js repl enabled (or a privileged about page like about:addons) I have retrieved the AddonWrapper for the new tab page add-on: ntp = await AddonManager.getAddonByID("[e-mail]")
• then double-checked it was the train hop version: ntp.version and that the location it is installed is the user profile one ntp.locationName
• if the newtab addon is the train hop one, then in about:suppport we see the whole list of locales registered by the newtabpage trainhop version
• then I force uninstalled it using await ntp.uninstall() and restarted the browser
• at this point about:support was only listing the langpacks I had installed

Comment 14

[Mike Conley (:mconley) (:⚙️) (PTO Dec 22 - Jan 2)]

So the issue here is likely related to the dynamic Fluent file registration that occurs for New Tab upon doing a train-hop (the most recent of which went out yesterday - although another one was active since late September).

We generate the list of supported locales by simply pulling down what's in https://github.com/mozilla-l10n/firefox-l10n, and looking for locales that have the newtab.ftl file defined. That's probably not the right approach if some of these locales are still in-development. We should probably simply use the list from https://searchfox.org/firefox-main/rev/1b3787c361452ef9c9e37d8ffd4c414135d547f5/browser/locales/all-locales from the release channel version.

@eemeli, are we doing the right thing here to register the Fluent files dynamically? We basically ported the technique used for the root certificate messaging work from RemoteL10n.sys.mjs.

Comment 15

[Francesco Lodolo [:flod]]

For the list of locales: the list of locales in firefox-l10n is a superset of locales shipping in Nightly, which is a superset of locales shipping in Beta/Release. We should care only about the latter.

As for the registration, I have the feeling we should register only locales that are available in the build. But letting Eemeli answer that part.

Comment 16

[Mike Conley (:mconley) (:⚙️) (PTO Dec 22 - Jan 2)]

(In reply to Francesco Lodolo [:flod] from comment #15)

For the list of locales: the list of locales in firefox-l10n is a superset of locales shipping in Nightly, which is a superset of locales shipping in Beta/Release. We should care only about the latter.

Got it. How do I get the subsets that I care about then? Is it safe to assume that this maps to all-locales for those channels?

Comment 17

[Francesco Lodolo [:flod]]

Nightly (all-locales): https://searchfox.org/mozilla-central/source/browser/locales/all-locales
Beta (shipped-locales): https://searchfox.org/firefox-main/source/browser/locales/shipped-locales

Comment 18

[Mike Conley (:mconley) (:⚙️) (PTO Dec 22 - Jan 2)]

What about Release? Or is that somehow always the same as Beta in this scenario?

Comment 19

[Francesco Lodolo [:flod]]

Release is shipped-locales from the release branch, beta is from the beta branch.

Both lists change very rarely, shipped-locales even more so.

Comment 20

[Mike Conley (:mconley) (:⚙️) (PTO Dec 22 - Jan 2)]

Got it, thank you!

Comment 21

[Eemeli Aro [:eemeli]]

(In reply to Mike Conley (:mconley) (:⚙️) from comment #14)

@eemeli, are we doing the right thing here to register the Fluent files dynamically? We basically ported the technique used for the root certificate messaging work from RemoteL10n.sys.mjs.

Prior to this (including when I looked at this specific code some time ago) I would've thought that that's fine, but I'd not noticed before that registering the sources sets the available locales.

So you should probably use Services.locale.availableLocales as the value for supported locales, to keep it from changing to something else.

Comment 22

[Mike Conley (:mconley) (:⚙️) (PTO Dec 22 - Jan 2)]

(In reply to Eemeli Aro [:eemeli] from comment #21)

So you should probably use Services.locale.availableLocales as the value for supported locales, to keep it from changing to something else.

Does that break the dynamic newtab Fluent file mapping if, for example, the XPI is installed on a machine just in "en-US"... and then that machine, through about:settings, installs the "pl" langpack? Like, presumably the "pl" newtab.ftl from the XPI wouldn't just get magically included, right?

Would it be better to just register the newtab.ftl for whatever is in shipped-locales for that particular release?

Comment 23

[Francesco Lodolo [:flod]]

(In reply to Mike Conley (:mconley) (:⚙️) from comment #22)

Would it be better to just register the newtab.ftl for whatever is in shipped-locales for that particular release?

That wouldn't solve this bug (as described in comment 8). We would still have a broken language switcher, showing 100+ languages that are not really available and that the user cannot switch to.

Comment 24

[Mike Conley (:mconley) (:⚙️) (PTO Dec 22 - Jan 2)]

(In reply to Francesco Lodolo [:flod] from comment #23)

(In reply to Mike Conley (:mconley) (:⚙️) from comment #22)

Would it be better to just register the newtab.ftl for whatever is in shipped-locales for that particular release?

That wouldn't solve this bug (as described in comment 8). We would still have a broken language switcher, showing 100+ languages that are not really available and that the user cannot switch to.

I see. Then perhaps the right strategy is to use available languages per eemeli, but then to notice when a new language is added to that list, and then register the relevant newtab.ftl then?

Comment 25

[Francesco Lodolo [:flod]]

(In reply to Mike Conley (:mconley) (:⚙️) from comment #24)

I see. Then perhaps the right strategy is to use available languages per eemeli, but then to notice when a new language is added to that list, and then register the relevant newtab.ftl then?

That sounds like the correct approach.

Comment 26

[Mike Conley (:mconley) (:⚙️) (PTO Dec 22 - Jan 2)]

Attachment: Bug 1994642 - Use Services.locales.availableLanguages for dynamic newtab.ftl registration for train-hops. r?eemeli!,#home-newtab-reviewers!

This registers the newtab.ftl Fluent files in a train-hopped XPI just for the
base application's availableLocales. If that set changes (for example, if the
user installs a new language pack in the base app), then the we re-register,
always using the intersection of the available languages from the base app
and the list of locales included with the XPI.

Comment 27

[Donal Meehan [:dmeehan]]

Release triage note: this patch will likely not be uplifted to release since there is already a train hop.

Comment 28

[Mike Conley (:mconley) (:⚙️) (PTO Dec 22 - Jan 2)]

(In reply to Donal Meehan [:dmeehan] from comment #27)

Release triage note: this patch will likely not be uplifted to release since there is already a train hop.

Actually, we might want to consider the uplift. The change that's made in the patch here is in the underlying app code, and not the train-hoppable newtab code.

Comment 29

[Donal Meehan [:dmeehan]]

:moconley the planned Fx144 dot release deadline for release uplift requests is Friday 2025-10-24.
Not sure how risky this patch is to land and also get a beta uplift before then? Maybe this could at least get a beta uplift request to ride the train with Fx145

Comment 30

[Mike Conley (:mconley) (:⚙️) (PTO Dec 22 - Jan 2)]

This is fairly low-risk, imo. Let me start with the beta uplift.

Comment 31

[Mike Conley (:mconley) (:⚙️) (PTO Dec 22 - Jan 2)]

Attachment: Unsigned [e-mail] 146.0.0 to test on 145 Developer Edition

Testing instructions for Developer Edition version 145 with this XPI and the patch applied (to the Developer Edition build):

Setup:

0. Launch Developer Edition, open the Browser Console, and type Services.locale.availableLocales. Remember the result.
1. Download the attached XPI file to somewhere on disk.
2. In about:config , edit the following prefs
3. xpinstall.signatures.required set to false
4. If it doesn't already exist, create a new String preference called browser.newtabpage.trainhopAddon.version set to the string any
5. Go to about:addons , click "Extensions" in side bar
6. Click gear icon and "Install add-on from file"
7. Pick XPI file from the first step.
8. Restart Developer Edition build
9. You should see “New Tab” listed as an Extension in about:addons, and version "146.0.0" listed next to the New Tab Add-ons entry in about:support

Manual test:

1. Open the Browser Console
2. Type in Services.locale.availableLocales
3. Ensure that the list matches the set listed in the first step from Setup

Comment 32

[Phabricator Automation]

firefox-beta Uplift Approval Request

• User impact if declined: Users that are enrolled in a train-hop for newtab may see their list of available locales in about:settings expanded out far beyond what they've both downloaded, and what is currently supported.
• Code covered by automated testing: no
• Fix verified in Nightly: no
• Needs manual QE test: yes
• Steps to reproduce for manual QE testing: Testing instructions: https://bugzilla.mozilla.org/show_bug.cgi?id=1994642#c31
• Risk associated with taking this patch: low
• Explanation of risk level: This patch only touches the Fluent part of how the train-hopping mechanism works, and is very isolated from the rest of the browser.
• String changes made/needed: None.
• Is Android affected?: no

Comment 33

[Mike Conley (:mconley) (:⚙️) (PTO Dec 22 - Jan 2)]

Attachment: Bug 1994642 - Use Services.locales.availableLanguages for dynamic newtab.ftl registration for train-hops. r?eemeli!,#home-newtab-reviewers!

This registers the newtab.ftl Fluent files in a train-hopped XPI just for the
base application's availableLocales. If that set changes (for example, if the
user installs a new language pack in the base app), then the we update the
registration, always using the intersection of the available languages from
the base app and the list of locales included with the XPI.

Original Revision: https://phabricator.services.mozilla.com/D269093

Comment 34

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/2a168f62ddf5
https://hg.mozilla.org/integration/autoland/rev/11457e1fd2cd
Use Services.locales.availableLanguages for dynamic newtab.ftl registration for train-hops. r=eemeli,home-newtab-reviewers,thecount

Comment 35

[amarc]

https://hg.mozilla.org/mozilla-central/rev/11457e1fd2cd

Comment 36

[Pulsebot]

https://github.com/mozilla-firefox/firefox/commit/4f58a3092d87
https://hg.mozilla.org/releases/mozilla-beta/rev/02f98712a1db

Comment 37

[Alex Cornestean]

Verified as Fixed. Tested on the latest Nightly (146.0a1/20251021202405) and Beta Developer Edition (145.0b5/20251021171141 from https://treeherder.mozilla.org/jobs?repo=mozilla-beta&revision=02f98712a1dbee11ace63cdd92ca8429aa2787c1) under Windows 11 and Ubuntu 24.04 LTS as per the STR from Comment 31.
Also tested on Beta Developer Edition 145.0b4 from https://ftp.mozilla.org/pub/devedition/candidates/145.0b4-candidates/build1/ for comparison.

On Nightly and Beta 145.0b5, which contain the fix, running Services.locale.availableLocales the second time after performing the setup will show the same list of locales as the first time, confirming the fix.

By comparison, on Beta 145.b04, without the fix, running Services.locale.availableLocales the second time after performing the setup will show a list of 130 locales as opposed to just 1 when running the snippet the first time.

Comment 38

[Donal Meehan [:dmeehan]]

:mconley, do you want to add a release uplift request here? We could take it in the planned Fx144 dot release.

Comment 39

[Mike Conley (:mconley) (:⚙️) (PTO Dec 22 - Jan 2)]

Yeah, I'll do that now.

Comment 40

[Mike Conley (:mconley) (:⚙️) (PTO Dec 22 - Jan 2)]

Testing instructions for Release version 144 with the patch applied:

Setup:

1. Launch Firefox Release in a new profile, open the Browser Console, and type Services.locale.availableLocales. Remember the result.
2. Download this signed XPI somewhere on disk
3. In about:config , if it doesn't already exist, create a new String preference called browser.newtabpage.trainhopAddon.version set to the string any
4. Go to about:addons , click "Extensions" in side bar
5. Click gear icon and "Install add-on from file"
6. Pick XPI file from the second step.
7. Restart the browser
8. You should see version "145.1.20251009.134757" listed next to the New Tab Add-ons entry in about:support

Manual test:

1. Open the Browser Console
2. Type in Services.locale.availableLocales
3. Ensure that the list matches the set listed in the first step from Setup

Comment 41

[Phabricator Automation]

firefox-release Uplift Approval Request

• User impact if declined: Users that are enrolled in a train-hop for newtab may see their list of available locales in about:settings expanded out far beyond what they've both downloaded, and what is currently supported.
• Code covered by automated testing: no
• Fix verified in Nightly: yes
• Needs manual QE test: yes
• Steps to reproduce for manual QE testing: See https://bugzilla.mozilla.org/show_bug.cgi?id=1994642#c40
• Risk associated with taking this patch: low
• Explanation of risk level: This patch only touches the Fluent part of how the train-hopping mechanism works, and is very isolated from the rest of the browser. It's been verified on Nightly and Beta.
• String changes made/needed: None.
• Is Android affected?: no

Comment 42

[Mike Conley (:mconley) (:⚙️) (PTO Dec 22 - Jan 2)]

Attachment: Bug 1994642 - Use Services.locales.availableLanguages for dynamic newtab.ftl registration for train-hops. r?eemeli!,#home-newtab-reviewers!

This registers the newtab.ftl Fluent files in a train-hopped XPI just for the
base application's availableLocales. If that set changes (for example, if the
user installs a new language pack in the base app), then the we update the
registration, always using the intersection of the available languages from
the base app and the list of locales included with the XPI.

Original Revision: https://phabricator.services.mozilla.com/D269093

Comment 43

[Pulsebot]

https://github.com/mozilla-firefox/firefox/commit/0d8324d8ca5a
https://hg.mozilla.org/releases/mozilla-release/rev/532436d8ebf7

Comment 44

[Alex Cornestean]

Verified as Fixed. Tested on the latest Release (144.0.2/20251027123126) under Windows 11 and macOS 11.3.1 as per the STR from Comment 40.

Running Services.locale.availableLocales the second time after performing the setup will show the same list of locales as the first time, confirming the fix.