1603112 - Need to flush locale/intl.properties stringbundle to reload correct intl.accept_language

Status: RESOLVED FIXED

(Core :: Internationalization, defect, P3)

Description

[Jan Horak [:jhorak]]

The intl.accept_language is wrongly set to en_US when using user installed langpack.

The nsStringBundleService observes correctly for intl:app-locales-changed but the flushBundleCache [code] does not flush the chrome://global/locale/intl.properties from mSharedBundles and therefore the intl.accept_language is not updated to the new locale setting.

To "fix" this, I can remove the chrome://global/locale/intl.properties from kContentBundles array [code] but I'm not sure which consequences that brings.

Comment 1

[Jan Horak [:jhorak]]

Most likely related to the bug 1582402.

Comment 2

[Jan Horak [:jhorak]]

mconley: maybe I answered the question from https://phabricator.services.mozilla.com/D46751?vs=on&id=167408#1422395 in the report. The mSharedBundles are never flushed, they are expected to stay static during execution I guess, because if the build is localized (like upstream binaries) it's very unlikely that user installs owns langpack which cause the string bundles to change. But the linux distros is different story. Maybe you could help or address to someone who can?

Comment 3

[Mike Conley (:mconley) (:⚙️)]

I believe kmag wrote the shared memory stuff for nsStringBundle as part of the early efforts to reduce content process memory overhead. Going to redirect to him.

Comment 4

[Kris Maglione [:kmag]]

This is somewhat complicated. Most of the shared memory string bundles are pre-created and cached from a list in nsContentUtils, and are never flushed, so they need to be created initially with the correct locale. If we flushed them in the string bundle service, that would just create inconsistencies across different consumers, so we don't.

I suppose we could flush the ones that aren't in the nsContentUtils cache, but I'm not sure that's the right solution. What's causing this bundle to be initialized before we initialize the language pack in the first place?

Comment 5

[Jan Horak [:jhorak]]

The intl.properties is not in nsContentUtils. Could removing chrome://global/locale/intl.properties from kContentBundles be possible? The intl.properties files are not extra big in the langpacks.

Looks like the request for intl.properties is from installDistributionAddons:

0 get _worker() ["resource://gre/modules/PromiseWorker.jsm":185:34]
this = [object Object]
1 postMessage() ["resource://gre/modules/PromiseWorker.jsm":312:8]
2 AsyncFunctionNext(val = "[object Object],[object Object]") ["self-hosted":689:4]
this = [object AsyncFunctionGenerator]
3 awaitPromise(promise = "[object Promise]") ["resource://gre/modules/addons/XPIProvider.jsm":226:14]
4 installDistributionAddons(aManifests = "[object Object]", aAppChanged = "false", aOldAppVersion = ""71.0"") ["resource://gre/modules/addons/XPIProvider.jsm":2794:32]
this = [object Object]
5 checkForChanges(aAppChanged = "false", aOldAppVersion = ""71.0"", aOldPlatformVersion = ""71.0"") ["resource://gre/modules/addons/XPIProvider.jsm":2887:21]
this = [object Object]
6 startup(aAppChanged = "false", aOldAppVersion = ""71.0"", aOldPlatformVersion = ""71.0"") ["resource://gre/modules/addons/XPIProvider.jsm":2384:11]
this = [object Object]
7 callProvider(aProvider = "[object Object]", aMethod = ""startup"", aDefault = "null", aArgs = "false,71.0,71.0", ""71.0"", ""71.0"") ["resource://gre/modules/AddonManager.jsm":215:30]
8 _startProvider(aProvider = "[object Object]", aAppChanged = "false", aOldAppVersion = ""71.0"", aOldPlatformVersion = ""71.0"") ["resource://gre/modules/AddonManager.jsm":651:16]
this = [object Object]
9 startup() ["resource://gre/modules/AddonManager.jsm":875:13]
this = [object Object]
10 startup() ["resource://gre/modules/AddonManager.jsm":3534:25]
this = [object Object]
11 observe(aSubject = "null", aTopic = ""addons-startup"", aData = "null") ["resource://gre/modules/addonManager.js":87:28]
this = [object Object]

And C++ stacktrace:

#0  0x00007fffd7d5ec09 in nsStringBundleService::getStringBundle(char const*, nsIStringBundle**) (this=0x60d000035f20, aURLSpec=0x6060001deca8 "chrome://global/locale/intl.properties", aResult=0x7ffffff760a0)
at /home/jhorak/f/firefox/firefox-71.0/intl/strres/nsStringBundle.cpp:819
#1  0x00007fffd7d60b15 in nsStringBundleService::CreateBundle(char const*, nsIStringBundle**) (this=0x60d000035f20, aURLSpec=0x6060001deca8 "chrome://global/locale/intl.properties", aResult=0x7ffffff760a0)
at /home/jhorak/f/firefox/firefox-71.0/intl/strres/nsStringBundle.cpp:910
#2  0x00007fffd7c4f478 in nsPrefBranch::GetDefaultFromPropertiesFile(char const*, nsTSubstring<char16_t>&) (this=0x60800002a8a0, aPrefName=0x6030002596c8 "intl.accept_languages", aReturn=...)
at /home/jhorak/f/firefox/firefox-71.0/modules/libpref/Preferences.cpp:2840
#3  0x00007fffd7c4d64f in nsPrefBranch::GetComplexValue(char const*, nsID const&, void**) (this=0x60800002a8a0, aPrefName=0x7fffee1b9da0 <str> "intl.accept_languages", aType=..., aRetVal=0x7ffffff76cc0)
at /home/jhorak/f/firefox/firefox-71.0/modules/libpref/Preferences.cpp:2325
#4  0x00007fffd7cc74c1 in mozilla::Preferences::GetLocalizedString(char const*, nsTSubstring<char16_t>&, mozilla::PrefValueKind)
(aPrefName=0x7fffee1b9da0 <str> "intl.accept_languages", aResult=..., aKind=mozilla::PrefValueKind::User) at /home/jhorak/f/firefox/firefox-71.0/modules/libpref/Preferences.cpp:4695
#5  0x00007fffd7cc70af in mozilla::Preferences::GetLocalizedCString(char const*, nsTSubstring<char>&, mozilla::PrefValueKind)
(aPrefName=0x7fffee1b9da0 <str> "intl.accept_languages", aResult=..., aKind=mozilla::PrefValueKind::User) at /home/jhorak/f/firefox/firefox-71.0/modules/libpref/Preferences.cpp:4682
#6  0x00007fffe1dcf042 in mozilla::dom::IndexedDatabaseManager::Init() (this=0x60d000067410) at /home/jhorak/f/firefox/firefox-71.0/dom/indexedDB/IndexedDatabaseManager.cpp:290
#7  0x00007fffe1d93c58 in mozilla::dom::IndexedDatabaseManager::GetOrCreate() () at /home/jhorak/f/firefox/firefox-71.0/dom/indexedDB/IndexedDatabaseManager.cpp:227
#8  0x00007fffe20d145a in mozilla::dom::WorkerPrivate::GetLoadInfo(JSContext*, nsPIDOMWindowInner*, mozilla::dom::WorkerPrivate*, nsTSubstring<char16_t> const&, bool, mozilla::dom::WorkerPrivate::LoadGroupBehavir, mozilla::dom::WorkerType, mozilla::dom::WorkerLoadInfo*)
(aCx=0x61d000085280, aWindow=0x0, aParent=0x0, aScriptURL=..., aIsChromeWorker=true, aLoadGroupBehavior=mozilla::dom::WorkerPrivate::InheritLoadGroup, aWorkerType=mozilla::dom::WorkerTypeDedicated, aLoadInfo0x7ffffff78600) at /home/jhorak/f/firefox/firefox-71.0/dom/workers/WorkerPrivate.cpp:2493
#9  0x00007fffe20cf6a4 in mozilla::dom::WorkerPrivate::Constructor(JSContext*, nsTSubstring<char16_t> const&, bool, mozilla::dom::WorkerType, nsTSubstring<char16_t> const&, nsTSubstring<char> const&, mozilla::do::WorkerLoadInfo*, mozilla::ErrorResult&, nsTString<char16_t>)
(aCx=0x61d000085280, aScriptURL=..., aIsChromeWorker=true, aWorkerType=mozilla::dom::WorkerTypeDedicated, aWorkerName=..., aServiceWorkerScope=..., aLoadInfo=0x0, aRv=..., aId=...)
at /home/jhorak/f/firefox/firefox-71.0/dom/workers/WorkerPrivate.cpp:2291
#10 0x00007fffe2051624 in mozilla::dom::ChromeWorker::Constructor(mozilla::dom::GlobalObject const&, nsTSubstring<char16_t> const&, mozilla::ErrorResult&) (aGlobal=..., aScriptURL=..., aRv=...)
at /home/jhorak/f/firefox/firefox-71.0/dom/workers/ChromeWorker.cpp:22
#11 0x00007fffde9a5d22 in mozilla::dom::ChromeWorker_Binding::_constructor(JSContext*, unsigned int, JS::Value*) (cx=0x61d000085280, argc=1, vp=0x621000123eb0) at WorkerBinding.cpp:284
#12 0x00007fffe92d3310 in CallJSNative(JSContext*, bool (*)(JSContext*, unsigned int, JS::Value*), js::CallReason, JS::CallArgs const&)
(cx=0x61d000085280, native=0x7fffde9a5070 <mozilla::dom::ChromeWorker_Binding::_constructor(JSContext*, unsigned int, JS::Value*)>, reason=js::CallReason::Call, args=...)

And second:

#0  0x00007fffd7d5fc09 in nsStringBundleService::getStringBundle(char const*, nsIStringBundle**) (this=0x60d000035f20, aURLSpec=0x6060001dc9c8 "chrome://global/locale/intl.properties", aResult=0x7ffffff76ac0)
at /home/jhorak/f/firefox/firefox-71.0/intl/strres/nsStringBundle.cpp:819
#1  0x00007fffd7d61b15 in nsStringBundleService::CreateBundle(char const*, nsIStringBundle**) (this=0x60d000035f20, aURLSpec=0x6060001dc9c8 "chrome://global/locale/intl.properties", aResult=0x7ffffff76ac0)
at /home/jhorak/f/firefox/firefox-71.0/intl/strres/nsStringBundle.cpp:910
#2  0x00007fffd7c50478 in nsPrefBranch::GetDefaultFromPropertiesFile(char const*, nsTSubstring<char16_t>&) (this=0x60800002a820, aPrefName=0x60300026e308 "intl.accept_languages", aReturn=...)
at /home/jhorak/f/firefox/firefox-71.0/modules/libpref/Preferences.cpp:2840
#3  0x00007fffd7c4e64f in nsPrefBranch::GetComplexValue(char const*, nsID const&, void**) (this=0x60800002a820, aPrefName=0x7fffed843b60 <str> "intl.accept_languages", aType=..., aRetVal=0x7ffffff776e0)
at /home/jhorak/f/firefox/firefox-71.0/modules/libpref/Preferences.cpp:2325
#4  0x00007fffd7cc84c1 in mozilla::Preferences::GetLocalizedString(char const*, nsTSubstring<char16_t>&, mozilla::PrefValueKind)
(aPrefName=0x7fffed843b60 <str> "intl.accept_languages", aResult=..., aKind=mozilla::PrefValueKind::User) at /home/jhorak/f/firefox/firefox-71.0/modules/libpref/Preferences.cpp:4695
#5  0x00007fffdc8e43f8 in mozilla::dom::Navigator::GetAcceptLanguages(nsTArray<nsTString<char16_t> >&) (aLanguages=...) at /home/jhorak/f/firefox/firefox-71.0/dom/base/Navigator.cpp:330
#6  0x00007fffe206027f in mozilla::dom::workerinternals::(anonymous namespace)::PrefLanguagesChanged(char const*, void*) () at /home/jhorak/f/firefox/firefox-71.0/dom/workers/RuntimeService.cpp:1080
#7  0x00007fffd7ccd239 in mozilla::Preferences::RegisterCallbackAndCall(void (*)(char const*, void*), nsTSubstring<char> const&, void*, mozilla::Preferences::MatchKind)
(aCallback=0x7fffe2060180 <mozilla::dom::workerinternals::(anonymous namespace)::PrefLanguagesChanged(char const*, void*)>, aPref=..., aClosure=0x0, aMatchKind=mozilla::Preferences::ExactMatch)
at /home/jhorak/f/firefox/firefox-71.0/modules/libpref/Preferences.cpp:5046
#8  0x00007fffe206010f in mozilla::Preferences::RegisterCallbackAndCall<22, void>(mozilla::TypedPrefChangeFunc<void>::Type, char const (&) [22], void*) (aCallback=..., aPref=..., aClosure=0x0)
at /home/jhorak/f/firefox/firefox-71.0/objdir/dist/include/mozilla/Preferences.h:437
#9  0x00007fffe2058154 in mozilla::dom::workerinternals::RuntimeService::Init() (this=0x6110001a3780) at /home/jhorak/f/firefox/firefox-71.0/dom/workers/RuntimeService.cpp:1590
#10 0x00007fffe205707f in mozilla::dom::workerinternals::RuntimeService::GetOrCreateService() () at /home/jhorak/f/firefox/firefox-71.0/dom/workers/RuntimeService.cpp:1162
#11 0x00007fffe20d0353 in mozilla::dom::WorkerPrivate::Constructor(JSContext*, nsTSubstring<char16_t> const&, bool, mozilla::dom::WorkerType, nsTSubstring<char16_t> const&, nsTSubstring<char> const&, mozilla::dom::WorkerLoadInfo*, mozilla::ErrorResult&, nsTString<char16_t>)                                                                                                                                                    
(aCx=0x61d000085280, aScriptURL=..., aIsChromeWorker=true, aWorkerType=mozilla::dom::WorkerTypeDedicated, aWorkerName=..., aServiceWorkerScope=..., aLoadInfo=0x7ffffff78600, aRv=..., aId=...)
at /home/jhorak/f/firefox/firefox-71.0/dom/workers/WorkerPrivate.cpp:2308
#12 0x00007fffe2052214 in mozilla::dom::ChromeWorker::Constructor(mozilla::dom::GlobalObject const&, nsTSubstring<char16_t> const&, mozilla::ErrorResult&) (aGlobal=..., aScriptURL=..., aRv=...)
at /home/jhorak/f/firefox/firefox-71.0/dom/workers/ChromeWorker.cpp:22
#13 0x00007fffde9a6cb2 in mozilla::dom::ChromeWorker_Binding::_constructor(JSContext*, unsigned int, JS::Value*) (cx=0x61d000085280, argc=1, vp=0x621000123eb0) at WorkerBinding.cpp:284
#14 0x00007fffe92d3f00 in CallJSNative(JSContext*, bool (*)(JSContext*, unsigned int, JS::Value*), js::CallReason, JS::CallArgs const&)

Comment 6

[Axel Hecht [:Pike]]

I filed a bug to figure out what we should actually do there for IndexedDB.

Can we really restrict the problem at hand to accept-language, or is that just the problem we have found right now?

Related question, how do we handle the content process size questions when we have Fluent for the current content use-cases?

Comment 7

[Jan Horak [:jhorak]]

Hm, also the getStringBundle request for intl.accept_languages is called from there: RuntimeService. And then even in mozilla::net::nsHttpHandler::SetAcceptLanguages - originating there:

0 URLFetcher(url = ""http://detectportal.firefox.com/success.txt"", timeout = "5000") ["resource://gre/modules/CaptiveDetect.jsm":29:6]
this = [object Object]
1 _startDetection() ["resource://gre/modules/CaptiveDetect.jsm":363:21]
this = [object Object]

After both above calls the correct locale is loaded.
I'm not sure if we can deal with all of these calls after the correct intl.resource has been loaded.

Comment 8

[Jan Horak [:jhorak]]

Attachment: Bug 1603112 Do not include intl.properties in shared bundles to apply changes from installed langpacks; r=kmag

When the intl.properties is in shared string bundles (ie. in shared memory) the
newly loaded langpacks the intl.properties is not reloaded. This leads for
example to wrong intl.accept_languages setting - en-US, en for any installed
langpack.

Comment 9

[Kris Maglione [:kmag]]

I'm accepting this patch for now, but please file a follow-up bug to fix the callers which load this string bundle before language packs are ready. Aside from being inefficient in the cases where we wind up having to reload it, that's likely to wind up causing other problems on its own.

Comment 10

[Kris Maglione [:kmag]]

Also, it's really unfortunate that we have so many separate places that manually read and parse "intl.accept_languages" rather than just one place that's responsible for reading, parsing, and and providing a parsed list...

Comment 11

[Jan Horak [:jhorak]]

Filled follow up bug 1604732.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/912f361b1c11
Do not include intl.properties in shared bundles to apply changes from installed langpacks; r=kmag

Comment 13

[Oana Pop-Rus]

https://hg.mozilla.org/mozilla-central/rev/912f361b1c11