1517842 - Date.toLocaleDateString ignore regional settings with en-US Firefox on en-US Windows

Status: RESOLVED INVALID

(Core :: JavaScript: Internationalization API, defect)

Description

[Gert van den Berg]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:64.0) Gecko/20100101 Firefox/64.0

Steps to reproduce:

Set Windows 10 "Preferred Languages" to only "English (South Africa)"
Add "English (South Africa)" as first preferred language in Firefox.

Run console.log(new Date().toLocaleString()); // in JS console

Other notes: "Windows Display Language" is "English (United States)" (the only option).


Actual results:

1/4/2019, 9:54:50 PM

Date returned in US format.


Expected results:

2019-01-04 21:54:50

Date should be returned in en_ZA format. (Either for a standard source or based on Windows regional )

(Slashes instead of dashes are acceptable as well, time format is not critical)

Comment 1

[Gert van den Berg]

http://www.localeplanet.com/support/browser.html gives the same result and is quick to check.

Full Javascript result from there:
navigator.browserLanguage: undefined
navigator.language: en-ZA
navigator.userAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:64.0) Gecko/20100101 Firefox/64.0
date: Fri Jan 04 2019 21:50:09 GMT+0200 (South Africa Standard Time)
date.toLocaleDateString(): 1/4/2019
date.toLocaleTimeString(): 9:50:09 PM
date.toLocaleString(): 1/4/2019, 9:50:09 PM
date.toUTCString(): Fri, 04 Jan 2019 19:50:09 GMT
date.getTimezoneOffset(): -120

Full language preferences: (Header is easiest source)
Accept-Language: en-ZA,en;q=0.8,en-US;q=0.5,af;q=0.3

Comment 2

[Francesco Lodolo [:flod]]

(In reply to Gert van den Berg from comment #0)
> Add "English (South Africa)" as first preferred language in Firefox.

How did you do this? 

Can you go to about:support and copy and paste the content of the last table (internationalization data).

Comment 3

[Gert van den Berg]

Internationalization & Localization
Application Settings
Requested Locales 	["en-US"]
Available Locales 	["en-US"]
App Locales 	["en-US"]
Regional Preferences 	["en-ZA"]
Default Locale 	"en-US"
Operating System
System Locales 	["en-US"]
Regional Preferences 	["en-ZA"]

It was added by going to Options > typing "lang" in the search, using the "Choose..." option that comes up. (I would prefer that that adds the OS's preferred language automatically, I saw bug #265400 for that)

(I'm assuming this can be changed independent of the UI language - I sometimes prefer Afrikaans content, but the Browser UI in Afrikaans is unusable, since everything else on the PC is in English and figuring out what a menu option different from any other application on the PC mean, means completely relearning the UI)

(I think the OS regional settings makes most sense to use for this, it allows for finer-grained control - like fixing the date separator from a slash (which is often used for en-ZA, including by Windows) to the correct ISO8601 dash instead (but any date format other than the horribly confusing mm/dd/yyyy is acceptable))

(If I want to install the language pack it wants to translate the UI as well, which I don't want (it tend to result in spelling inconsistent with the rest of the OS UI, which is not available in en-ZA))

(My windows regional setting was likely correct when Firefox was installed, but it probably was downloaded from Chrome which somehow ended up with en-US at the top of Accept-Languages) (the page that getfirefox.com redirects to doesn't show which language will be downloaded)

Chaing one of the lines on https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/toLocaleDateString to
console.log(event.toLocaleDateString('en-ZA'));
returns an acceptable (order is right, separator is incorrect, but acceptable) format, so does "console.log(new Date().toLocaleString('en-ZA'));", so it seems to be a case of the Firefox UI language being used instead of the language set in the OS regional preferences or the option set in under the language preferences. (the OS setting makes most sense - it is the way to change the date format in every other application)

Comment 4

[Francesco Lodolo [:flod]]

> It was added by going to Options > typing "lang" in the search, using the
> "Choose..." option that comes up. (I would prefer that that adds the OS's
> preferred language automatically, I saw bug #265400 for that)

What you change in preferences right now is the Accept Language header, i.e. in which languages you want web content served to you. It doesn't have anything to do with the UI language (en-US in your case).

https://firefox-source-docs.mozilla.org/intl/locale.html#regional-preferences

> Gecko will only look into OS Preferences if the language portion of the locale of the OS and Firefox match

In your case, given en-US and en-ZA I would expect Firefox to look into regional preferences, even without creating a pref for it. If you create intl.regional_prefs.use_os_locales in about:config and set it to true, does it uses your regional settings?

> console.log(event.toLocaleDateString('en-ZA'));
> returns an acceptable (order is right, separator is incorrect, but acceptable) format, so does "console.log(new Date().toLocaleString('en-ZA'));"

en-ZA seems to use `GGGGG y/MM/dd` in CLDR for short date, so `2018/01/04` is the expected output, and should be consistent with other browsers
https://www.unicode.org/cldr/charts/34/by_type/date_&_time.generic.html#679dff5b9b612aa0

Comment 5

[Gert van den Berg]

I also installed the en-ZA language pack to test and restarted the browser. (I don't see another way to activate the pack)

Afterwards, Javscript still returns the en-US date format by default.

The support info is now:
Internationalization & Localization
Application Settings
Requested Locales 	["en-US"]
Available Locales 	["en-ZA","en-US"]
App Locales 	["en-US","en-ZA"]
Regional Preferences 	["en-ZA"]
Default Locale 	"en-US"
Operating System
System Locales 	["en-US"]
Regional Preferences 	["en-ZA"]

Comment 6

[Gert van den Berg]

intl.regional_prefs.use_os_locales is set to true (After I installed the language pack, I set intl.locale.requested to en-ZA temporarily to check (which is way too hard for a average user), it worked while that was set, I don't know if that is what changed intl.regional_prefs.use_os_locales)

Current state of support page: (Also installed Afrikaans language pack when )
Internationalization & Localization
Application Settings
Requested Locales 	["en-US"]
Available Locales 	["af","en-ZA","en-US"]
App Locales 	["en-US","en-ZA"]
Regional Preferences 	["en-ZA"]
Default Locale 	"en-US"
Operating System
System Locales 	["en-US"]
Regional Preferences 	["en-ZA"]

With intl.locale.requested unset, it gives me en-US dates, even with intl.locale.requested set to true. (If intl.locale.requested is set to en-ZA it works, but that should not be necessaty)

Other settings that looks like it might be relevant:
intl.multilingual.enabled;false

> en-ZA seems to use `GGGGG y/MM/dd` in CLDR for short date, so `2018/01/04` is the expected output, and should be consistent with other browsers

Everything defaults to that order for date format - that is acceptable. I think officially, we are supposed to use ISO 8601 (I can't find a nice source for short date formats anywhere though), but changing that introduces more inconsistencies with OS defaults and other things, which might be more trouble than it is worth. (dd/mm/yyyy is also commonly used, which is why the US format ends up extremely confusing) (as long as the month is in the middle, it will be understood) (And getting CLDR updated is certainly a better place for that)

Comment 7

[Francesco Lodolo [:flod]]

@zibi
Any clue why the regional settings are ignored? Am I misunderstanding our docs?

Comment 8

[Francesco Lodolo [:flod]]

And I realized this could be by design: you're using en-US build on en-US, regional settings are ignored. Still, I would expect `intl.regional_prefs.use_os_locales` set to true to work in that scenario.

Installing the language pack for en-ZA doesn't do anything, since your OS is en-US. You need to enable it manually, which means having a `intl.locale.requested` set to `en-ZA` (that's a list of locales, not a boolean).

Comment 9

[Gert van den Berg]

Quick note: Getfirefox.com (also going to mozilla.org > Firefox > Download) always sends me to the en-US download page without indicating the English variant that will be downloaded anywhere. (Even if the browser going there indicates a preference for en-ZA) This likely mean that just about no English users actually install the variant that they want.

(To me, it makes most sense to follow the OS's regional preferences for regional preferences in the browser - at least on a clean install with no settings set) (It would also be nice that if a preferred language is changed, that the browser asks whether the relevant dictionaries or language packs should be installed) (The language packs seems a bit useless currently - a random user isn't going to figure out to add a setting in about:config change it)

(Android Firefox seems to deal with it properly (but it might actually have a en-ZA UI language), it would be interesting to see if the equivalent of the Windows region settings - setting LANG (without installing a non-default language pack) on *nix works (Technically probably LANG=en_ZA.UTF-8 LANGUAGE=en_US.UTF8 LC_MESSAGES=en_US.UTF8))

Comment 10

[Francesco Lodolo [:flod]]

(In reply to Gert van den Berg from comment #9)
> Quick note: Getfirefox.com (also going to mozilla.org > Firefox > Download)
> always sends me to the en-US download page without indicating the English
> variant that will be downloaded anywhere. (Even if the browser going there
> indicates a preference for en-ZA) This likely mean that just about no
> English users actually install the variant that they want.

Because en-ZA is basically unmaintained, so you fall back to en-US for mozilla.org page. With bug 1440488 on top of that.

> (The language packs seems a bit useless currently
> - a random user isn't going to figure out to add a setting in about:config
> change it)

We're adding UI to switch the preference (bug 1425941), and do more.

> (Android Firefox seems to deal with it properly (but it might actually have
> a en-ZA UI language)

It does.

Comment 11

[Gert van den Berg]

Interesting, in Firefox ESR on Debian, getfirefox.com takes me to the en-GB site. (accept-languages seems to have had that added automatically (possibly from the en_za l10n package, it is gone when that is removed), which might be a factor)

On Debian, the packaged Firefox ESR (60) works as expected (see note later), a manual download of the en-US stable (64) one gives similar results as on Windows (US date formats). (Debian seems to have installed their firefox-esr-l10n-en-za package). Without the localization package, Debian Firefox ESR also shows en_US dates on a en_ZA locale (so at least JS behaviour seems consistent between Debian and Windows)

It seems like this behaviour might be by design (I think that ignoring the OS regional setting by default (always actually) when the FF language mismatch is not ideal) (but the download page is what got me here in the first place) (the only language settings (the accept-language settings) in Firefox might make sense to use / extend for the settings (with smarter defaults)) (the previously linked Locale management documentation seems to indicate that it should follow the OS setting if both match en-* though...)

(I think multi-country languages, like English, are more complicated that other languages - the user doesn't notice that the UI (and the download page) is in a different locale, since the language is the same as the intended language)

> Because en-ZA is basically unmaintained, so you fall back to en-US for mozilla.org page. With bug 1440488 on top of that.

It is an interesting one. Just returning the text as on the en-GB site for the en-ZA site should be fine... (the differences shouldn't be apparent in that context) I'm assuming that is not that simple to implement though..

(Ideally for ZA users easily getting to the Afrikaans, en_ZA and (isi)Xhosa (and other local languages when they are added) and some common international builds would be nice (most users probably have en_US Windows, with either en_ZA or their primary language's regional settings))

Comment 12

[Zibi Braniecki [:zbraniecki][:gandalf]]

The reason we do that is to avoid leaking your locale to the Web, which would be a fingerprint vector.

For that reason, we do not communicate to the Web that you chose to use different regional preferences locale than your browser's locale.

In result, all dates/times formatted within the browser UI will respect the en-ZA option, but when we communicate to a web page we'll use the app-locales, rather than regional-preferences-locales.

If someone wants to explore the depth of the fingerprint vector and convince the privacy team that it's ok to add one bit of leakage, then I'm happy to change that :)

Comment 13

[Alex]

Doesn't navigator.languages already leak my chosen locale (And fallbacks) to the web? As it stands now sites have to explicitly ask the browser for my locale to format any dates or times, before they could just ignore it.

And, isn't that what things like privacy.resistFingerprinting are for?

Comment 14

[Gert van den Berg]

If it is blocked for fingerprinting reasons, can the default be anything other than the en_US format please?

That date format is confusing to everyone else and seems to only used by around 6% of the world population.

Comment 15

[Zibi Braniecki [:zbraniecki][:gandalf]]

If it is blocked for fingerprinting reasons, can the default be anything other than the en_US format please?

The default is matching your browser locale. If you download Firefox en-GB, it'll be en-GB. If you use en-US, it's en-US.

Comment 16

[Jason Orendorff [:jorendorff]]

Zibi, is everything here working the best we can make it? It seems like the main complaints are UI complaints that were already being addressed in other bugs, and the remaining complaint about the behavior exposed to web sites is (for better or worse) by design.

I'm closing it -- but please reopen if there is work to do here.

Comment 17

[Jason Orendorff [:jorendorff]]

-> RESO INVALID, which just means it's not really a bug

Comment 18

[Zibi Braniecki [:zbraniecki][:gandalf]]

Based on Comment 6 the only remaining bit needed is ability to switch requested locales to en-ZA. This is UI request and should be handled in FIrefox Preferences bug if it's not already fixed.