1201232 - Locale string format is not working when Persian locale is set on the machine (Windows 8.1)

Status: VERIFIED FIXED

(Core :: JavaScript: Internationalization API, defect)

Description

[ebrahim]

Attachment: A.png

User Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:41.0) Gecko/20100101 Firefox/41.0
Build ID: 20150831172306

Steps to reproduce:

1. Just set Persian as locale on Region setting, on a Windows 8.1 machine.
2. Execute "(123).toLocaleString()" or (123).toLocaleString('en') on a console.


Actual results:

"ناعدد" and "NaN" will be raised which both are equal and indicates something is broken.


Expected results:

۱۲۳ or 123 should be raised instead.

Comment 1

[Axel Hecht [:Pike]]

We need this to be tested on a variety of windows versions.

My VM with Windows 7 doesn't reproduce this, or at least I can't.

Comment 2

[André Bargull [:anba]]

This issue is reproducible on Win10, tested with a nightly from 2015-09-15. I've also tried Win7, but got the same result as #1, that means not reproducible with that Windows version. And it seems to be limited to Persian, when I tested other locales, the bug wasn't reproducible.

This could be an ICU bug similar to [1, 2], it may make sense to build a standalone program to check unum_formatDouble works for Persian on Win8.1 and/or Win10.

[1] http://bugs.icu-project.org/trac/ticket/8214
[2] http://bugs.icu-project.org/trac/ticket/8561

Comment 4

[Liz Henry (:lizzard) (relman/hg->git project)]

Marking all current versions as affected. there are steps to reproduce this in some of the duplicate bugs, including at https://bugzilla.mozilla.org/show_bug.cgi?id=1221286#c6 where it means that users about to download an addon or other file can't tell the file size. 

This doesn't seem to be a recent regression, but it also seems like a fairly annoying user facing bug. Can anyone help find someone to work on the issue?

Comment 5

[Liz Henry (:lizzard) (relman/hg->git project)]

Marking wontfix for 42 and 43.  I'm requesting tracking for 45 since it would be good to have help finding someone to take this and work on it.

Comment 6

[Sylvestre Ledru [:Sylvestre]]

Ehsan, if I remember correctly, you worked on a similar bug about the download information.
Do you know who could help here? Thanks

Comment 7

[(no longer active)]

The download manager bug was bug 1009795 but that was essentially us working around this issue.  This seems like a dupe of bug 999003 to me, but Waldo is the expert here.

Comment 8

[Sylvestre Ledru [:Sylvestre]]

Even if I would like this to be fixed, we shipped enough releases with this bug to not block on it.

Comment 9

[André Bargull [:anba]]

The issue is no longer reproducible in Firefox 45 (or later), because it contains a newer ICU release. So it was an ICU internal issue after all. Resolving as works-for-me.

Comment 10

[Francesco Lodolo [:flod]]

Not sure if we should reopen this one or file a new bug, but I'm getting reports from the Persian localizers that this is still broken in 57 and 58.

Comment 11

[Francesco Lodolo [:flod]]

Attachment: file_1573.jpg

Comment 12

[Amir Farsi]

Attachment: Firefox 58 Nightly Download showing NaN Mb of NaN Mb instead of x mb of y mb

Comment 13

[Amir Farsi]

Attachment: Firefox 58 Nightly Download showing NaN Mb of NaN Mb abd NaN Mb instead of x mb of y mb

It showing ناعدد instead of NaN because we translated NaN as ناعدد in Pontoon.
OS: Windows 10 Enterprise.
In Windows 10, when prefered language in it's language settings is Persian or English, Firefox status is like it.

Comment 14

[Amir Farsi]

As you can see in screenshots, this problem remained from Firefox 42 to Firefox 58. This problem is in Download Window and About Window(Update).

Comment 15

[Amir Farsi]

Attachment: MyRegionalSettings screenshot

There isn't any different between regional settings. When Persian language prefered, Firefox 58 working like this bug.

Comment 16

[Amir Farsi]

Attachment: My Windows Number Formatting and Advanced language settings

Windows Edition: Windows 10 Enterprise(64 Bit)
Windows Version: 1703
OS Build 15063.483

Comment 17

[Francesco Lodolo [:flod]]

Zibi, any clue of what the problem could be, or suggestions to narrow down the cause?

Comment 18

[Zibi Braniecki [:zbraniecki][:gandalf]]

Amir, can you verify if the steps to reproduce from comment 0 are still reproduceable for you?

Comment 19

[Amir Farsi]

Attachment: When use native digits in Windows Regional Settings setted on Context/National vs It's on Never

I think this picture/diagram can describe the probelem in detail.
When windows regional settings, format setted on Persian(Iran) and Use native dgits setted on national/native and windows wants to use Persian digits in all of Applications, this Firefox shows ناعدد or NaN
But if in Regional settings we set Format as English or set format as Persian(Iran), but sett Use native digits on Never and Windows don't want to use Persian digits in any Application, then Firefox also behaviouring correctly and showing English digits instead of ناعدد or NaN

Result: Firefox can't display Persian digits and instead of them showing ناعدد which is translation of NaN! This problem happenning around all of places in firefox.(From update to about:downloads)

Note: Google Chrome can show Persian digits correctly in it's download section when it's UI is Persian. Both Firefox and Chrome are working on same windows 10 with same settings on one PC.

Note2: Older versions of Firefox could also show Persian digits. But from a version this problem happened to today. It was from time that Windows 8 released and i saw it in Version 42 of FF first time. I don't when FF started to have this bug? From version 30 or 35 or...

Comment 20

[Amir Farsi]

Attachment: ChromePersianDownload showing Persian Digits correctly

Comment 21

[Zibi Braniecki [:zbraniecki][:gandalf]]

Amir: can you please answer my question from Comment 18?

Comment 22

[Francesco Lodolo [:flod]]

To clarify, open Tools->Web Developer->Web console, and type

(123).toLocaleString()

press enter, then

(123).toLocaleString('en')

press enter, and copy and past the output here.

Comment 23

[Amir Farsi]

Attachment: FfConsole58ToLocaleString Working Correctly!

In Firefox 58 nightly, i executed that things that script command in consolse and it working correctly! I attached screenshot of console.

Comment 24

[Zibi Braniecki [:zbraniecki][:gandalf]]

Ok. So it seems that our JS engine is producing the right strings, but our UI is somehow messing them up.

I'm not sure how to approach it, because I don't know of any step in between the `Intl.NumberFormat` constructing a formatted number, and us displaying it in the UI, that would justify the NaN switch.

:flod - can you look for an easy STR for me on Windows 10? I'll try to debug it then.

Comment 25

[Francesco Lodolo [:flod]]

Attachment: nan_persian_settings.png

Simply set your "Numbers and date" setting to Persian.

These are screenshots using the Italian build with those settings.

Comment 26

[Amir Farsi]

Attachment: PersianRegionalFormat.jpg

Hello.
My numbers and settings is on Persian (Iran) all of the times, and as result, this problem(NaN or ناعدد) happening.
More info: in attached screenshot.

As you saw, even in Italian version, Firefox only can detect and show English digits(When format in Region is on Italian) can not show italian digits(However, itlian digits are like English digits but with different unicode code)

Comment 27

[Zibi Braniecki [:zbraniecki][:gandalf]]

I was able to reproduce it. Taking.

Comment 28

[Zibi Braniecki [:zbraniecki][:gandalf]]

I was able to isolate the problem to our JS Engine and reported bug 1422415. It may or may not be trivial to fix there.

The workaround for now would be to round the number to 3 decimals in DownloadUtils to avoid hitting the bug in SM. I'll see first if we can fix it the right way, and may come back with a workaround if not.

Comment 29

[Zibi Braniecki [:zbraniecki][:gandalf]]

Andre took bug 1422415 and has a first version of a patch ready, so I think it makes sense to get the right fix rather than a workaround.

I also filed bug 1422658 which I'll work on to further improve how to select the default locale for intl operations in DownloadUtils (to use RegionalPrefsLocales rather than AppLocales).

Thank you all for reporting and providing feedback and apologies for how long it took me to debug it. From here I expect that we should make it work correctly within Firefox 59 timeframe.

Comment 30

[Amir Farsi]

Attachment: Uploading in Firefox Send showing NaN too

I found new problem about NaN. I wanted to upload a file on Firefox Send. I saw it showing ناعدد = NaN also!

Comment 31

[Zibi Braniecki [:zbraniecki][:gandalf]]

Attachment: Bug 1201232 - Use Services.intl for Intl.* APIs in Gecko.

Review commit: https://reviewboard.mozilla.org/r/215420/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/215420/

Comment 32

[Zibi Braniecki [:zbraniecki][:gandalf]]

Comment on attachment 8945179 [details]
Bug 1201232 - Use Services.intl for Intl.* APIs in Gecko.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/215420/diff/1-2/

Comment 33

[Zibi Braniecki [:zbraniecki][:gandalf]]

Hi :paolo - this is a minor patch that makes us switch to use mozIntl.* rather than Intl.* in chrome code. Since you were the last reviewer of code in DownloadUtils, I decided to try asking for your review :)

It's a minor change that should align us better with our selection of locales (Services.intl will use user selection of regional preference locales).
I also removed the western arabic numeral system limitation to allow persian locales to use eastern arabic numerals correctly.

Comment 34

[:Paolo Amadini]

Comment on attachment 8945179 [details]
Bug 1201232 - Use Services.intl for Intl.* APIs in Gecko.

https://reviewboard.mozilla.org/r/215420/#review222306

rs=me for the code, with the nit below. As for the contents of the change, I can't tell how these changes affect localization in the way you described, so I'd recommend asking a peer of the internationalization library for a review.

Also, please include part of the description in comment 33 in the commit message.

::: toolkit/mozapps/downloads/DownloadUtils.jsm:54
(Diff revision 2)
>  
>  const MS_PER_DAY = 24 * 60 * 60 * 1000;
>  
>  var localeNumberFormatCache = new Map();
>  function getLocaleNumberFormat(fractionDigits) {
> -  // Backward compatibility: don't use localized digits
> +  let key = fractionDigits;

Inline fractionDigits.

Comment 35

[Axel Hecht [:Pike]]

Comment on attachment 8945179 [details]
Bug 1201232 - Use Services.intl for Intl.* APIs in Gecko.

https://reviewboard.mozilla.org/r/215420/#review222308

Both of these changes seem to be carrying forward bug 1344543, removing hacks for Fennec's lack of Intl at the time. We hard-coded latin numbers for the gDecimalSeparator hack that we removed in that bug, but didn't remove the hack.

Also, seems that one conditional code-path slipped through the cracks.

+1 from me to use mozIntl to pick up OS prefs for chrome, where that doesn't fingerprint people.

-1 from me for the world in which I didn't find docs on this.

::: toolkit/components/narrate/NarrateControls.jsm:168
(Diff revision 2)
>      return this._languagePromise.then(language => {
>        this.voiceSelect.clear();
>        let win = this._win;
>        let voicePrefs = this._getVoicePref();
>        let selectedVoice = voicePrefs[language || "default"];
>        let comparer = win.Intl ?

The conditional sounds wrong here now. This seemed to be a "do we support Intl", probably for Android.

Comment 36

[Zibi Braniecki [:zbraniecki][:gandalf]]

Comment on attachment 8945179 [details]
Bug 1201232 - Use Services.intl for Intl.* APIs in Gecko.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/215420/diff/2-3/

Comment 37

[Zibi Braniecki [:zbraniecki][:gandalf]]

Thank you! Updated the patch and requesting r? from pike.

Comment 38

[Code Review Bot [:reviewbot]]

Comment on attachment 8945179 [details]
Bug 1201232 - Use Services.intl for Intl.* APIs in Gecko.

https://reviewboard.mozilla.org/r/215420/#review222546


Static analysis found 1 defect in this patch.
 - 1 defect found by mozlint

You can run this analysis locally with:
 - `./mach lint check path/to/file` (Python/Javascript/wpt)


If you see a problem in this automated review, please report it here: http://bit.ly/2y9N9Vx


::: toolkit/components/narrate/NarrateControls.jsm:168
(Diff revision 3)
>      return this._languagePromise.then(language => {
>        this.voiceSelect.clear();
>        let win = this._win;
>        let voicePrefs = this._getVoicePref();
>        let selectedVoice = voicePrefs[language || "default"];
> -      let comparer = win.Intl ?
> +      let comparer = new Services.intl.Collator()).compare;

Error: Parsing error: Unexpected token ) [eslint: None]

Comment 39

[Zibi Braniecki [:zbraniecki][:gandalf]]

Comment on attachment 8945179 [details]
Bug 1201232 - Use Services.intl for Intl.* APIs in Gecko.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/215420/diff/3-4/

Comment 40

[Axel Hecht [:Pike]]

Comment on attachment 8945179 [details]
Bug 1201232 - Use Services.intl for Intl.* APIs in Gecko.

https://reviewboard.mozilla.org/r/215420/#review222552

Comment 41

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3a9687c8e978
Use Services.intl for Intl.* APIs in Gecko. r=Paolo,Pike

Comment 42

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/3a9687c8e978

Comment 43

[Zibi Braniecki [:zbraniecki][:gandalf]]

Hi Amir, can you verify that the bug is fixed in the upcoming Nightly?

Comment 44

[Amir Farsi]

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #43)
> Hi Amir, can you verify that the bug is fixed in the upcoming Nightly?

Hi Zibi.
I updated my nightly build.
Now, Firefox shows Persian digits in download units correctly in all of places. There isn't any NaN.
It seems 98 % of this bug fixed.
The remained 2% of bug:
In some of places, Firefox shows Latin digits, not Persian digits.
For eaxmple: 
1-In download window, All of numbers(Downloaed Mb of Filesize Mb) are in Persian digits, but remained time(Houres, Minutes, Secounds) are in Latin yet.
2-In Firefox send, all of digits are in Persian, but uploaded percentage is in Latin yet.

Thanks.

Comment 45

[Francesco Lodolo [:flod]]

(In reply to Amir Farsi from comment #44)
> 2-In Firefox send, all of digits are in Persian, but uploaded percentage is
> in Latin yet.

For this, can you file an issue here with a screenshot?
https://github.com/mozilla/send/

Comment 46

[Amir Farsi]

Attachment: DownloadRemainedTimeEnglish.jpg

I added the screenshot of Downloading Window after 98% of this bug fixed.

Comment 47

[Amir Farsi]

Attachment: RemainedHouresAreLatin.jpg

The ramined houres, minutes and seconds that are in English yet. It's while i checked pontoon(persian translations) and it seems all these are traslated.

Comment 48

[Francesco Lodolo [:flod]]

(In reply to Amir Farsi from comment #44)
> 1-In download window, All of numbers(Downloaed Mb of Filesize Mb) are in
> Persian digits, but remained time(Houres, Minutes, Secounds) are in Latin
> yet.

This would be a different bug, I filed bug 1434844 for the Remaining time in Downloads.

Thanks for confirming that this one is fixed.

Comment 49

[Amir Farsi]

(In reply to Francesco Lodolo [:flod] from comment #45)
> (In reply to Amir Farsi from comment #44)
> > 2-In Firefox send, all of digits are in Persian, but uploaded percentage is
> > in Latin yet.
> 
> For this, can you file an issue here with a screenshot?
> https://github.com/mozilla/send/

Hi Franssesco.
It's here:
https://github.com/mozilla/send/issues/747

Comment 50

[Amir Farsi]

It seems that 2% of this bug will solve in mentoined bugs.
Anyway, I should thank all the participants in this bug(Specially, Zibi).
You solved one of old localization bugs in Mozilla.