1140430 - Single locale builds aren't aware of their locale, consequently using incorrect capitalization schemes (and text directions, and…)

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: Locale switching and selection, defect)

Description

[Emin Mastizada]

Attachment: aurora-38.0a2.png

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:38.0) Gecko/20100101 Firefox/38.0
Build ID: 20150228155808

Steps to reproduce:

I use daily builds of Aurora l10n version (Azerbaijani):
http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/latest-mozilla-aurora-android-api-11-l10n/fennec-37.0a2.az.android-arm.apk
After update few days ago I got issue with Font.

Nexus 5 Android 5.0.1
Firefox for Android 38.0a2


Actual results:

The letter "İ" and "I" is displayed same


Expected results:

The letter "İ" is used in Azerbaijani and in Turkish, and "İ" is the same as letter i, but "I" is like Russian letter "ы" like "y" at some places of word in English. So these two letters are different. It was ok in Aurora few days ago.

Comment 1

[Richard Newman [:rnewman]]

In that screenshot, are you referring to the uppercasing of "Əlfəcinlər"?

I don't see any changes that would explain this in the `az` l10n repo, so that rules out that possibility.

Could you install an older Aurora build and see if İ is displayed correctly?

Comment 2

[Emin Mastizada]

(In reply to Richard Newman [:rnewman] from comment #1)
> In that screenshot, are you referring to the uppercasing of "Əlfəcinlər"?
> 
> I don't see any changes that would explain this in the `az` l10n repo, so
> that rules out that possibility.
> 
> Could you install an older Aurora build and see if İ is displayed correctly?

As all letters are upper case, It should be ƏLFƏCİNLƏR with "İ", and another one should be QABAQCIL SAYTLAR with "I".
Tried to downgrade, but unsuccessful, translations and font is updated in old version. Checked in friend's phone using Android 4.4.4, it uses system font where Turkish and Azerbaijani letters are absent.

Comment 3

[Jonathan Kew [:jfkthame]]

AFAICS, the string "Əlfəcinlər" (Bookmarks) is provided (correctly) in mixed-case in the az localization, but is rendered with uppercasing applied. My guess is that the correct locale is not being specified for the toUpperCase() operation, and therefore "i" is uppercased as the default "I" instead of the "İ" that would be correct for Azerbaijani (or Turkish).

There are a couple of places in the Java code of mobile/android/base/home where .toUpperCase() is applied to title strings. No locale is passed, which means the operation will use "the default locale", whatever that happens to be; I don't know if the Firefox/Android code takes any steps to set this appropriately, or if it'll use a system default. Perhaps it should explicitly pass its locale to the .toUpperCase() method, to ensure the correct behavior?

Comment 4

[Richard Newman [:rnewman]]

Assuming I'm understanding your comment:

When you downgraded to a known-good version, you saw the same issue.


What locale is your phone set to?

What locale is Fennec itself set to? (Inside 'Settings').


My suspicion is that one of two things is happening:


1. Your phone installed an Android update, and it affected either the locale-aware uppercasing, or the font support on the device.

2. Your locale is set to non-Azerbaijani somewhere that matters, and so you're getting the right string but uppercased using non-Azerbaijani localization rules, like this in JS:

$ "Əlfəcinlər".toUpperCase()
> "ƏLFƏCINLƏR"

Comment 5

[Richard Newman [:rnewman]]

(In reply to Jonathan Kew (:jfkthame) from comment #3)
> No locale is passed, which
> means the operation will use "the default locale", whatever that happens to
> be; I don't know if the Firefox/Android code takes any steps to set this
> appropriately, or if it'll use a system default. Perhaps it should
> explicitly pass its locale to the .toUpperCase() method, to ensure the
> correct behavior?

The current default locale is supposed to be correct from the environment, or overwritten by the locale switcher — that we have the right string implies that we have the right default locale.

An explicit locale is only used when doing unusual stuff, like showing the list of locales in the locale picker itself.

Perhaps more importantly, nothing has changed in this regard for months, as far as I know. That leads to my two hypotheses.

Comment 6

[Jonathan Kew [:jfkthame]]

(In reply to Richard Newman [:rnewman] from comment #5)
> The current default locale is supposed to be correct from the environment,

Where "from the environment" means "the device's current Android system locale"? 

> or overwritten by the locale switcher — that we have the right string
> implies that we have the right default locale.

I just tried installing a localized Turkish build of Nightly on my (en-US) device, and am seeing this problem. This build shows Turkish strings -- as expected; remember it's the localized Turkish build, not the multi-locale release -- but it uses en-US uppercasing in the home page display. I presume that's because my *system* locale remains en-US; but I would have expected the localized build to use its own locale-specific uppercasing to display its strings, not my system locale.

I also notice that if I try to change language within Nightly, via menu Ayarlar [Settings] / Dil [Language], it offers me two choices: Sistem varsayılanı [System default] and English (United States). However, whichever of these I choose, it continues to display the Turkish strings. And it continues to uppercase them incorrectly.

In contrast, if I install the current multilocale Aurora build, and change its language to Turkish, I do get the correct uppercasing behavior (on my en-US android device). So it appears the problem here may be specific to localized (as opposed to multilocale) builds.

Comment 7

[Emin Mastizada]

(In reply to Jonathan Kew (:jfkthame) from comment #6)
> (In reply to Richard Newman [:rnewman] from comment #5)
> > The current default locale is supposed to be correct from the environment,
> 
> Where "from the environment" means "the device's current Android system
> locale"? 
> 
> > or overwritten by the locale switcher — that we have the right string
> > implies that we have the right default locale.
> 
> I just tried installing a localized Turkish build of Nightly on my (en-US)
> device, and am seeing this problem. This build shows Turkish strings -- as
> expected; remember it's the localized Turkish build, not the multi-locale
> release -- but it uses en-US uppercasing in the home page display. I presume
> that's because my *system* locale remains en-US; but I would have expected
> the localized build to use its own locale-specific uppercasing to display
> its strings, not my system locale.
> 
> I also notice that if I try to change language within Nightly, via menu
> Ayarlar [Settings] / Dil [Language], it offers me two choices: Sistem
> varsayılanı [System default] and English (United States). However, whichever
> of these I choose, it continues to display the Turkish strings. And it
> continues to uppercase them incorrectly.
> 
> In contrast, if I install the current multilocale Aurora build, and change
> its language to Turkish, I do get the correct uppercasing behavior (on my
> en-US android device). So it appears the problem here may be specific to
> localized (as opposed to multilocale) builds.

Yeah, just switched to Turkish in System, and Azerbaijani worked good. I wonder if Firefox will use system or locale specific in release. It works good in telegram app (we are localizing this app too), they do not use toUpperCase() function, they just enter strings in uppercase in translation files. So, I think if we will use uppercase in translation and ignore warnings, it will be solved. Or, Firefox should pass locale in toUpperCase() functions

Comment 8

[Richard Newman [:rnewman]]

Sorry, I missed that this bug report was about a single-locale repack.


(In reply to Jonathan Kew (:jfkthame) from comment #6)

> Where "from the environment" means "the device's current Android system
> locale"? 

Unless overridden by the in-app locale switcher, yes.


> I just tried installing a localized Turkish build of Nightly on my (en-US)
> device, and am seeing this problem. This build shows Turkish strings -- as
> expected; remember it's the localized Turkish build, not the multi-locale
> release -- but it uses en-US uppercasing in the home page display. I presume
> that's because my *system* locale remains en-US; but I would have expected
> the localized build to use its own locale-specific uppercasing to display
> its strings, not my system locale.

Single-locale builds are an odd duck; they're pretty much just an en-US build with different strings swapped in.

There are open bugs around making them work correctly (e.g., Bug 1031974). This will be particularly important if/when we ship RTL single-locale builds: they won't know that they're RTL!

That work (or even thorough testing) hasn't been prioritized, because statistically speaking nobody uses single-locale builds on Android.


> I also notice that if I try to change language within Nightly, via menu
> Ayarlar [Settings] / Dil [Language], it offers me two choices: Sistem
> varsayılanı [System default] and English (United States). However, whichever
> of these I choose, it continues to display the Turkish strings. And it
> continues to uppercase them incorrectly.

Like I said, single-locale builds are just en-US builds with the strings swapped out. You can't switch it to any other language. 

The locale picker should really be hidden for single-locale builds, but nobody has done that work yet.


(In reply to Emin Mastizada from comment #7)

> Yeah, just switched to Turkish in System, and Azerbaijani worked good. I
> wonder if Firefox will use system or locale specific in release.

In a multilocale build like release, it uses either the system locale or the in-app selected locale. That is: if you were to install a release build from Play, and switch it to Azerbaijani (either in the OS or inside Firefox), we would use Azerbaijani capitalization rules.

(Android's Azerbaijani capitalization rules might be wrong, but that's another matter. If so, please file another bug!)


> Or, Firefox should pass locale in toUpperCase()
> functions

I think the bug here is this: single-locale builds should be aware of their locale. That way they'll work just as if you had a multi-locale build installed, with the correct locale selected -- a tested code path. No other code changes are required.

So, let's morph this bug to "do the right thing in single-locale builds". If you also see problems in release, please file another bug and CC me!

Comment 9

[Emin Mastizada]

Works good in last version of Android (7) and Firefox for Android (54)