346644 - nsICollation should be at least a bit more cross platform

Status: RESOLVED DUPLICATE of bug 943287

(Core :: Internationalization, defect)

Description

[timeless]

firefox and intl itself expect ()s to be part of the collation behavior. however on unix it isn't.

Steps to reproduce:
1. install firefox-linux/firefox-solaris
2. if you don't have a windows box, install wine, if you don't have wine, trust me.
3. install firefox-win
4. run firefox and select view>character encoding>auto-detect
5. run firefox-win and repeat

if you were running either of them w/ a locale other than en-US, then you're being way too silly, go back and try again using en-US.

expected results:
(Off) sorts before whatever alphabetical items are in the list.

actual results (firefox-unix):
(Off) sorts sorta into the middle (between Korean and Russian).

Comment 1

[Jungshik Shin]

On linux with glibc 2.4 :

$ cat sorttest
(Off)
English
Chinese
Korean
Russian
Japanese
French

$ LC_ALL=en_US sort sortest  # fr_FR, de_DE, en_US.ISO-8859-1 ....
Chinese
English
French
Japanese
Korean
(Off)
Russian

$ LC_ALL=en_US.UTF-8 sort ~/tmp/sortest 
(Off)
Chinese
English
French
Japanese
Korean
Russian

$ LC_ALL=C sort ~/tmp/sortest 
(Off)
Chinese
English
French
Japanese
Korean
Russian

$ LC_ALL=en_US.CP1252 sort ~/tmp/sortest 
(Off)
Chinese
English
French
Japanese
Korean
Russian


Perhaps you found a 'bug' in en_US, fr_FR, de_DE and other locales that use ISO-8859-1  (of glibc) ....

And, why do we rely on the OS for collation? well ....

Comment 2

[O. Atsushi (Torisugari)]

'O' is larger than 'K' and smaller than 'R', so it's kinda reasonable; "Korean" < "Off" < "Russian", in my opinion. It just skips evaluating '(' for some reason.

According to Unicode Collation Algorithm, "punctuation", such as spaces, double quotes and, probably, open bracket, is defined as Level-4, while "case" is Level-3. i.e. Even case-sensitive sort may ignore '('.

Comment 3

[O. Atsushi (Torisugari)]

Attachment: UCA v1

> And, why do we rely on the OS for collation? well ....

Is there any chance to replace ::strxfrm(...) and ::strcoll(...) with Mozilla's own implementation? At least it should be replaced with 16bit version of strxfrm(...), and preferably strxfrm_l(...).

Comment 4

[Makoto Kato [:m_kato]]

After landing bug 1214169, we can replace with ICU's nsICollation except to Android.

Comment 5

[Makoto Kato [:m_kato]]

I will replace with ICU version by bug 943287.