215033 - mozilla chooses incorrect font when i'm trying to save or load file

Status: RESOLVED EXPIRED

(Core :: XUL, defect)

Description

[Marat Afanasyev]

User-Agent:       Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.4) Gecko/20030804
Build Identifier: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.4) Gecko/20030804

mozilla chooses incorrect font when i'm trying to save or load file, i guess
there's something wrong in 'file-open' and 'file-save as' dialogs. I have
login-class russian, LANG=ru_RU.KOI8-R. setting LANG to C doesn't change anything

Reproducible: Always

Steps to Reproduce:
1. start mozilla
2. choose file-open
3. see japanese or chinese hierogliphs instead of english or russian letters

Actual Results:  
nothing special. I'd tried to build mozilla on a fresh installation. All was the
same

Expected Results:  
show a correct file-list

Comment 1

[Jungshik Shin]

If I understand correctly what's reported, this bug is invalid. 

I'm not sure what the last step means: 

> see japanese or chinese hierogliphs instead of english or 
> russian letters

Do you have files with Chinese or Japanese  names? What is your filesystem
charset(encoding)? No matter what it may be, they cannot come up correctly if
you run Mozilla under KOI8-R or C locale. Mozilla assumes that the filesystem
charset/encoding (that is used to represent file names and directory names) is
that of the current system locale on Unix. Note that unlike NTFS or HFS+ (Mac
OS) that always store file names in UTF-16, POSIX file system treats file names
just as sequences of _bytes_ (null-terminated) without embedded '/'. Therefore,
if you run Mozilla under ru_RU.KOI8-R, filenames (seuences of bytes) are
interpreted as KOI8-R.

If you want to make Chinese or Japanese file names  come up correctly under
Russian locale, you have to switch to UTF-8 locale (say, ru_RU.UTF-8) because
only Unicode can represent Chinese, Japanese and Russian at the same time. [1] 
This involves renaming all your files (in various legacy encodings/charset) to
have UTF-8 names. It'd not present much trouble if FreeBSD's locale support and
I18N were much better now than a couple of years ago. Back then, FreeBSD didn't
have nl_langinfo, iconv(3) and some critical I18N APIs (of SUS/POSIX) 

[1] Well, _to some extent_, old CJK encodings can represent Russian and Greek,
but you'd not want to switch from one legacy encoding(KOI8-R) to another(say,
EUC-JP). Besides, all Cyrillic letters and Greek letters would be rendered in
'full-width' (or 'double width') cells, which is really annoying.

Comment 2

[Marat Afanasyev]

Attachment: open file dialog

as you can see, mozilla treats file-list as dbcs, which is incorrect.

Comment 3

[Jungshik Shin]

Well, you must have other LC_* variables set to something other than C or
ru_RU.KOI8-R. What's the result of the following?

  % env | egrep '(LC_|LANG|LING)'

Note that LANG has the lowest priority among locale related env. variables. If
FreeBSD now has a working nl_langinfo(), it'd refer to LC_ALL, LC_CTYPE and LANG
(in that order) to determine the codeset of the locale. 

Comment 4

[Marat Afanasyev]

marat@lexsus# env | egrep '(LC_|LANG|LING)'
LANG=ru_RU.KOI8-R

Comment 5

[Marat Afanasyev]

> If I understand correctly what's reported, this bug is invalid.
>
> I'm not sure what the last step means:
>
>> see japanese or chinese hierogliphs instead of english or
>> russian letters
>
> Do you have files with Chinese or Japanese  names? What is your
filesystem
> charset(encoding)? No matter what it may be, they cannot come up
correctly if
> you run Mozilla under KOI8-R or C locale. Mozilla assumes that the
filesystem
> charset/encoding (that is used to represent file names and directory
names) is
> that of the current system locale on Unix. Note that unlike NTFS or HFS+
(Mac
> OS) that always store file names in UTF-16, POSIX file system treats
file names
> just as sequences of _bytes_ (null-terminated) without embedded '/'.
Therefore,
> if you run Mozilla under ru_RU.KOI8-R, filenames (seuences of bytes) are
> interpreted as KOI8-R.
>

I have two character sets to work with. en_US and ru_RU. therefore i
cannot understand why filenames are interpreted as having dbcs or unicode
charset. BTW, I forgot to mention that mozilla was built with
-mcpu=atlon-tbird using gcc32. But I doubt that -mcpu switch changes
appearence of widgets so dramatically.

> If you want to make Chinese or Japanese file names  come up correctly
under
> Russian locale, you have to switch to UTF-8 locale (say, ru_RU.UTF-8)
because
> only Unicode can represent Chinese, Japanese and Russian at the same
time. [1]
> This involves renaming all your files (in various legacy
encodings/charset) to
> have UTF-8 names. It'd not present much trouble if FreeBSD's locale
support and
> I18N were much better now than a couple of years ago. Back then, FreeBSD
didn't
> have nl_langinfo, iconv(3) and some critical I18N APIs (of SUS/POSIX)
>

I do not need unicode support. KOI8-R suits me well.

> [1] Well, _to some extent_, old CJK encodings can represent Russian and
Greek,
> but you'd not want to switch from one legacy encoding(KOI8-R) to
another(say,
> EUC-JP). Besides, all Cyrillic letters and Greek letters would be
rendered in
> 'full-width' (or 'double width') cells, which is really annoying.
>
>

All I want to know: what should I do?

Comment 6

[Jungshik Shin]

That's really bizzare. So, you just have LANG without any other LC_'s set. Does
it make any difference if you run Mozilla as below?

  $ env LC_ALL=ru_RU.KOI8-R mozilla

Comment 7

[Jungshik Shin]

I figured out what's going on. Does FreeBSD have iconv(3)? If not, have you
installed libiconv (GNU)? I guess neither is the case. Moreover, FreeBSD(at
least your version) doesn't seem to have a working mb*wc*() and wc*mb(). Or,
even if it does, its wchar_t is not UCS-4 nor UCS-2. So, what Mozilla does is to
fall back and assumes that the file sytem charset is ISO-8859-1.

A few options:

  1. install GNU libiconv [1] and make sure that it's detected at the     
configuration time.  This alone may not be enough because nl_langinfo() is also
necessary for the ICONV codepath to be compiled in.

  2. Ask FreeBSD C library developers to pay more attention to I18N and
implement iconv(3), nl_langinfo() and wc*mb* and mb*wc* if they're not yet
supported. If they're supported in a recent version of FreeBSD C library,
upgrade to it. 

  3. If there's a glibc 2.2 (or later) port for FreeBSD, you can use it, too.


Darin, would it be possible to use our uconv in xpcom/io on 'old' platforms
without a decent I18N API support? Perhaps, possible, but you'd not like to,
would you? 

[1] Search for 'libiconv Bruno Haible' at google. 

Comment 8

[Jungshik Shin]

Ooops. I'm sorry for my rant about FreeBSD C library. My diagnosis was wrong. It
seems to be a byte-swapping issue. Cyrillic letters are in U+0400 block and when
they're byte-swapped, they 'metamorphose' into Chinese characters. 

Can you try attachment 125288 [details] (bug 208809) and attachment 125485 [details] (to bug
209048)?  You can find out how to run them in bug 208809 and bug 209048. 

Comment 9

[Jungshik Shin]

I keep making mistake. Forget about Cyrillic letters (well, it's partly true.
Some Cyrillic letters would turn to Chinese characters and others would turn to
something else, all of them having codepoints ending with 04). Anyway, it's a
byteswapping issue. When was your last cvs update? The relevant file is
xpcom/io/nsNativeCharsetUtils.cpp If it's not up to date, try again after
updating it and recompile in xpcom directory. 

If that doesn't solve the problem, let me know the result of two sample programs
I mentioned in the prev. comment. I also like to know the output of 'iconv
--list' (hmm, this may be GNU extension and is not available on FreeBSD)

Comment 10

[Gervase Markham [:gerv]]

This is an automated message, with ID "auto-resolve01".

This bug has had no comments for a long time. Statistically, we have found that
bug reports that have not been confirmed by a second user after three months are
highly unlikely to be the source of a fix to the code.

While your input is very important to us, our resources are limited and so we
are asking for your help in focussing our efforts. If you can still reproduce
this problem in the latest version of the product (see below for how to obtain a
copy) or, for feature requests, if it's not present in the latest version and
you still believe we should implement it, please visit the URL of this bug
(given at the top of this mail) and add a comment to that effect, giving more
reproduction information if you have it.

If it is not a problem any longer, you need take no action. If this bug is not
changed in any way in the next two weeks, it will be automatically resolved.
Thank you for your help in this matter.

The latest beta releases can be obtained from:
Firefox:     http://www.mozilla.org/projects/firefox/
Thunderbird: http://www.mozilla.org/products/thunderbird/releases/1.5beta1.html
Seamonkey:   http://www.mozilla.org/projects/seamonkey/

Comment 11

[Gervase Markham [:gerv]]

This bug has been automatically resolved after a period of inactivity (see above
comment). If anyone thinks this is incorrect, they should feel free to reopen it.