Closed Bug 150131 Opened 22 years ago Closed 3 years ago

how to make a WM render chars. outside the char repertoire of the locale properly in Window title bar

Categories

(Core :: Internationalization, defect)

x86
Linux
defect
Not set
minor

Tracking

()

RESOLVED INVALID

People

(Reporter: u32858, Assigned: jshin1987)

References

Details

(Keywords: intl, relnote)

Attachments

(2 files)

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0rc2) Gecko/20020510
BuildID:    2002051013

Sending message dialog box has ???? for japanese subject

Reproducible: Sometimes
Steps to Reproduce:
1.send an email with a japanese iso-2022-jp subject
2.
3.

Actual Results:  ????

Expected Results:  おーはよ
looks like we are missing some translation somewhere!! Naoki, can you take care
of this one?
Xianglan, could you try this on Linux?
Assignee: ducarroz → nhotta
QA Contact: esther → ji
I don't see this problem in Japanese locale.
Reporter, are you talking about the problem with window title on the dialog?
The window title display depends on the locale. If mozilla is started in a
non-Japanese locale, the window title will have Japanese characters displayed as
question marks. Could you attach a screenshot? Thanks.
Keywords: intl
Please also attach the sent mail. I wonder if that was really sent as ISO-2022-JP.
Attached image utf-8 japanese subject
utf-8 japanese subject
iso-2022-jp japanese subject
Hi, Ji, please see screen shots.
Hi nhotta,

From - Tue Jun 11 11:55:59 2002
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00800000
Message-ID: <3D0566BB.6010500@jguk.org>
Date: Tue, 11 Jun 2002 11:55:55 +0900
From: "J. Grant" <jg@jguk.org>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0rc2) Gecko/20020510
X-Accept-Language: en-gb, en
MIME-Version: 1.0
To: jg <jg@jguk.org>
Subject: =?ISO-2022-JP?B?GyRCRnxLXDhsGyhCIGlzby0yMDIyLWpwIHRlc3Q=?=
Content-Type: multipart/mixed;
 boundary="------------080506060608010109080403"

This is a multi-part message in MIME format.
--------------080506060608010109080403
Content-Type: text/plain; charset=ISO-2022-JP
Content-Transfer-Encoding: 7bit


--------------080506060608010109080403
Content-Type: application/pdf;
 name="mmc2r02.pdf"
Content-Transfer-Encoding: base64
Content-Disposition: inline;
 filename="mmc2r02.pdf"



reporter, from your screen shots i can tell that you are on en-gb locale and
what you're actually seeeing is not headers that are incorrect but window
titles. This result is expected if you are not log on into japanese locale. 
Status: UNCONFIRMED → NEW
Ever confirmed: true
Summary: Sending message dialog box has ???? for japanese subject → Sending message dialog box (window title) has ???? for japanese subject
Component: Composition → Internationalization
Product: MailNews → Browser
This is not mail specific, change product to browser.
Linux bug, reassign to shanjian, cc to ftang.
Assignee: nhotta → shanjian
Actually, this bug was fixed in a sense for Linux a long time ago.
Mozilla does its best and the rest is up to a Window manager.

Mozilla sets  _NET_WM_NAME window property(UTF8_STRING) and
window managers(e.g. I believe the latest version
of kwin in KDE works fine) that know about this window property CAN
interpret this and display multilingual subject/title in
the window title bar. For details, see 
bug 9449 (comment 39 to comment 58) as well as 
attachment 41935 [details] [diff] [review]. 

See also bug 74753. 
 
Jungshik, according to your comments in bug # 9449 this bug was fixed for Linux
only for a special case: same locale > This bug has been RESOLVED (at least
partially) for Linux/Unix/X11 by Frank's patch dated January 4, 2000 (see my
comment on bug 2426) in that the window title bar can correctly display
non-ISO-8859-1 strings as long as the locale under which window manager is
running is the same as the locale/encoding in which the page viewed in
browser(i.e. title)
is encoded.>
That's not the case for this bug report, mozilla is not running on ja-jp locale
, under that locale the window titles are displaying correctly. So this
particular bug was not fixed... going back to bug# 9449

I'm afraid you got me wrong. (I wonder why you quoted my
comment 31 written much earlier which is irrelevant to
this bug when I specifically wrote that comment
39 to comment 59 of bug 9449 had to be refered to)
I've been following bug 9449
long enough to know how to tell one case from the other. 
The first case (displaying Japanese title under Japanese
locale) was fixed by Frank in January 2000. The second
case (displaying Japanese title under,say, Russian locale)
was fixed later by  lya Konstantinov's patch that
has been checked into Mozilla source on 2001-07-13
(see comment 61 of bug 9449).

In other words, this bug was _fixed_ for Linux REGARDLESS of
whether the string to put on the window title bar 
is covered by the current locale or NOT. You should
have read from comment 39 to comment 59(in bug 9449). Those comments
are about putting _NET_WM_NAME window property.(please,
see also attachment 41935 [details] [diff] [review]) 
This property has to be interpreted by window managers
that understand it and used by them to put on any
UTF-8 string this property contains REGARDLESS of
whether that string is within the character
repertoire of the present locale or NOT.  

Then, why this bug report? That's because the reporter
does not use a Window manager that understands
_NET_WM_NAME (UTF8_STRING) window property. Mozilla
does as best as it can by doing its part
(that is, setting _NET_WM_NAME property) and it's NOT Mozilla
BUT Window managers that  have to do the
*rest* of the job , that is, interpreting
_NET_WM_NAME and rendering its content
properly in the title bar. 
> 
> ------- Additional Comments From marina@netscape.com  2002-06-11 10:07 -------
> reporter, from your screen shots i can tell that you are on en-gb locale and
> what you're actually seeeing is not headers that are incorrect but window
> titles. This result is expected if you are not log on into japanese locale.
> 

how did you decide en-gb? the headers should not effect it i think

$ locale
LANG=ja_JP
LC_CTYPE=ja_JP
LC_NUMERIC=en_GB
LC_TIME=en_GB
LC_COLLATE=en_GB
LC_MONETARY=en_GB
LC_MESSAGES=en_GB
LC_PAPER="ja_JP"
LC_NAME="ja_JP"
LC_ADDRESS="ja_JP"
LC_TELEPHONE="ja_JP"
LC_MEASUREMENT="ja_JP"
LC_IDENTIFICATION="ja_JP"
LC_ALL=

my locale is set for japanese, there should be no problem, does mozilla not take
it from the enviroment variables?


JG
> Then, why this bug report? That's because the reporter
> does not use a Window manager that understands
> _NET_WM_NAME (UTF8_STRING) window property. Mozilla
> does as best as it can by doing its part
> (that is, setting _NET_WM_NAME property) and it's NOT Mozilla
> BUT Window managers that  have to do the
> *rest* of the job , that is, interpreting
> _NET_WM_NAME and rendering its content
> properly in the title bar. 

  Given that many users still use window managers that
don't understand _NET_WM_NAME property, I think 
this has to be 'release-noted'. That is, 
the window title bar can render a string made up 
of characters outside the character repertoire
of the present locale ONLY IF a window manager
compliant to extended WM spec
(http://www.freedesktop.org/standards/wm-spec/)
is used. Those window managers include 
kwin (KDE 2.2 or later) and blackbox
(see the screenshots at
  http://www.debian.or.jp/~kubota/mojibake/window-managers.html
)
> how did you decide en-gb? the headers should not effect it i think

> $ locale

> LC_MESSAGES=en_GB

> my locale is set for japanese, there should be no problem, does mozilla not take
> it from the enviroment variables?

  The cause of your problem is that Mozilla uses LC_MESSAGES
insted of LC_CTYPE to determine the character repertoire of
the present locale. I argued to Erik that LC_MESSAGES MUST be DISREGARDED
when determining the character repertoire, but I failed to
persuade him. It was perhaps two or three years ago.

 
Just like you set LC_MESSAGES to en_GB with
LC_CTYPE to ja_JP(.eucJP), I usually set 
LC_MESSAGES to C/POSIX/en_US with LC_CTYPE set to ko_KR.eucKR
(well, these days, I now use en_US.UTF-8 for LC_MESSAGES
and ko_KR.UTF-8 for the rest of LC_*). This leads Mozilla
not to be able to render Korean string in the window title
bar even though LC_CTYPE is set to ko_KR.eucKR UNLESS
I use a window manager compliant to extended WM spec
mentioned in my previous comment.  Therefore, I was forced
to write a shell wrapper to invoke Mozilla in which
I reset LC_MESSAGES to ko_KR.eucKR to work around
this problem in Mozilla. 

To see it yourself, try Mozilla with
LC_MESSAGES set to ja_JP (or LC_ALL set to ja_JP).

Now, I'll try once more to make my case that
LC_MESSAGES should NOT be refered to when
determining the character repertoire of the
present locale. LC_CTYPE MUST be used instead.
More specifically,

1. LC_ALL should be used if set.
2. LC_CTYPE should be used if set.
3. LANG should be used if set.
Hi Jshin,

I think your idea is the best way forward, i can not think of a reason why this
should not be the way mozilla works. It would solve more problems if it is as
such IMO.

JG
> Now, I'll try once more to make my case that
> LC_MESSAGES should NOT be refered to when
> determining the character repertoire of the
> present locale. LC_CTYPE MUST be used instead.

  I filed a separate bug for this. It's bug 151112. 


> More specifically,

> 1. LC_ALL should be used if set.
> 2. LC_CTYPE should be used if set.
> 3. LANG should be used if set.

  I'm taking back the above 'specifics'.
It turned out that Mozilla uses 'setlocale(category,"")'
of which the behavior is implementation dependent according
to Posix and Single Unix spec. What I described above
is what glibc does, but other C library may do it
differently. Whatever it may be on a given platform, it would be 
consistent with what users of that platform expect. 
Therefore, we don't have to worry about it.  
> the window title bar can render a string made up 
> of characters outside the character repertoire
> of the present locale ONLY IF a window manager
> compliant to extended WM spec
> (http://www.freedesktop.org/standards/wm-spec/)
> is used. Those window managers include 
> kwin (KDE 2.2 or later) and blackbox.

I was wrong about blackbox. blackbox does not yet take
care of _NET_WM_NAME properly. It's in its TODO list.
There are some lines of code put in already(although
#ifdef-ed out), but it's not yet fully implemented. 

Some clarification:

Even with a window manager aware of _NET_WM_NAME property,
arbitrary UTF-8 strings would not be rendered correctly
if the window manager is launched under non-UTF-8 locale
(e.g. ja_JP.eucJP, ko_KR.eucKR, fr_CA.iso8859-1, etc)
This is akin to running MS IE(or Mozila with
Roy's or my patch for bug 9449) under MS Windows 9x/ME, which
is not based on Unicode but dependent on 'ANSI' code page
(in the example above, it's EUC-JP, EUC-KR, or ISO8859-1
in Unix/Linux) 

On the other hand, if a window manager is launched
under UTF-8 locale(ko_KR.UTF-8, en_GB.UTF-8,
ja_JP.UTF-8, etc), which is similar to running
MS IE(or Mozilla with Roy's/my patch for bug 9449) 
under Win 2k/XP, the locale under which Mozilla
is launched does NOT matter. No matter what locale under
which Mozilla is running(even C/POSIX) may be, 
_WM_NET_NAME-aware window managers should be able to 
render UTF-8 strings properly in the window title bar
provided that  the fontset (in the configuration
of the window manager) is set correctly to
cover the glyphs required of rendering a given
UTF-8 string. 

reassign this bug to Jungshik, he knows this well. 
Assignee: shanjian → jshin
Accepting. 

I think the product field has to be changed to 'documentation'
(is this right category for 'release note' issue? when I get
confirmation that it is,
I'll change the product field) because
there's little, if any, that has to be modified in Mozilla
source code. Something along the line of comment 20,
however, has to be release-noted. 

This is a bit complicated issue because what's required for
multilingual UTF-8 title to get rendered correctly 
is beyond the control of Mozilla (window managers and
UTF-8 locales). It'll take pretty long
before every user switches to UTF-8 locales. It'll also take
a while for _NET_WM_NAME-aware window managers to replace
non-NET_WM_NAME-aware window managers on users' desktops.  
Either of two conditions (that is, one or the other or
both) has to be met for multilingual strings
to be rendered correctly in the window title bar. 
Status: NEW → ASSIGNED
Blocks: 104320
*** Bug 157602 has been marked as a duplicate of this bug. ***
changing the summary line to more accurately reflect what's at issue
Summary: Sending message dialog box (window title) has ???? for japanese subject → how to make WM render chars. outside the repertoire of the locale of WM properly in Window title bar : to be release noted
Keywords: relnote
Summary: how to make WM render chars. outside the repertoire of the locale of WM properly in Window title bar : to be release noted → how to make a WM render chars. outside the char repertoire of the locale properly in Window title bar
In comment #19, I wrote: 

> It turned out that Mozilla uses 'setlocale(category,"")'
> of which the behavior is implementation dependent according
> to Posix and Single Unix spec. What I described above
> is what glibc does, but other C library may do it
> differently.

  I was wrong again. About a month ago, I learned that
how setlocale(category,"") works is not implementation-dependent
but is explicitely specified by Single Unix Spec., which is
the same as what glibc does and what I described in comment #17
which I mistakenly took back in comment #19. 
 

*** Bug 109599 has been marked as a duplicate of this bug. ***
> This is a bit complicated issue because what's required for
> multilingual UTF-8 title to get rendered correctly 
> is beyond the control of Mozilla (window managers and
> UTF-8 locales). It'll take pretty long
> before every user switches to UTF-8 locales. 

  Thansk to RedHat 8, it has already happened and I hope other
Linux vendors will follow soon. On other Unix platforms, Solars
8/9 has offered UTF-8 locales for a while. So has AIX. 
It's unfortunate that
RH 8 uses legacy encodings for CJK,though. For all other languages,
it uses UTF-8. Even for CJK encodings, by modifying /etc/X11/gdm/locale.aliases,
one can use UTF-8 locale. (there's another change to make in XLC_LOCALE
directory). I've been using ko_KR.UTF-8 locale since last spring. 

> It'll also take a while for _NET_WM_NAME-aware window managers to replace
> non-NET_WM_NAME-aware window managers on users' desktops.  

  Kwin in KDE3 is one of NET_WM_NAME-aware WM.  I believe metacity
(the default WM for Gnome2 in RH8) also honors _NET_WM_NAME. 
Anyway, in KDE, all you have to do to display multilingual title displayed
correctly in the window title bar is :

  - go to Kontrol Panel (KDE Control Panel)
  - go to 'Apperance and Usage' (I'm translating Korean menu back to English
     so that  the original Enlgish menu name may be a little different)
     and choose 'fonts'
  - For Window title font, select one of 'Pan-Unicode fonts' such as 
     'CODE2000' (http://home.att.net/~jameskass) or Bitstream
     Cyberbit or Arial MS Unicode. You may not use the last one
     because of a license issue.
  - Of course, your fontconfig configuration file (/etc/fonts/fonts.conf or
    ~/ .fonts.conf) has to have a directory where you have aforementioned
    fonts. 

I'll test it with metacity in Gnome2 (Solaris9 reportedly uses metacity as well
in its Gnome desktop) and get back here with the test result.
In case of metacity under RH8, it just works out of the box even under legacy CJK
encodings are used in CJK locales. That is, UTF-8 locale is not required for
metacity to honor _NET_WM_NAME. That's supposed to be the case, but 
some WMs don't work that way and UTF-8 is required for them to render
the full repertoire of Unicode string. 
*** Bug 179654 has been marked as a duplicate of this bug. ***
*** Bug 211077 has been marked as a duplicate of this bug. ***
*** Bug 150958 has been marked as a duplicate of this bug. ***
Depends on: 315947
*** Bug 70853 has been marked as a duplicate of this bug. ***
QA Contact: ji → i18n
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: