128875 - Mozilla crashes when invoking IME in UTF-8 locale

Status: RESOLVED WORKSFORME

(Core :: Internationalization, defect)

Description

[ji]

This is originally reported in Chinese linux forum and confirmed with the trunk
build.

On RH7.2, in zh_CN.UTF-8 locale, mozilla crashes immediately when invoking chinput.

Steps to reproduce:
1. Set locale to zh_CN.UTF-8 and XMODIFIERS=@im=chinput
2. Launch browser, bring up HTML composer.
3. Press Ctrl <space> to invoke the Chinese IME. Mozilla crashes. 

Note:
The crash doesn't happen in zh_CN.GB2312 locale.
Callstack to follow.

Comment 1

[ji]

Nominating for nsbeta1 since it's a crash.
Incident ID: 3627405
libX11.so.6 + 0x8c0ad (0x404430ad)
libX11.so.6 + 0x8ebcb (0x40445bcb)
libX11.so.6 + 0x80d91 (0x40437d91)
libX11.so.6 + 0x811a7 (0x404381a7)
libX11.so.6 + 0x702b4 (0x404272b4)
libX11.so.6 + 0x6987a (0x4042087a)
libX11.so.6 + 0x69c5d (0x40420c5d)
libX11.so.6 + 0x69455 (0x40420455)
libX11.so.6 + 0x7dc8f (0x40434c8f)
libX11.so.6 + 0x7ec19 (0x40435c19)
libX11.so.6 + 0x7edbb (0x40435dbb)
libX11.so.6 + 0x7dcd8 (0x40434cd8)
libX11.so.6 + 0x4df6b (0x40404f6b)
libgdk-1.2.so.0 + 0x17add (0x4035aadd)
libgdk-1.2.so.0 + 0x17d5d (0x4035ad5d)
libglib-1.2.so.0 + 0x11773 (0x4038d773)
libglib-1.2.so.0 + 0x11d39 (0x4038dd39)
libglib-1.2.so.0 + 0x11eec (0x4038deec)
libgtk-1.2.so.0 + 0x94333 (0x402a9333)
nsAppShell::Run()
nsAppShellService::Run()
netscape-bin + 0x7e89 (0x0804fe89)
netscape-bin + 0x86d7 (0x080506d7)
libc.so.6 + 0x1c507 (0x404d4507)

Comment 2

[Roy Yokoyama]

over to katakai.  Do you have time to look at this before 1.0?

Comment 3

[Masaki Katakai]

I'll try.

Ji, can you test on other UTF-8 locales? e.g. ja_JP.UTF-8 and
zh_TW.UTF-8 locale?

Comment 4

[Masaki Katakai]

I remember that the same problem had been filed once into BugZilla JP
for ja_JP.UTF-8 locale of Kondara distribution.

http://bugzilla.mozilla.gr.jp/show_bug.cgi?id=1567

I'm thinking this is bug of Xlib.

Ji, can you try this pref can solve this issue?

user_pref("xim.input_style","over-the-spot");

Comment 5

[ji]

Yes, it also crashes in ja_JP.UTF-8 locale.
And using over-the-spot pref solves the problem.

Comment 6

[Masaki Katakai]

No way to fix in Mozilla side. It seems that X lib problem.
The very simple example of callback style (Mozilla's defualt)
also causes dumpcs core. Here is a stack,

Program received signal SIGSEGV, Segmentation fault.
0x400c0886 in _XwcTextListToTextProperty () from /usr/X11R6/lib/libX11.so.6
(gdb) where
#0  0x400c0886 in _XwcTextListToTextProperty () from /usr/X11R6/lib/libX11.so.6
#1  0x400b1e76 in _XlcSetConverter () from /usr/X11R6/lib/libX11.so.6
#2  0x400b1c53 in _XlcConvert () from /usr/X11R6/lib/libX11.so.6
#3  0x400a1e86 in _Ximctstombs () from /usr/X11R6/lib/libX11.so.6
#4  0x4009a6e7 in _XimCbDispatch () from /usr/X11R6/lib/libX11.so.6
#5  0x4009ac38 in _XimCbDispatch () from /usr/X11R6/lib/libX11.so.6
#6  0x4009a423 in _XimCbDispatch () from /usr/X11R6/lib/libX11.so.6
#7  0x400af6cb in _XimXConf () from /usr/X11R6/lib/libX11.so.6
#8  0x400afe1a in _XimRead () from /usr/X11R6/lib/libX11.so.6
#9  0x400a18cf in _XimTriggerNotify () from /usr/X11R6/lib/libX11.so.6
#10 0x4009c9dc in _XimLookupWCText () from /usr/X11R6/lib/libX11.so.6
#11 0x4007a013 in XFilterEvent () from /usr/X11R6/lib/libX11.so.6
#12 0x0804969a in main ()
#13 0x401481be in __libc_start_main (main=0x8049090 <main>, argc=1,
ubp_av=0xbffff7dc, 
    init=0x8048bb8 <_init>, fini=0x804a168 <_fini>, rtld_fini=0x4000ddf0
<_dl_fini>, 
    stack_end=0xbffff7cc) at ../sysdeps/generic/libc-start.c:129
(gdb) 

I'll report this problem to X side.

Workaround is to set "over-the-spot". We should describe this into
release notes.

Comment 7

[Masaki Katakai]

I've post this problem against xfree86.org, waiting the answer...

Comment 8

[ji]

Katakai-san, a user says he doesn't see the crash in UTF-8 locale on TurboLinux
7 and there might be some differences in
/usr/X11R6/lib/X11/locale/zh_CN.UTF-8/XLC_LOCALE

TurboLinux 7 version of XLC_LOCALE is as follow:
# XFree86 NLS for Chinese locale zh_CN.UTF-8
# Modified from xc/nls/XLC_LOCALE/en_US.UTF-8
# by James Su <[e-mail]>

#
# XLC_FONTSET category
#
XLC_FONTSET

on_demand_loading True

object_name generic

# We leave the legacy encodings in for the moment, because we don't
# have that many ISO10646 fonts yet.
# fs0 class (7 bit ASCII)
fs0 {
charset {
name ISO8859-1:GL
}
font {
primary ISO8859-1:GL
vertical_rotate all
}
}

# fs1 class (ISO8859 families)
fs1 {
charset {
name ISO8859-1:GR
}
font {
primary ISO8859-1:GR
}
}

# fs2 class (Chinese Han Character)
fs2 {
charset {
name GB2312.1980-0:GL
}
font {
primary GB2312.1980-0:GL
}
}

# fs3 class (Chinese Han Character GBK)
fs3 {
charset {
name GBK-0:GLGR
}
font {
primary GBK-0:GLGR
substitute GB13000.1993-1:GLGR
}
}

# fs4 class
fs4 {
charset {
name ISO10646-1
}
font {
primary ISO10646-1
}
}
END XLC_FONTSET

#
# XLC_XLOCALE category
#
XLC_XLOCALE

encoding_name UTF-8
mb_cur_max 6
state_depend_encoding False

# cs0 class
cs0 {
side GL:Default
length 1
ct_encoding ISO8859-1:GL
}

# cs1 class
cs1 {
side GR:Default
length 1
ct_encoding ISO8859-1:GR
}

# cs2 class
cs2 {
side GR
length 2
ct_encoding GB2312.1980-0:GL; GB2312.1980-0:GR
}

# cs3 class
cs3 {
side none
ct_encoding ISO10646-1
}

END XLC_XLOCALE 

Could it be a problem of XLC_XLOCALE on RH7.2?

Comment 9

[ji]

Did more testing. On RH7.2, there is no zh_CN.UTF-8 directory in
/usr/X11R6/lib/X11/locale/
I created one and copied over TurboxLinux version of XLC_LOCALE to 
/usr/X11R6/lib/X11/locale/zh_CN.UTF-8
mozilla still crashes with this.

Comment 10

[Masaki Katakai]

Sorry I don't have RH7.2 here now. 

Ji, Can you find the following file in RH ?

/usr/X11R6/lib/X11/locale/locale.dir

I think we should modify this when you put XLC_LOCALE.

en_US.UTF-8/XLC_LOCALE  zh_CN.UTF-8

to

zh_CN.UTF-8/XLC_LOCALE  zh_CN.UTF-8

Also can you find the message below in locale.dir ? I can see
the message on Kondara Japanese distribution and I understand
UTF-8 locales are still not working properly yet.

# Note: The UTF-8 locales don't work correctly yet. Work in progress.

Comment 11

[ji]

Katakai-san, changing the line in locale.dir doesn't help.
locale.dir file on RH7.2 has the same notes indicating that UTF-8 locales don't
work correctly yet.

Comment 12

[Frank Tang]

nsbeta1-, needhelp. 

Comment 13

[Jungshik Shin]

> I think we should modify this when you put XLC_LOCALE.

> en_US.UTF-8/XLC_LOCALE  zh_CN.UTF-8

> to

> zh_CN.UTF-8/XLC_LOCALE  zh_CN.UTF-8

In locale.dir, there's another line like the above but with ":". That
entry has to be changed the same way. That is,

en_US.UTF-8/XLC_LOCALE   : zh_CN.UTF-8

has to be replaced by

zh_CN.UTF-8/XLC_LOCALE   : zh_CN.UTF-8

Comment 14

[Jungshik Shin]

Attachment: XLC_LOCALE file for ko_KR.UTF-8

I have a similar problem with ko_KR.UTF-8 locale (bug 138443) 
Attached is my XLC_LOCALE file for ko_KR.UTF-8 locale.
It has to be put in /usr/X11R6/lib/X11/locale/ko_KR.UTF-8 directory.
Two lines in locale.dir should be changed as shown below
to make this effective.

en_US.UTF-8/XLC_LOCALE	     ko_KR.UTF-8
en_US.UTF-8/XLC_LOCALE:      ko_KUR.UTF-8   (# note there's a colon here)

ko_KR.UTF-8/XLC_LOCALE	      ko_KR.UTF-8
ko_KR.UTF-8/XLC_LOCALE:       ko_KR.UTF-8   (# note there's a colon )

My XLC_LOCALE for ko_KR.UTF-8 is that of Turbo Linux for zh_CN.UTF-8 locale
in that iso10646-1 entry is put at the end instead of at the beginning.
The difference is I have non-Korean entries like jis0208,jis0201,
gb2312 while zh_CN.UTF-8 in Turbo Linux only has SC entry
in addition to iso8859-1 and iso10646-1 entries.

Comment 15

[Jungshik Shin]

*** Bug 138443 has been marked as a duplicate of this bug. ***

Comment 16

[Jungshik Shin]

I confirmed that
setting the xim.input_style to 'over-the-spot' made Mozilla not crash
under ko_KR.UTF-8 locale. 

Katakai-san,
Have you heard back anything from XFree86? Where did you report
the problem? You may consider posting it to [e-mail] (i18n
mailing list) if you haven't yet. 

BTW, it seems like you don't have this problem under Solaris(Sparc) judging
from that you're certain this is XFree86's problem, right? 

Comment 17

[Jungshik Shin]

> No way to fix in Mozilla side. It seems that X lib problem.
> The very simple example of callback style (Mozilla's defualt)
> also causes dumpcs core. Here is a stack,

  I've just installed XFree86 4.2.0 (with some RedHat patches applied)
from Rawhide (Redhat development version) and this problem is now gone.
Mozilla doesn't crash anymore in composer when I input Korean
characters even if it's launched under ko_KR.UTF-8 locale
and xim.input_style is 'on-the-spot' (as opposed to 'over-the-spot').

Therefore, this should be 'release-noted'.

  On Linux and any other system using XFree86 earlier than
4.2.0, xim.input_style should be set to 'over-the-spot' to prevent
Mozilla from crashing under a UTF-8 locale  if XIM server is
used to input CJK characters. Under a non-UTF-8 locale, 
'on-the-spot' input style can be used without a problem.

Comment 18

[Brant Gurganus]

By the definitions on <http://bugzilla.mozilla.org/bug_status.html#severity> and
<http://bugzilla.mozilla.org/enter_bug.cgi?format=guided>, crashing and dataloss
bugs are of critical or possibly higher severity.  Only changing open bugs to
minimize unnecessary spam.  Keywords to trigger this would be crash, topcrash,
topcrash+, zt4newcrash, dataloss.

Comment 19

[Andrew Young]

I'm having a similar, but slightly different issue with RedHat 8.0 and Mozilla
1.0.1 and Firebird 0.6.  If I use a UTF-8 locale (en_US.UTF-8 or ja_JP.UTF-8)
and try to invoke XIM (using shift+space in my case with kinput2 and canna)
nothing happens.  It doesn't crash, but it doesn't open XIM input either. 
Changing the xim.input_style has no affect on the problem.  If I switch to
another locale, like ja_JP it works fine.  All other X apps I'm using seem to
handle the UTF-8 locales ok as far as XIM is concerned.  Looking at the
/usr/X11R6/lib/X11/locale/ directory, there is no ja_JP.UTF-8 locale, but it
maps to the en_US.UTF-8 locale via the locale.dir file.  My
en_US.UTF-8/XLC_LOCALE file is basicly the same as the ones already mentioned. 
(Also of note is that en_US.UTF-8 is the only en_US locale on RedHat 8.0)

Comment 20

[Jungshik Shin]

Can you try what I wrote at 
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=75829
for enabling ja_JP.UTF-8 and ko_KR.UTF-8 locales on RH 8.0? 
With that change, can you check whether you still have the problem? 
 

Comment 21

[Andrew Young]

I tried that, but it didn't fix this issue.  Thanks for the info though, I'm
using your XLC_LOCALE files now.

Comment 22

[Jungshik Shin]

Andrews, do you use Metacity? Upgrading it may solve the problem. See bug 210134. 

Comment 23

[Andrew Young]

My issue doesn't happen in the ja_JP locale, only in the ja_JP.UTF-8 locale, bug
210134 is talking about problems with the ja_JP locale.  I would think that if
it were a bug with Metacity, I would have problems in both locales, but I don't.
 Also, that bug says that the conversion window comes up, but there is no way to
commit the conversion after you're done entering text.  The problem I have is
that the conversion window never comes up at all.

Thanks for the suggestion though.

Comment 24

[Jungshik Shin]

That's strange. I'm now running Mozilla 1.3 (I'll try a nightly later. I
shouldn't be doing this now....) under ja_JP.UTF-8(RH 8, Metacity. I haven't
upgraded Metacity because I usually use KDE) and Kinput2 'works' (although it
doesn't seem to work perfectly) When I type 'asahi', I can get three Hiragana
characters and a list of candiates (Kanji words). 

Comment 25

[Andrew Young]

Well, I rebuilt from the same sources I had built from before and it fixed
itself.  The only difference is that this time I specified gtk2 as the default
toolkit instead of whatever toolkit (xlib?) is the default for Mozilla under
Linux.  So that fixes it as far as I'm concerned I guess, although I'm really
confused as to why it wasn't working.  Sorry for the confusion.

Comment 26

[Ed Anderson]

> I've just installed XFree86 4.2.0 (with some RedHat patches applied)
> from Rawhide (Redhat development version) and this problem is now gone.
> Mozilla doesn't crash anymore in composer when I input Korean
> characters even if it's launched under ko_KR.UTF-8 locale
> and xim.input_style is 'on-the-spot' (as opposed to 'over-the-spot').

I am running XFree86 4.2.1 on Debian Sid, but I still have this problem.  I am
running the Debian release of Mozilla 1.5-2.

Comment 27

[Jungshik Shin]

If you set the XIM input style to over-the-spot, does it work? 

Anyway, why don't you upgrade your XF86 to the latest in 4.3 branch (there was a
bug fixed after 4.3 release that caused crash at flash site when XIM server is
running).  I haven't hit this crash problem for over a year :-)

Comment 28

[Wayne Mery (:wsmwk)]

WFM per comment 27.
Please reopen if you still see this problem.