854825 - Japanese (Korean/Chinese) XIM input mode indicator has not been working for Thunderbird and Firefox under linux. (with known solution)

Status: NEW

(Core :: DOM: UI Events & Focus Handling, defect, P5)

Description

[ISHIKAWA, Chiaki]

I am using TB 17.0.4 under Debian GNU/Linux.

For like 8 months, I have noticed that the input mode indicator for
Japanese input that uses XIM protocol (when I use kinput2 for example)
is not working right when I use TB to write Japanese messages and
such.  (And for that matter, this is the same with FF, but I don't
write lengthy message in Japanese using FF so often).

The symptom is like this.

Before, when all was well, a small rectangle at the lower-left corner
of TB (or FF) window, attached as an extra small rectangle [not
inside], showed Japanese input mode that uses XIM protocol by using
some Japanese characters to indicate the mode when the Japanese input mode is on (and I assume Korean or
Chinese characters for people who use XIM input methods to type in
these characters)

But for like 8 months, the programs, both TB and FF, have shown only
black rectangle and no characters are visible at all there.

The lack of visible input mode is sometimes very irritating because
we can't tell which characters Hiragana, Katakana, we will type in.
(Or if I make a typing mistake and it is hard to tell when unintended character shows up because of the typing mistake or  using the incorrect mode, etc.)

This problem began happening after an upgrade of Debian package(s) last
summer and so I suspect that it may have something to do with the
X-related libraries, but I could not figure out exactly why until
yesterday.  (I suppose ordinary users can't figure out the cause of
such a problem even though the symptom is easy to describe.)

In a post to a Japanese BBS, I found a mention by an anonymous poster
that explains this as follows:

There seems to have been a problem in GTK2 library.  Starting in gtk2
2.21.6, function on_status_window_expose_event() in
modules/input/gtkimcontextxim.c has been changed (to the worse. I
think it means buggy?)  If we revert the function,
on_status_window_expose_event(), to a previous code found in 2.21.5
(or earlier), all is well.  We can run make below modules/input
directory only, and use im-xim.so thus produced to fix the problem,
etc.

This was posted July 5th, 2012, LAST YEAR.

I wish I had known this earlier.

I have checked this out and this indeed fixes the problem for me (!).

I am submitting this information, even if the original cause is
outside the mozilla code base proper, because I believe this behavior
affects the usability of TB and FF by XIM input users very much.  

Some users can do the local fix, or ask an admin-like help to do this for them based on what I write below. [I am submitting a bug fix to gtk org as well.]

[ looking at the code, I realize that, maybe through GNOME desktop
setting, if I change the base default color to white or something, I
should be able to regain the white rectangle where the XIM input mode
is indicated by characters without the library swap.  But gnome
desktop setting change by gconf editor and the like is so confusing to
an ordinary user. I experimented for a while and gave up. ]

The anonymous poster didn't mention the name of his linux
distribution, but I suspect this problem may have been observed by
many XIM input users, under linux, of East Asian languages for some
time. Debian is very conservative in upgrading packages and so it hit me since last August or sometime. But I believe there had been many users hit by the problem earlier. I believe he/she felt the same and posted the solution that worked to share the information in  a large BBS here in Japan.
 
TIA

WHAT I DID exactly:

My Debian seems to use GTK 2.24.10 from unstable release (from reading
/usr/share/doc/libgtk2.0-dev/changelog.Debian.gz
and so I downloaded 2.24.10 source and run configure.

$ configure

I patched modules/input/gtkimcontextxim.c
as was explained by the anonymous poster. [Reverted the code for the
particular function to an earlier one.]

I ran make at the top-most directory. Just trying to run make below
modules/input doesn't work. The library compilation seems to need an
upper-level library or something in the first place. You need to run
this whole make once anyway.

After a while the compilation finishes.
The library seems to be created here in gtk2+ tree.

    ./modules/input/.libs/im-xim.so

So I saved it to easy place to pick up later.

cp ./modules/input/.libs/im-xim.so /tmp

Now, where is the system-supplied im-xim.so located?

$ mlocate im-xim.so
/usr/lib/i386-linux-gnu/gtk-2.0/2.10.0/immodules/im-xim.so
/usr/lib/i386-linux-gnu/gtk-3.0/3.0.0/immodules/im-xim.so

Oh my.? Do i use gtk-3.0 ???
and not even 2.24.10???

Anyway I tried replacing the gtk-2.0 version of im-xim.so.
(I think TB is using gtk-2.0 version right now : at least TB 17.0.2.)

Before I did the following, I quit TB and a few other programs just in
case.

# cd /usr/lib/i386-linux-gnu/gtk-2.0/2.10.0/immodules

I replaced im-xim.so with the one I created using the
downloaded and patched gtk2 source.

# cp im-xim.so im-xim.so.BACKUP (just in case)
# mv /tmp/im-xim.so ./	      (replaced from the newly created one.)

I restarted TB, and voila, now I can get the
XIM indicator (Japanese character) in white background as before!!!

BTW, trying to link in only im-xim.so using
LD_PRELOAD=the-path-to-im-xim.so didn't work.
During loading it complained something about
missing  module. I see the following message on the console,
and Japanese input did not work at all.
Gtk-Message: Failed to load module "canberra-gtk-module"

Hope this helps XIM input users who are missing XIM input mode status 
indicator.


PS: the code analysis.

I looked at the code repository at http://www.gtk.org/.
The previous code for gtk2 (2.21.5 or older)  paints the
whole rectangle area for the XIM mode indicator
with a couple of different colors (one in full
size, the other in a size which is
one pixel smaller to draw the border, I think.).
https://git.gnome.org/browse/gtk+/tree/modules/input/gtkimcontextxim.c?id=2.21.5

/* Draw the background (normally white) and border for the status window
 */
static gboolean
on_status_window_expose_event (GtkWidget      *widget,
			       GdkEventExpose *event)
{
  gdk_draw_rectangle (widget->window,
		      widget->style->base_gc [GTK_STATE_NORMAL],
		      TRUE,
		      0, 0,
		      widget->allocation.width, widget->allocation.height);
  gdk_draw_rectangle (widget->window,
		      widget->style->text_gc [GTK_STATE_NORMAL],
		      FALSE,
		      0, 0,
		      widget->allocation.width - 1, widget->allocation.height - 1);

  return FALSE;
}

Please note the difference of the third argument to
gdk_draw_rectangle(): TRUE for the first invocation and FALSE for the
second. Third argument controlls the "filling" of graphics operation.
I think the first one fills the rectangle with the base color (i.e.,
the default background, which is white on my desktop.)

The new code quoted below uses a different graphics library
API, but paints the two areas (the sizes being different by one pixel)
in a SINGLE color, which I think corresponds to BLACK, in my desktop
setting.

https://git.gnome.org/browse/gtk+/tree/modules/input/gtkimcontextxim.c?id=2.24.17

/* Draw the background (normally white) and border for the status window
 */
static gboolean
on_status_window_expose_event (GtkWidget      *widget,
			       GdkEventExpose *event)
{
  cairo_t *cr;

  cr = gdk_cairo_create (widget->window);

  gdk_cairo_set_source_color (cr, &widget->style->base[GTK_STATE_NORMAL]);
  cairo_rectangle (cr,
                   0, 0,
                   widget->allocation.width, widget->allocation.height);
  cairo_fill (cr);

  gdk_cairo_set_source_color (cr, &widget->style->text[GTK_STATE_NORMAL]);
  cairo_rectangle (cr,
                   0, 0,
                   widget->allocation.width - 1, widget->allocation.height - 1);
  cairo_fill (cr);

  return FALSE;
}

One of the two instances of the color in the  new routine ought to be
changed to white or something, but I am not sure how that can be done.
(In the old code, the color is not directly specified, but the two are
complementary colors (base_gc[GTK_STYLE_NORMAL] vs
text_gc[GTK_STYLE_NORMAL])
and is probably tied to the proper background
color of default character drawing. It is not entirely clear
how the color in new API should be set. It is now
base[GTK_STYLE_NORMAL] vs text[GTK_STYLE_NORMAL]
I believe the use of same color parameter won't draw the border
properly, and if the color happens to be color of the character
drawing (black), the characters for indicating mode won't be readable.)

Again, this is one of those mozilla code base can't do a thing about it (right?). 

But for sharing this piece of information is very important for XIM input users of TB (and FF), thus I am posting this now. Once the gtk library maintainer fixes the problem, I think we are all the better.
Until that time, this bugzilla entry should be searchable by the irrate XIM users.

TIA

Comment 1

[ISHIKAWA, Chiaki]

I have filed a bug entry in gtk web site:
https://bugzilla.gnome.org/show_bug.cgi?id=696624

Again, this bug is not TB, FF per se.
These programs large number of external libraries and
TB, and FF are at the mercy of bugs introduced in them.

The problem here is that it is NOT ENTIRELY CLEAR to ordinary
users of TB and FF whether the problem(s) they suddenly face
are due to the external factors such as libraries, or not.

So naturally people hit with such a problem may try to find an entry in
mozilla bug entry. 

That is why I filed this bugzilla entry and hope this will be opened until 
GTK organization properly fixes the issue and the fixed gtk2 library package
is distributed widely.


(I have uploaded the screen capture images to GTK
bugzilla entries to make it easy for people without XIM experience
what I am talking about.)

https://bug696624.bugzilla-attachments.gnome.org/attachment.cgi?id=239950
https://bug696624.bugzilla-attachments.gnome.org/attachment.cgi?id=239951

TIA

PS:
Again, this is NOT the bug of TB and/or FF per se.
It is a bug in the external library of GTK2+ and
that happens to affect the XIM input of East Asian languages such as Japanese
negatively.
I want to let the affected users know there is a fix for this problem and thus
filed this bugzilla entry.
TB and FF developers don't have to change their source code, but
have to wish and encourage gtk2+ source code be modified.

Comment 2

[Firefox Bug Husbandry Bot]

https://bugzilla.mozilla.org/show_bug.cgi?id=1552613

Comment 3

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Moving all open keyboard/IME handling bugs to DOM: UI Events & Focus Handling component.

Comment 4

[Wayne Mery (:wsmwk)]

Is this known to still be an issue?

Comment 5

[ISHIKAWA, Chiaki]

Wayne,

This issue is a bit tricky.
Someone who got stuck with gtk2 series for maintenance reasons, etc. really needs apply the patch referenced
in https://bugzilla.gnome.org/show_bug.cgi?id=696624.

I could patch GTK2 with the proposed patch and have used the modified version under linux myself.
https://bugzilla.gnome.org/show_bug.cgi?id=696624#c23
It is just that gnome did not seem to have officially released the newer version with the patch since
it was focusing on GTK3 and later. Tough luck.
GTK3 seems (?) to have been fixed with a patch mentioned in the same URL.

Unfortunately, situation got very interesting and worse for the XIM input method user like me.
The newer GTK4(?) seems to have dropped the support for XIM by default, and I have been
forced to create im-xim.so dynamic library [simply copied the binary from patched GTK2 series on my computer]
and insert it into an appropriate place on my computer.

I hope information in this bugzilla can remain on the web as an easy-to-reach bugfix/workaround information for this issue.
Well since it is NOT really a TB code bug, but rather a TB UX due to an issue in GTK2/4 library, and I bet 0.1% of Japanese linux users use XIM method such as based on kinput2 these days.
So. maybe I can copy the info here in my own blog and let this bugzilla close.

Oh wait, there used to be sizable number of Chinese users who used XIM method. But after more than 5 years, I am not sure how many Chinese and Korean users use XIM mechanism.