197081 - AIX: Preedit string is duplicated when committed

Status: RESOLVED FIXED

(Core :: Internationalization, defect)

Description

[Philip K. Warren]

With the patch for Bug 195944 applied, AIX creates a proper status window and
on-the-spot editing appears to work correctly. When testing input of Japanese
characters however, we noticed the following problem:

1) Open mozilla -editor in Ja_JP locale.
2) Switch to Hiragana Input Mode.
3) Switch to Romanized Input Mode.
4) Type in "kanji".
5) Convert input string.
6) Hit enter to commit string.
7) The editor now displays the Japanese characters for "kanji", but they are
duplicated. It is as if the user has input "kanjikanji".

This seems to be identical to the problem described in Bug 34517, but that was
fixed quite a long time ago.

Comment 1

[Philip K. Warren]

Attachment: First InsertTextImpl stack trace

This is the stack trace within InsertTextImpl when the conversion key is
pressed.

Comment 2

[Philip K. Warren]

Attachment: Second InsertTextImpl stack trace

This is the stack trace in InsertTextImpl when the preedit string is committed
with the enter key.

Comment 3

[Philip K. Warren]

This does not happen with GDK IM style GDK_IM_PREEDIT_POSITION |
GDK_IM_STATUS_AREA. When I perform these same actions in that IM style, the
preedit string is selected the entire time in the editor. I'll post some
screenshots to illustrate what is happening.

Comment 4

[Philip K. Warren]

Attachment: GDK_IM_PREEDIT_POSITION before commit

Comment 5

[Philip K. Warren]

Attachment: GDK_IM_PREEDIT_POSITION after commit

Comment 6

[Philip K. Warren]

Attachment: GDK_IM_PREEDIT_CALLBACKS before commit

Comment 7

[Philip K. Warren]

Attachment: GDK_IM_PREEDIT_CALLBACKS after commit

Comment 8

[Philip K. Warren]

I haven't made much progress on this bug, but I have been comparing what occurs
on Mozilla in Linux versus what occurs on AIX, and here are my findings:

Linux (input of "kanji" in the editor):

preedit_start_cbproc

preedit_draw_cbproc                                       <-- "k" typed
InsertTextImpl(aStringToInsert=k, offset=0)

preedit_draw_cbproc                                       <-- "a" typed
InsertTextImpl(aStringToInsert=か, offset=1)

preedit_draw_cbproc                                       <-- "n" typed
InsertTextImpl(aStringToInsert=かn, offset=1)

preedit_draw_cbproc                                       <-- "j" typed
InsertTextImpl(aStringToInsert=かんj, offset=2)

preedit_draw_cbproc                                       <-- "i" typed
InsertTextImpl(aStringToInsert=かんじ, offset=3)

preedit_draw_cbproc                                       <-- "space" typed (for
conversion)
InsertTextImpl(aStringToInsert=漢字, offset=3)

preedit_draw_cbproc                                       <-- "enter" typed
InsertTextImpl(aStringToInsert=, offset=2)
InsertTextImpl(aStringToInsert=漢字, offset=0)



AIX (input of "kanji" in the editor):

preedit_start_cbproc

preedit_draw_cbproc                                       <-- "ka" typed
InsertTextImpl(aStringToInsert=か, offset=0)

preedit_draw_cbproc                                       <-- "nj" typed
InsertTextImpl(aStringToInsert=かん, offset=1)

preedit_draw_cbproc                                       <-- "i" typed
InsertTextImpl(aStringToInsert=かんじ, offset=2)

preedit_draw_cbproc                                       <-- "F4" typed (for
conversion)
InsertTextImpl(aStringToInsert=漢字, offset=3)

preedit_done_cbproc                                       <-- "Enter" typed
InsertTextImpl(aStringToInsert=漢字, offset=2)

preedit_done_cbproc
preedit_done_cbproc



AIX calls the preedit done callback when the enter key is typed, while Linux
submits an empty string and then the converted text when the enter key is
pressed. I have stepped through this code in the debugger, and on Linux one can
clearly see the preedit string being deleted when InsertTextImpl is called with
an empty string and then the converted string is added, leaving the two of the
Japanese characters for "Kanji". On AIX, the empty string is never submitted, so
the preedit string remains and the composed string is appended leaving four
characters. In addition, AIX sends the preedit done callback several more times
than the Linux version. (On Linux, the preedit done callback is only triggered
when the input mode is switched back to English and the status window disappears.

Comment 9

[Masaki Katakai]

Your evaluation is correct,

IME needs to delete the preedit before commit,

preedit_draw_cbproc                                       <-- "enter" typed
InsertTextImpl(aStringToInsert=, offset=2)          <== erase from IME
InsertTextImpl(aStringToInsert=xxxxx, offset=0)    <== this comes from commit

To erase the preedit, AIX seems to send preedit_done_cbproc
instead. What will happen when you call below at predit_done_cbproc?

4238         IMEComposeStart(nsnull);
4239         IMEComposeText(nsnull, nsnull, 0, nsnull);
4240         IMEComposeEnd(nsnull);

Comment 10

[Philip K. Warren]

Attachment: Patch v1

I have tested this patch and it is now working properly for AIX. I am able to
input Japanese using the AIX input methods without any trouble.

Comment 11

[Philip K. Warren]

Taking ownership of bug.

Comment 12

[Masaki Katakai]

Philip, please confirm that the following can work with the patch,

- try to disable IME while preedit string exists
  1. shift+space to turn IME on
  2. type something
     now preedit string are displayed with red underline
  3. shift+space to turn IME off

I understand that the behavior depends on IME.
If the behavior is correct for you, I'll give r= for the patch.

Comment 13

[Philip K. Warren]

This patch needs further testing. I would like to get other patches (Bug 195944
and Bug 196898) fixed before 1.4 alpha however.

Comment 14

[Philip K. Warren]

I have tested this patch in both the default AIX IM and the Wnn7 IM. The Wnn7 IM
uses Ctrl+Space to enable IM, and Ctrl-Space to disable it. With this patch,
when you enable IM, enter some characters, and then disable IM, the preedit
string disappears. However, when you re-enable IM, the old preedit string is
saved. This seems to be the correct behavior (I see the same behavior in aixterm
and dtterm).

With the AIX IM, it is not possible to Enable/Disable IM with Ctrl+Space or any
other keyboard combinations.

I am confident that this is the correct patch for AIX.

Comment 15

[Masaki Katakai]

More questions,

Will preedit_start_cbproc() be called again after you commit string?
   (do not disable XIM here)

   1. turn IME on
   2. type something
   3. return key to commit
       preedit_done_cbproc() is called
       is any preedit_draw_cbproc() called?
       If preedit_draw_cbproc() is never called, mPreedit keeps
       the preedit string? should we call mPreedit->Reset() to
       reset the preedit??
   4. type something
       preedit should be displayed, but is preedit_start_cbproc() called?

I'd like to confirm preedit is reset at commit.
nsIMEPreedit::Reset() needs to be called if preedit_start_cbproc()
is never called.

> With the AIX IM, it is not possible to Enable/Disable IM with Ctrl+Space or any
> other keyboard combinations.

Could you tell me how to enable/disable IME on AIM?

Comment 16

[Philip K. Warren]

Here is what occurs when text is typed:

preedit_start_cbproc - Called before first preedit_draw
preedit_draw_cbproc  - "ka" typed
preedit_draw_cbproc  - "nj" typed
preedit_draw_cbproc  - "i" typed
preedit_draw_cbproc  - convert key typed
preedit_done_cbproc  - enter typed (preedit string committed)

preedit_start_cbproc - Called again before typing second string
preedit_draw_cbproc  - "ka" typed
preedit_draw_cbproc  - "nj" typed
preedit_draw_cbproc  - "i" typed
preedit_draw_cbproc  - convert key typed
preedit_done_cbproc  - enter typed (second preedit string committed)

So preedit_start_cbproc is indeed called before the preedit string is drawn, so
the preedit string is reset properly.

To use the default AIX Japanese IM, you need either a Japanese keyboard or you
need to use xmodmap to change some default settings for an English keyboard.
Here are the xmodmap settings I use on an English keyboard to test:

! Keypad 7
keycode 79 = Hiragana Katakana
! Keypad 8
keycode 80 = Eisu_toggle
! Keypad 9
keycode 81 = Zenkaku_Hankaku
! Keypad 4
keycode 83 = Henkan MaeKouho
! Keypad 5
keycode 84 = Muhenkan

Then, to enable the AIX IM I type the '7' key on the keypad to switch to
Hiragana input mode. Then Alt+7 changes to Romanized input mode. '4' is the
conversion key, and '5' is the non-convert key. I am unaware of any way to
disable the AIX Japanese IM once it has been enabled.

Comment 17

[Philip K. Warren]

Fixed.

Checking in nsWindow.cpp;
/cvsroot/mozilla/widget/src/gtk/nsWindow.cpp,v  <--  nsWindow.cpp
new revision: 1.413; previous revision: 1.412
done