Open Bug 1077411 Opened 10 years ago Updated 2 years ago

Combining Accents do not work with KeyboardEvent on Linux

Categories

(Core :: DOM: Events, defect, P5)

Product:
Core
Component:
DOM: Events
Version:
32 Branch
Platform:
x86_64
Windows 7
Type:
defect

Tracking

()

Status:
UNCONFIRMED

People

(Reporter: mozilla, Unassigned)

References

(
URL
)

Details

(Keywords: testcase)

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:32.0) Gecko/20100101 Firefox/32.0 Build ID: 20140923175406 Steps to reproduce: When using Firefox on Linux (reproduced with versions 26-32), dead keys are not handled correctly in KeyboardEvents. if I press ´ followed by e, I expect to get a keypress event for the character é. Instead, I get a keypress for ` followed by a keypress for e, with no indication that the two should be combined. To reproduce: Listen to the 'onkeypress' event with a JavaScript snippet such as this: Use a simple test page such as this: <!DOCTYPE html> <html <head> <script> window.onkeypress = function(ev) { console.info("key press: ", ev.key, ev.char, ev.charCode); } </script> </head> <body> <div id='text' style="width: 500px; height: 500px; background-color: blue;"/> </body> </html> Now open the page in Firefox, and press the following keys: ´ (acute accent) e Actual results: The following output is printed to the console: "key press: " "DeadAcute" 0 "key press: " "e" 101 Expected results: No keypress event should have been generated for the accent (it is a dead key). Instead, the accent should have been combined with the subsequent e character, producing an é. In the console, the following output is expected: "key press: " "é" 233 On Windows, the correct output is generated. The problem can only be reproduced on Linux.
I'm not sure I agree with this request. Composition is a high-level event and should not be performed on such a low level as keypress events. I'd prefer to get the non-composed character. An API to _also_ get the composed stuff would be nice though.
Perhaps the severity of this bug/feature_request is not being understood properly: This issue affects everyone who tries to connect to novnc from a linux machine. How many hundreds of thousands of noVnc installs are out there? most compute cloud providers use it to provide console access to vps instances. Now, imagine a sysadmin on a linux desktop needing a redmond version of the same firefox browser to simply get a remote console. For many, it means keeping a redmond vm image handy. The mere idea of having a different platform to run the same browser on is erroneous. This has been an issue for quite a long time, here is a reference to noVnc bug report https://github.com/novnc/noVNC/issues/350 Why not fix it? It would save so many hours of frustration to a very large audience of users, not to mention all those ill wishes directed towards mozilla developers, everytime someone spins up a vm to run a browser in. Plain good karma all round.
Just a comment about novnc: if you're using novnc to connect to a QEMU/KVM VM that has support for the QEMU keyboard extension, you will not face the problems described here because the code doesn't rely on the key composition mechanism of the browser. This was developed in https://github.com/novnc/noVNC/issues/21. I have suggested changing #350 to CLOSED too.
It's not a solution for every case, so it would still be helpful for the browser to expose combining characters as events.
Priority: -- → P5

Is anyone looking at all at this? Is it something that has been discussed in whatwg? I would guess this needs some standardisation.

In an effort to get some collaboration I've also informed the other parties of this deficiency:

https://bugs.chromium.org/p/chromium/issues/detail?id=1135952
https://github.com/whatwg/dom/issues/900

This bug is also affecting text input in WASM-based games and other app which, for portability reasons, do not use DOM but implement their own UI, typically on top of WebGL.

Regarding the links above, I think that there are two different issues:

  • One is that the keypress event does not contain the accentuated composed character, Firefox on Linux has this problem but not on Windows and Chrome works everywhere.
  • The other is that when a dead key is sent, the associated character is not mentioned, that is the issue linked above that seems to affect everyone.

In my environment (Ubuntu 21.04), dead keys are handled as composition of IME. So no keypress event is fired. Which environment can reproduce this?

Composition events don't seem to solve this unfortunately. From the whatwg issue:

The standard currently refers to using composition events, but they are not a replacement as they only give you the current composed state, not the symbol that was just included to update the state.

Composition events also only work on input fields, not other keyboard event targets such as a canvas. So even if they did contain the proper information, they would still not be a valid replacement.

I'm not sure why you aren't seeing keypress events though. Are you using a more "full" IME, e.g. for Japanese? I've been testing with no IME (on Linux) and any western layout with dead keys (e.g. US International or Swedish).

Ah, is this a case when no editor has focus? If so, I reproduce the symptom.

And the KeyboardEvent.key values are set as you expected only when IME works. Otherwise, the dead key state is not managed by Linux (meaning GTK nor X). Therefore, only the raw characters are exposed to the web when you type base character (a, e, etc). Additionally, there is no way to get composed character at least via GDK. Therefore, it seems that it's impossible to expose composed character via KeyboardEvent.key with dead keys.

I don't suppose you can get the raw X event somewhere? What we (noVNC) would ideally like is something like XK_dead_acute to "DeadAcute" or similar. I'm afraid I'm not too familiar with GDK's event handling.

And base X11, without any IME, has some basic IME-like behaviour for the dead keys. It hides the real event using XFilterEvent() and then injects a fake new event with the raw keysym swapped out for the combined on (and also a keycode of 0). E.g.:

KeyPress event, serial 37, synthetic NO, window 0x3200001,
    root 0x754, subw 0x0, time 1206484575, (408,-145), root:(1514,153),
    state 0x10, keycode 21 (keysym 0xfe51, dead_acute), same_screen YES,
    XLookupString gives 2 bytes: (c2 b4) "´"
    XmbLookupString gives 0 bytes: 
    XFilterEvent returns: True

KeyRelease event, serial 37, synthetic NO, window 0x3200001,
    root 0x754, subw 0x0, time 1206484647, (408,-145), root:(1514,153),
    state 0x10, keycode 21 (keysym 0xfe51, dead_acute), same_screen YES,
    XLookupString gives 2 bytes: (c2 b4) "´"
    XFilterEvent returns: False

KeyPress event, serial 37, synthetic NO, window 0x3200001,
    root 0x754, subw 0x0, time 1206485039, (408,-145), root:(1514,153),
    state 0x10, keycode 38 (keysym 0x61, a), same_screen YES,
    XLookupString gives 1 bytes: (61) "a"
    XmbLookupString gives 1 bytes: (61) "a"
    XFilterEvent returns: True

KeyPress event, serial 37, synthetic NO, window 0x3200001,
    root 0x754, subw 0x0, time 1206485039, (408,-145), root:(1514,153),
    state 0x10, keycode 0 (keysym 0xe1, aacute), same_screen YES,
    XKeysymToKeycode returns keycode: 38
    XLookupString gives 0 bytes: 
    XmbLookupString gives 2 bytes: (c3 a1) "á"
    XFilterEvent returns: False

KeyRelease event, serial 37, synthetic NO, window 0x3200001,
    root 0x754, subw 0x0, time 1206485135, (408,-145), root:(1514,153),
    state 0x10, keycode 38 (keysym 0x61, a), same_screen YES,
    XLookupString gives 1 bytes: (61) "a"
    XFilterEvent returns: False

On Windows things get much messier as they don't seem to have a standardised way of doing things. For reference, this is the mess we've had to come up with for TigerVNC:

https://github.com/TigerVNC/tigervnc/blob/468687f5d7f036b3e78e767b32d3b9b14a4b3b17/vncviewer/win32.c#L349
https://github.com/TigerVNC/tigervnc/blob/468687f5d7f036b3e78e767b32d3b9b14a4b3b17/vncviewer/keysym2ucs.c#L841

TL;DR; We repeat the dead key until we get a character back from Windows. We then have a lookup table where we can map that key back to the expected Unicode combining character, and then from that to the X11 keysym (that VNC needs).

I/KeymapWrapperWidgets HandleKeyPressEvent(aWindow=7f4ac1d86800, aGdkKeyEvent={ type=GDK_KEY_PRESS, keyval=dead_grave(0xFE50), state=0x00002010, hardware_keycode=0x00000022, time=4515613, is_modifier=FALSE })
D/KeymapWrapperWidgets 7f4ac1d24a60 InitInputEvent, aModifierState=0x00002010, aInputEvent={ mMessage=eKeyDown, mModifiers=0x0080 (Shift: FALSE, Control: FALSE, Alt: FALSE, Meta: FALSE, OS: FALSE, AltGr: FALSE, CapsLock: FALSE, NumLock: TRUE, ScrollLock: FALSE })
I/KeymapWrapperWidgets 7f4ac1d24a60 InitKeyEvent, modifierState=0x00002010 aKeyEvent={ mMessage=eKeyDown, isShift=FALSE, isControl=FALSE, isAlt=FALSE, isMeta=FALSE , mKeyCode=0x00, mCharCode=NULL (0x0000), mKeyNameIndex=Dead, mKeyValue="", mCodeNameIndex=BracketLeft, mCodeValue="", mLocation=KEY_LOCATION_STANDARD, mIsRepeat=FALSE }
I/KeymapWrapperWidgets   HandleKeyPressEvent(), dispatched eKeyDown event and it wasn't consumed
D/KeymapWrapperWidgets 7f4ac1d24a60 InitInputEvent, aModifierState=0x00002010, aInputEvent={ mMessage=eKeyPress, mModifiers=0x0080 (Shift: FALSE, Control: FALSE, Alt: FALSE, Meta: FALSE, OS: FALSE, AltGr: FALSE, CapsLock: FALSE, NumLock: TRUE, ScrollLock: FALSE })
I/KeymapWrapperWidgets 7f4ac1d24a60 InitKeyEvent, modifierState=0x00002010 aKeyEvent={ mMessage=eKeyPress, isShift=FALSE, isControl=FALSE, isAlt=FALSE, isMeta=FALSE , mKeyCode=0x00, mCharCode=NULL (0x0000), mKeyNameIndex=Dead, mKeyValue="", mCodeNameIndex=BracketLeft, mCodeValue="", mLocation=KEY_LOCATION_STANDARD, mIsRepeat=FALSE }
I/KeymapWrapperWidgets   HandleKeyPressEvent(), didn't dispatch eKeyPress event (status=nsEventStatus_eIgnore)
<snip>
I/KeymapWrapperWidgets FilterEvents(aXEvent={ type=KeyPress, xkey={ keycode=0x00000026, state=0x00002010, time=4518957 } }, aGdkEvent={ state=0x00000000 }), detected first keypress
I/KeymapWrapperWidgets HandleKeyPressEvent(aWindow=7f4ac1d86800, aGdkKeyEvent={ type=GDK_KEY_PRESS, keyval=a(0x61), state=0x00002010, hardware_keycode=0x00000026, time=4518957, is_modifier=FALSE })
D/KeymapWrapperWidgets 7f4ac1d24a60 InitInputEvent, aModifierState=0x00002010, aInputEvent={ mMessage=eKeyDown, mModifiers=0x0080 (Shift: FALSE, Control: FALSE, Alt: FALSE, Meta: FALSE, OS: FALSE, AltGr: FALSE, CapsLock: FALSE, NumLock: TRUE, ScrollLock: FALSE })
I/KeymapWrapperWidgets 7f4ac1d24a60 InitKeyEvent, modifierState=0x00002010 aKeyEvent={ mMessage=eKeyDown, isShift=FALSE, isControl=FALSE, isAlt=FALSE, isMeta=FALSE , mKeyCode=0x41, mCharCode=NULL (0x0000), mKeyNameIndex=USE_STRING, mKeyValue="'a' (0x0061)", mCodeNameIndex=KeyA, mCodeValue="", mLocation=KEY_LOCATION_STANDARD, mIsRepeat=FALSE }
I/KeymapWrapperWidgets   HandleKeyPressEvent(), dispatched eKeyDown event and it wasn't consumed
D/KeymapWrapperWidgets 7f4ac1d24a60 InitInputEvent, aModifierState=0x00002010, aInputEvent={ mMessage=eKeyPress, mModifiers=0x0080 (Shift: FALSE, Control: FALSE, Alt: FALSE, Meta: FALSE, OS: FALSE, AltGr: FALSE, CapsLock: FALSE, NumLock: TRUE, ScrollLock: FALSE })
I/KeymapWrapperWidgets 7f4ac1d24a60 InitKeyEvent, modifierState=0x00002010 aKeyEvent={ mMessage=eKeyPress, isShift=FALSE, isControl=FALSE, isAlt=FALSE, isMeta=FALSE , mKeyCode=0x00, mCharCode=NULL (0x0000), mKeyNameIndex=USE_STRING, mKeyValue="'a' (0x0061)", mCodeNameIndex=KeyA, mCodeValue="", mLocation=KEY_LOCATION_STANDARD, mIsRepeat=FALSE }
I/KeymapWrapperWidgets   HandleKeyPressEvent(), dispatched eKeyPress event (status=nsEventStatus_eIgnore)

Unfortunately, there is no last KeyPress event of your log in GDK level...

I think we might be misunderstanding each other here, since that looks like precisely what we want. :)

I/KeymapWrapperWidgets HandleKeyPressEvent(aWindow=7f4ac1d86800, aGdkKeyEvent={ type=GDK_KEY_PRESS, keyval=dead_grave(0xFE50), state=0x00002010, hardware_keycode=0x00000022, time=4515613, is_modifier=FALSE })

I'd like the keyval and hardware_keycode values here exposed in JavaScript.

I/KeymapWrapperWidgets 7f4ac1d24a60 InitKeyEvent, modifierState=0x00002010 aKeyEvent={ mMessage=eKeyDown, isShift=FALSE, isControl=FALSE, isAlt=FALSE, isMeta=FALSE , mKeyCode=0x00, mCharCode=NULL (0x0000), mKeyNameIndex=Dead, mKeyValue="", mCodeNameIndex=BracketLeft, mCodeValue="", mLocation=KEY_LOCATION_STANDARD, mIsRepeat=FALSE }

The problem is that keyval gets translated to the ambiguous "Dead" here.

I/KeymapWrapperWidgets HandleKeyPressEvent(aWindow=7f4ac1d86800, aGdkKeyEvent={ type=GDK_KEY_PRESS, keyval=a(0x61), state=0x00002010, hardware_keycode=0x00000026, time=4518957, is_modifier=FALSE })

Same thing here. I'd like keyval and hardware_keycode values to propagate to JavaScript since this event contains the uncomposed "a" rather than the composed "á".

The strange thing about your log is that the composed event is missing. So I wonder if we are looking at the right thing...

Can I enable that debug output easily and try here?

Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.