1132380 - [Stingray] Inconsistent behaviors when using MozInputContext.sendKey

Status: RESOLVED DUPLICATE of bug 1110030

(Core :: DOM: Events, defect)

Description

[Gina Yeh [:gyeh] [:ginayeh]]

When we use MozInputContext.sendKey() in main process (b2g), before-key events and after-key events are dispatched to the parents of the event target, and key events is dispatched to the event target itself. However, when we use the API in an OOP App, only key events are dispatched to the event target. No before-key events and after key-events are received by its parents.

Comment 1

[Gina Yeh [:gyeh] [:ginayeh]]

In the implementation of MozInputContext.sendKey(), we synthesize a key by sending "keydown", "keypress"(optional), and "keyup" events via nsIDOMWindowUtils.sendKeyEvent().

dom/inputmethod/forms.js
> let doKeypress = domWindowUtils.sendKeyEvent('keydown', json.keyCode,
>                                              json.charCode, json.modifiers, flags);
> if (doKeypress) {
>   domWindowUtils.sendKeyEvent('keypress', json.keyCode,
>                               json.charCode, json.modifiers, flags);
> }
>
> if(!json.repeat) {
>   domWindowUtils.sendKeyEvent('keyup', json.keyCode,
>                               json.charCode, json.modifiers, flags);
> }

Then, we dispatch these events to the corresponding widget.

dom/base/nsDOMWindowUtils.cpp
> WidgetKeyboardEvent event(true, msg, widget);
> event.modifiers = GetWidgetModifiers(aModifiers);

> nsEventStatus status;
> nsresult rv = widget->DispatchEvent(&event, status);
> NS_ENSURE_SUCCESS(rv, rv);

When an application is launched in process, GetWidget() returns an instance of nsWindow of on Gonk platform. Then, the same flow are triggered as if we press an actual key.

/widget/gonk/nsWindow.cpp
> NS_IMETHODIMP
> nsWindow::DispatchEvent(WidgetGUIEvent* aEvent, nsEventStatus& aStatus)
> {
>   if (mWidgetListener)
>     aStatus = mWidgetListener->HandleEvent(aEvent, mUseAttachedEvents);
>   return NS_OK;
> }

However, when an application is launched out of process, GetWidget() returns an instance of PuppetWidget. Then, the event is handled by nsViewManager and is dispatched by nsPresShell in the child process.

widget/PuppetWidget.cpp
> NS_IMETHODIMP
> PuppetWidget::DispatchEvent(WidgetGUIEvent* event, nsEventStatus& aStatus)
> {
>   ...
>   if (mAttachedWidgetListener) {
>     aStatus = mAttachedWidgetListener->HandleEvent(event, mUseAttachedEvents);
>   }
>   ...
> }

As a result, parent process isn't aware that there's a new key event, and neither mozbrowserbeforekey* events nor mozbrowserafterkey* events are dispatched to the parent of the event target.

Comment 2

[Gina Yeh [:gyeh] [:ginayeh]]

I'm not sure if we should regard the key generated by software keyboard as a native key. If so, we should also dispatch mozbrowser* key events. Masayuki, any thoughts?

Comment 3

[Gina Yeh [:gyeh] [:ginayeh]]

Since Yuan Xulei is working on IME API, could you provide some suggestions? Thanks.

Comment 4

[kubota]

Is it possible to be modified to throw an event to directly input element without the MozInputContext.sendKey()?
If possible, or you will be able to create a patch on the Mozilla side?
If not possible, is there a way to check the difference between "software keyboard events" and "USB(standard) keyboard events" in the system app?

Comment 5

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

(In reply to Gina Yeh [:gyeh] [:ginayeh] from comment #2)
> I'm not sure if we should regard the key generated by software keyboard as a
> native key. If so, we should also dispatch mozbrowser* key events. Masayuki,
> any thoughts?

Basically, virtual keyboard is just a keyboard device implemented by software. So, it should behave same as physical keyboard (except KeyboardEvent.code if the virtual keyboard isn't emulating physical keyboard, e.g., a key can input some characters with swipe).

Comment 6

[Yuan Xulei [:yxl] (Leave from May 18)]

(In reply to Gina Yeh [:gyeh] [:ginayeh] from comment #3)
> Since Yuan Xulei is working on IME API, could you provide some suggestions?
> Thanks.

When implementing the IME API, we didn't take into consideration mozbrowserbeforekey* events nor mozbrowserafterkey.
I don't know the impact of regarding the key events of software keyboard as native ones.
cc Tim, he may know.

Comment 7

[Tim Guan-tin Chien [:timdream] (please needinfo)]

(In reply to Masayuki Nakano (:masayuki) (Mozilla Japan) from comment #5)
> Basically, virtual keyboard is just a keyboard device implemented by
> software. So, it should behave same as physical keyboard (except
> KeyboardEvent.code if the virtual keyboard isn't emulating physical
> keyboard, e.g., a key can input some characters with swipe).

I understand the analogy here but I don't see the use cases. Should a virtual keyboard app allow to implement a key, say volume up/down and ended up changing the system volume? There are also extra complexity here if we decided to generate before-key and after-key events with sendKey() method but we also rely on the same method to complete [1] (fyr the flow chart there).

So if Masayuki understand my point and agrees, I proposed do not emit before-key and after-key events with sendKey() calls, whether it's inproc or OOP. Thanks for catching the inconsistency BTW.

[1] https://wiki.mozilla.org/WebAPI/InputMethod_API_with_hardware_keyboard

Comment 8

[C.M.Chang[:chunmin]]

This bug will be fixed in bug 1110030