630576 - Fennec should send key events to content even without input focus

Status: RESOLVED INCOMPLETE

(Firefox for Android Graveyard :: Keyboards and IME, defect, P2)

Description

[Wesley Johnston (:wesj)]

Spinning off from 614202. Key events traveling down the IME code paths aren't firing key events in either parent or child processes. Since we don't get the key events on Android, we should at least look at the IME ones and try our best to determine what was pressed in order to simulate the correct key events.

Comment 1

[Matt Brubeck (:mbrubeck)]

Nominating blocking because this might break Google Docs (bug 624163).

Comment 3

[Alex Pakhotin (:alexp)]

I'm trying to understand, what we really want and can do here.

What is the main purpose of this bug? Fix Google Docs? Which other sites require this change?

Characters entered through an IME do not register through keyCode or charCode - this is how Gecko DOM is designed, as far as I understand. Are we going to replace IME events with key press events? This doesn't sound right. What are the other options?

Can you guys share your thoughts on this please?
Thanks.

Comment 4

[Matt Brubeck (:mbrubeck)]

Un-nominating as blocking, and moving the Google-Docs-specific discussion back to bug 624163.

Comment 7

[Marvin Roman]

As suggested in a separate thread that was marked duplicate: "oninput" is buggy too and only starts propagating after pressing enter or delete and then works as designed.

Comment 9

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

For information it looks like Webkit is firing IME/key events simultaneously.

https://bugs.webkit.org/show_bug.cgi?id=43020
http://joone4u.blogspot.com/2010/07/composition-events-are-handled_27.html

Comment 11

[Martijn Wargers (dead)]

Attachment: testcase

Comment 12

[Mark Finkle (:mfinkle) (use needinfo?)]

Alex - What's the status of this bug? Why would we want to track this issue? (Martijn)

Comment 13

[Martijn Wargers (dead)]

It's causing sites like ebay and google autocomplete to not work (see duped bugs).

Comment 14

[Alex Pakhotin (:alexp)]

(In reply to Mark Finkle (:mfinkle) from comment #12)
> Alex - What's the status of this bug?

The issue was discussed a while ago, but as far as I remember there was no final decision on what to do with it. The other issues, which seemed to be depending on this one were fixed some other way.

Apparently there are still cases where these key events are required, so I guess it needs one more approach. I'll try to unstuck it while looking into my queue of IME bugs.

Comment 15

[Wesley Johnston (:wesj)]

I saw we recently checked in bug 543789 which also should provide a workaround (compositionupdate events) for websites running into this problem.

Comment 16

[Makoto Kato [:m_kato]]

If commitText generates key message by sendKeyEvent(), this may be resolved.

BaseInputConnection.commitText on Android framework uses same idea w/ fullEditor=false.

Comment 17

[Alex Pakhotin (:alexp)]

Some very useful information from Masayuki Nakano:

I guess that you want to send key events for compatibility with some web pages which are handling only key events, right?

If so, we MUST not dispatch keypress event during composition.  It has banned by D3E.
http://dev.w3.org/2006/webapi/DOM-Level-3-Events/html/DOM3-Events.html#event-type-keypress

The reason is that keypress event indicates one or more characters are inputted by a key. So, if you have dispatched composition events already, it's false.

If web developers wants to catch updating the composition string of IME, they should use compositionupdate event which was implemented on Mozilla9 and later.
https://bugzilla.mozilla.org/show_bug.cgi?id=543789

On the other hand, there are keydown and keyup events. They should be dispatched during composition too (by D3E). However, unfortunately, we're not dispatching them during composition because some internal code is broken by doing that. I think that most handlers are NOT thinking about composition. For example, arrow keys move caret outside of the composing string. This is a major problem which blocks implementing TSF on Windows. If you want to send these events during composition too, you must check all keydown/keyup handlers of our tree. This is in my job queue, but the priority isn't high.

However, there is an exception. If IME commits a character immediately after first keypress, you can dispatch keypress event *instead* of compositionstart, compositionupdate, compositionend and text events. GTK2 widget is dosing so.
http://mxr.mozilla.org/mozilla-central/source/widget/src/gtk2/nsGtkIMModule.cpp?mark=403-405,413-413,444-444,1002-1002,1011-1011,1015-1015#369
http://mxr.mozilla.org/mozilla-central/source/widget/src/gtk2/nsGtkIMModule.cpp?mark=403-405,413-413,444-444,1002-1002,1011-1011,1015-1015#968

# Calling gtk_im_context_filter_keypress() causes nsGtkIMModule::OnCommitCompositionNative().

There is another event, textinput. This is fired immediately before editing text by keypress, compositionend, d&d, paste, script (e.g., element.value = "foo") and other reasons. D3E recommends that web developers should use this event rather than keypress event. I'll implement it Mozilla11 or 12 or later.
https://bugzilla.mozilla.org/show_bug.cgi?id=622245

FYI: We'll kill support inputting text via untrusted keypress event. You should know this bug.
https://bugzilla.mozilla.org/show_bug.cgi?id=698949

Comment 18

[Alex Pakhotin (:alexp)]

Attachment: WIP Patch

Some work in progress.
The key-down/up events are generated, but they clash with IME text input events, so there are still issues to be fixed: a single key press needs to be detected better and we have to make sure the IME composition is not screwed up.

Comment 20

[Alex Pakhotin (:alexp)]

If I understand correctly, a proper fix to this depends on the changes in many other parts of Firefox code.

Comment 21

[Brad Lassey [:blassey] (use needinfo?)]

(In reply to Alex Pakhotin (:alexp) from comment #20)
> If I understand correctly, a proper fix to this depends on the changes in
> many other parts of Firefox code.

are bugs on file?

Comment 22

[Chris Peterson [:cpeterson]]

For comparison, here are test results for different platforms:

* Firefox, Chrome, and Safari on Mac OS X 10.7:
keydown
keypress
input
keyup

* Stock browser on the Galaxy Nexus, Galaxy S2, and Droid Pro:
keydown
keypress
input
keyup

* Chrome on the Galaxy Nexus:
keydown
input
keyup

* Fennec on the Galaxy Nexus:
input

* Fennec on the Samsung S2:
input
input

* Fennec on the Droid Pro:
input
keyup

Comment 23

[Chris Peterson [:cpeterson]]

* And Fennec on the Kindle Fire:
input

Comment 24

[Brad Lassey [:blassey] (use needinfo?)]

(In reply to Chris Peterson (:cpeterson) from comment #22)
> * Fennec on the Samsung S2:
> input
> input

this is bug 735806

Comment 25

[Chris Peterson [:cpeterson]]

This bug should just address the key events when an input form is not focused (such as gmail keyboard shortcuts)

I split off bug 737658 to address the key down/press/up event sequence when we have input focus.

Comment 26

[Chris Peterson [:cpeterson]]

gmail keyboard shortcuts do work correctly. However, Google serves Fennec a basic mobile version of gmail that does not use keyboard shortcuts.

If you install the Phony addon, switch your User-Agent String to "Android (Nexus One)", and reload gmail, you will get a more advanced mobile version of gmail. If then switch your User-Agent String back to "Default", gmail will continue to serve you the advanced mobile version and the J/K keyboard shortcuts will work correctly.

We might want to contact Google and ask them to check for Fennec's (new) User-Agent String and serve us the advanced mobile version of gmail.

I tested with my Droid Pro's HKB.

Comment 27

[Brad Lassey [:blassey] (use needinfo?)]

really nice to have, but won't hold the release

Comment 28

[Chris Peterson [:cpeterson]]

Removed some unrelated bugs from the blocking list.

Blocked bug 686360 (JSLinux) is probably the best test case for this bug.

Comment 29

[Chris Peterson [:cpeterson]]

I'm not sure what to do with this bug. It's a vague meta bug from XUL Fennec that has acquired many "could be related to this" dependent bugs over the past 17 months.

I recommended that we retire this meta bug and retest the dependent bugs individually.

Comment 30

[Mark Finkle (:mfinkle) (use needinfo?)]

Let's file new bugs for specific cases.