950832 - OSK not appearing on the first tap when filling in input fields

Status: VERIFIED FIXED

(Firefox for Metro Graveyard :: Input, defect, P2)

Description

[Karen Rudnitski [:kar]]

Possibly related to bug 944215?

On my surface pro 2 working as a tablet (keyboard folded back). When inputting into input boxes, takes two taps before the OSK shows up. (The first tap you see the cursor in the form box like filling out delivery / shopping information), but you have to tap a second time in the field to bring up the OSK. Really annoying when you're trying to buy something.....

Comment 1

[Jim Mathies [:jimm]]

This regressed due to bug 941324. The apzc takes too long to send tap events, so the dom isn't up to date when windows comes in looking for focus info - 

mozilla::widget::winrt::UIABridge::get_ProviderOptions
UIABridge::GetPropertyValue: idProp=UIA_NativeWindowHandlePropertyId
mozilla::widget::winrt::UIABridge::Navigate
UIABridge::Navigate NavigateDirection_Parent
mozilla::widget::winrt::UIABridge::get_ProviderOptions
mozilla::widget::winrt::UIABridge::get_ProviderOptions
mozilla::widget::winrt::UIABridge::Navigate
UIABridge::Navigate NavigateDirection_Parent
mozilla::widget::winrt::UIABridge::GetFocus
* ^ Windows asking for focus information after input is delivered
Focus element flags: editable:0 focusable:1 readonly:1
* ^ Focus element not focusable
mozilla::widget::winrt::UIATextElement::ClearFocus
UIABridge::GetPropertyValue: idProp=UIA_IsControlElementPropertyId
UIABridge::GetPropertyValue: idProp=UIA_NativeWindowHandlePropertyId
mozilla::widget::winrt::UIABridge::get_BoundingRectangle
UIABridge::GetPropertyValue: idProp=UIA_ControlTypePropertyId
UIABridge::GetPropertyValue: idProp=49209
UIABridge: Unhandled property
UIABridge::GetPropertyValue: idProp=49213
UIABridge: Unhandled property
mozilla::widget::winrt::UIABridge::GetPatternProvider
UIABridge::GetPatternProvider=10014
mozilla::widget::winrt::UIABridge::GetPatternProvider
UIABridge::GetPatternProvider=10029
UIABridge::GetPropertyValue: idProp=UIA_IsPasswordPropertyId
Content.js::Gesture:SingleTap 
* ^ single tap delivered to content
### SelectionPrototype.js loaded
mozilla::widget::winrt::UIABridge::FocusChangeEvent
Focus element flags: editable:1 focusable:1 readonly:0

Comment 2

[Jim Mathies [:jimm]]

Commenting out this return fixes the problem.

http://mxr.mozilla.org/mozilla-central/source/widget/windows/winrt/MetroInput.cpp#964

Matt, any ideas? I don't remember why we switched to the apzc for taps in content.

This will be interesting to solve with content processes work later on.

Comment 3

[Jim Mathies [:jimm]]

Maybe we can use simulated input apis here.

Comment 4

[Jim Mathies [:jimm]]

(In reply to Jim Mathies [:jimm] from comment #3)
> Maybe we can use simulated input apis here.

hmm, no, because that will get routed to the apzc as well. :/

Comment 5

[Matt Brubeck (:mbrubeck)]

(In reply to Jim Mathies [:jimm] from comment #2)
> Matt, any ideas? I don't remember why we switched to the apzc for taps in
> content.

We did this to support double-tap to zoom.  Since we don't double-tap to dispatch click events, we need to delay all events after a tap until we know whether it's a single-tap or part of a double-tap.

Some possible solutions we discussed:

* Disable double-tap zoom.  In the long term we may do this anyway for web pages that are already sized to the screen (bug 941995), so this might not be a big loss.  It would also improve responsiveness of single-tap on all pages, by eliminating the 300ms delay while we check for double-tap.

* If the first tap is on an editable element, dispatch a click immediately without waiting for the double-tap delay.  (This would disable double-tap zoom only when tapping within an editable field.  This appears to be what Internet Explorer does.)

* Add a way to show the keyboard programmatically (using synthetic pointer events), and call it when the DOM focus changes to an editable element.

Comment 6

[Matt Brubeck (:mbrubeck)]

Attachment: patch

This eliminates the 300ms tap delay by handling all taps in our widget input code rather than APZC.  That has the side effect of disabling double-tap zoom.

While it's unfortunate to remove this feature, it's the lowest-risk fix for the short term.  Longer-term we can look at the feasibility of more elaborate fixes that would let us add the double-tap zoom feature back, or at alternate gestures for zooming to an element.

It's somewhat worrying that this still does not work on debug builds, and may not work on very slow or overloaded machines.  We should figure out a way to make this less perf-sensitive.

Comment 8

[Matt Brubeck (:mbrubeck)]

https://hg.mozilla.org/integration/fx-team/rev/c2912eda9a01

Taking this for iteration 21 and moving the 3 points from bug 929862 to here.

Comment 10

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/c2912eda9a01

Comment 11

[Matt Brubeck (:mbrubeck)]

Comment on attachment 8348968 [details] [diff] [review]
patch

[Approval Request Comment]
Bug caused by (feature/regressing bug #): bug 941324

User impact if declined: On-screen keyboard does not appear when it should, when tapping on a form field in Metro.

Testing completed (on m-c, etc.): Landed on m-c 12/17.

Risk to taking this patch (and alternatives if risky): Low-risk Metro-only patch (just disables the regressing feature by backing out the key part of the change).
 
String or IDL/UUID changes made by this patch: None.

Comment 12

[Matt Brubeck (:mbrubeck)]

https://hg.mozilla.org/releases/mozilla-aurora/rev/a380a4c5d8cc

Comment 13

[u279076]

Karen, can you please confirm this is fixed for you now in Firefox 28 and 29?

Comment 14

[Karen Rudnitski [:kar]]

Tried it on 29 and the OSK comes up on first tap on a variety of websites for me (john lewis, mozilla support, marks and spencer). Tried their search boxes. Also glad to see that the URL bar also brings up the OSK on the first tap - lovely!

I did find a google maps bug in the meantime while trying this out, so will log that separately.

Comment 15

[u279076]

Thanks for your help Karen.

Comment 16

[Kamil Jozwiak [:kjozwiak]]

Went through the following issue for iteration #21 without any issues. Used the following build:
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014-01-12-00-40-02-mozilla-aurora/
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014-01-12-03-02-04-mozilla-central/

- Went through several registration forms and ensured that taping on the text fields slides in the OSK (Amazon, Steam, UPlay, Ebay, BestBuy)
- Taped on several text fields back to back and ensured that the OSK was not dismissed
- Ensured that taping the user/password fields slides in the OSK (Facebook, Twitter, GitHub, LinkedIn)
- Ensured taping text fields on search engines slide in the OSK (Google, Yahoo, DuckDuckGo, Bing)
- Ensured that taping on the website dismisses the OSK
- Ensured that pressing "ESC" dismisses the OSK