817684 - Notify DOM on hover entry and exit

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: General, defect)

Description

[Eitan Isaacson [:eeejay]]

Besides mouse hovers, Android uses motion hover events when in accessibility "explore by touch" mode. Which lets the user move their finger on a device and have the item under their finger read out to them without affecting its state. See bug 763613.

Mouse hovers don't translate perfectly to touch events. Mouse cursors are always on the screen and on top of *something*, while touch events have a start when the user touches the screen, motion when they move their finger, and end when the finger is lifted.

Comment 1

[Eitan Isaacson [:eeejay]]

Attachment: Translate Android hover enter/exit to mouseenter/mouseleave.

Hopefully this is an acceptable way to notify the start and end of a hover.

Comment 2

[Wesley Johnston (:wesj)]

I'm not sure what you're trying to do here. This is not the way we typically notify sites of mouseenter/leave events, and will probably send confusing information to sites.

Comment 3

[Eitan Isaacson [:eeejay]]

(In reply to Wesley Johnston (:wesj) from comment #2)
> I'm not sure what you're trying to do here. This is not the way we typically
> notify sites of mouseenter/leave events, and will probably send confusing
> information to sites.

Fair enough. There is a bit of a confusion here between actual mouse events and touch-hover events (an accessibility mode oxymoron). I am partly responsible for that after bug 763613. I'll look for a more holistic solution. And post here.

Comment 4

[Eitan Isaacson [:eeejay]]

When explore by touch is enabled, and a user pans with one finger, the system intercepts the event and changes the action to hover. The Android MotionEvent looks like this:
getAction() = ACTION_HOVER_ENTER/ACTION_HOVER_MOVE/ACTION_HOVER_EXIT
getSource() = SOURCE_TOUCHSCREEN
getPointerCount() = 1

When the user pans with two fingers, the system is designed to intercept the two finger event and simulate a one finger touch (not a hover, so the MotionEvent would look like this:
getAction() = ACTION_DOWN/ACTION_MOVE/ACTION_UP
getSource() = SOURCE_TOUCHSCREEN
getPointerCount() = 1

When a user puts more than 2 fingers down, the system relays the event faithfully to underlying apps.

You could read the full source for this behavior here:
https://github.com/CyanogenMod/android_frameworks_base/blob/jellybean/services/java/com/android/server/accessibility/TouchExplorer.java

On the DOM end, neither MouseEvent, nor TouchEvent, or the future PointerEvent could relay a touch that is actually a hover. That is an oxymoron. My previous patch simply made these mouse hover events. But they are hard to use since it is hard to detect the start and end of the touch, this is needed for gesture detection.

Before I write a patch I would like to propose the following, please let me know if this is too silly:

Keep the current mousemove interpretation *only* when the MotionEvent source is SOURCE_MOUSE. This will make websites behave correctly when a bluetooth mouse is connected to a device.

When a hover event is not SOURCE_MOUSE, interpret is as a TouchEvent (in the future a PointerEvent), with an identifier as -1. It seems that both TouchEvent and PointerEvent could support signed identifiers. The default non-binding behavior of these events is to have recycling positive integers as identifiers. Hover would be negative, but this would not be illegal by any of these specs.

The only app that would distinguish between signed and unsigned identifiers would be our in-chrome AccessFu. The only time we would ever encounter negative identifiers is when accessibility and explore by touch is enabled in Android.

Feedback?

Comment 5

[Wesley Johnston (:wesj)]

Ahh.. I think I understand the problem better now and the MOUSE_ENTER/LEAVE patch is making more sense to me. Its not all that different than what we do at:

http://mxr.mozilla.org/mozilla-central/source/widget/windows/nsWindow.cpp#3979

Looking at the patch...

Comment 6

[Wesley Johnston (:wesj)]

Comment on attachment 687847 [details] [diff] [review]
Translate Android hover enter/exit to mouseenter/mouseleave.

Review of attachment 687847 [details] [diff] [review]:
-----------------------------------------------------------------

I think this is fine. It doesn't look like buttons is actually used anywhere. Can we kill it?

Comment 7

[Wesley Johnston (:wesj)]

Comment on attachment 687847 [details] [diff] [review]
Translate Android hover enter/exit to mouseenter/mouseleave.

Review of attachment 687847 [details] [diff] [review]:
-----------------------------------------------------------------

I think this is fine. It doesn't look like buttons is actually used anywhere. Can we kill it?

Comment 8

[Eitan Isaacson [:eeejay]]

(In reply to Wesley Johnston (:wesj) from comment #7)
> Comment on attachment 687847 [details] [diff] [review]
> Translate Android hover enter/exit to mouseenter/mouseleave.
> 
> Review of attachment 687847 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> I think this is fine. It doesn't look like buttons is actually used
> anywhere. Can we kill it?

So you think this is a better solution over a negative touch identifier like i suggested above. Matt, do you have thoughts?

Comment 9

[Matt Brubeck (:mbrubeck)]

(In reply to Eitan Isaacson [:eeejay] from comment #8)
> So you think this is a better solution over a negative touch identifier like
> i suggested above. Matt, do you have thoughts?

Yes, I like using mouseenter/mouseleave.  (When we implement Pointer Events we can use pointerenter/pointerleave also, or instead.)  The touch "identifier" property is not really meant to have any meaning except as an opaque token, and I'd rather not try to change that.

Comment 10

[Eitan Isaacson [:eeejay]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/f0eab0765e25

Thanks!

Comment 11

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/f0eab0765e25