1244402 - Let touch events on windows ride the trains

Status: RESOLVED FIXED

(Core :: Panning and Zooming, defect)

Description

[Kartikaya Gupta (email:[e-mail])]

In bug 1180706 I turned on touch events for windows nightly builds. This bug is to let it ride the trains. There will be some UX tweaking needed, as well as compat testing to make sure websites that feature-detect touch events don't suddenly start misbehaving on windows touch devices.

Comment 1

[Kohei Yoshino]

This could be first enabled in Developer Edition as well so that webdevs will be able to test their sites easier without having to toggle the pref. I'll post the site compatibility document at some point.

# Why Mozilla doesn't have any consistent policy on experimental features? Some features are available both in Nightly and DevEdition, some are only in Nightly, some are disabled by default. That really sucks.

Comment 3

[Liz Henry (:lizzard) (relman/hg->git project)]

Marking this as a feature to help QA/relman keep track of upcoming work. 
kats are we still planning to ship apz with e10s in 48?  Should/can we prevent that for touchscreen devices? I just noticed a bug that blocks this feature shipping, and this bug is marked as blocking apz for desktop.

Comment 4

[Kartikaya Gupta (email:[e-mail])]

We are still planning to ship APZ with e10s in 48, yes. Touch support on Windows will not ship in 48, it is still being worked on. This bug is marked as blocking bug 1013364 ("apz-desktop") but that's a very general meta bug for all desktop apz-related things. The 48 blockers list is hanging off bug 1254668 (and is empty).

Comment 5

[Liz Henry (:lizzard) (relman/hg->git project)]

Thanks, that clarifies things! I'll keep it in mind for the other bugs that come up.

Comment 6

[Kartikaya Gupta (email:[e-mail])]

This is getting close to done. Once all the things that are currently inflight land I'd be happy to flip the switch and let this ride the trains. There are still some outstanding bugs blocking this one but I don't consider any of those ship-blockers and at least some of them can be worked on further and uplifted before this goes to release.

Unfortunately the accessibility rewrite is still ongoing and so e10s is still not yet enabled by default on touch devices so we're still not getting a lot of users exercising this code.

Comment 7

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1244402 - Allow touch events on Windows to ride the trains.

Review commit: https://reviewboard.mozilla.org/r/83358/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/83358/

Comment 8

[Kartikaya Gupta (email:[e-mail])]

Note that this patch enables:
- DOM touchevents on platforms that don't have it already (windows)
- APZ touch scrolling on platforms that don't have it already (windows)
- touch-action on all platforms (windows, linux, android)
- accessible caret on all platforms that don't have it already (windows, linux)

Comment 9

[Jim Mathies [:jimm]]

Comment on attachment 8797705 [details]
Bug 1244402 - Allow touch events on Windows to ride the trains.

https://reviewboard.mozilla.org/r/83358/#review81890

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6955ebc6baf1
Allow touch events on Windows to ride the trains. r=jimm

Comment 11

[Kartikaya Gupta (email:[e-mail])]

I also sent an intent to ship to dev-platform: https://groups.google.com/d/msg/mozilla.dev.platform/6CGjsm1XpD4/GDbW6mCDAQAJ

Comment 12

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/6955ebc6baf1

Comment 13

[Kohei Yoshino]

Posted the site compatibility doc: https://www.fxsitecompat.com/en-CA/docs/2016/touch-event-support-has-been-re-enabled-on-windows-desktop/

Comment 14

[Kohei Yoshino]

Release Note Request (optional, but appreciated)
[Why is this notable]:
[Affects Firefox for Android]:
[Suggested wording]:
[Links (documentation, blog post, etc)]:

Comment 15

[Chris Mills [:cmills]]

I've added a note into all appropriate pages under the touch events docs to make the change clear (see https://developer.mozilla.org/en-US/docs/Web/API/Touch_events). 

I've also added a note to the Firefox 52 release notes to cover it:

https://developer.mozilla.org/en-US/Firefox/Releases/52#DOM_HTML_DOM

Please let me know if this description works for you. Thanks!

Comment 16

[Kartikaya Gupta (email:[e-mail])]

That works, thanks!

Comment 17

[qre]

ff version 53.0a2 (2017-02-08)
When I installed 53 version touch events were working. But till yesterday | today update. Now again they are not firing up (

Comment 18

[Kartikaya Gupta (email:[e-mail])]

Can you check in about:support to see what it says for "Multiprocess Windows" and for "Asynchronous Pan/Zoom"?

Comment 19

[qre]

Multiprocess Windows 	0/1 (Disabled by accessibility tools)
Asynchronous Pan/Zoom	none

Comment 20

[Kartikaya Gupta (email:[e-mail])]

Touch events are only available if e10s and APZ are on. For you it seems that they are off, because of accessibility tools. I know that the e10s team is still working on hammering out the accessibility problems. For now you can force-enable e10s by adding a pref browser.tabs.remote.force-enable in about:config and setting it to true.

Comment 21

[qre]

> For now you can force-enable e10s by adding a pref browser.tabs.remote.force-enable in about:config and setting it to true.

Following your instructions yeah - they're again firing up. Thanks!

its strange, i didn't opt anything in these days. Updates were auto when starting-up ff. And suddenly it turned that features off?

Comment 22

[Luke]

As far as I can tell, css touch-action is now enabled (see #8) even when electrolysis is disabled. (Test: http://tests.caniuse.com/?feat=css-touch-action with google translator for firefox addon.) Could this please be mentioned in the release notes?

Comment 23

[Chris Mills [:cmills]]

(In reply to Luke from comment #22)
> As far as I can tell, css touch-action is now enabled (see #8) even when
> electrolysis is disabled. (Test:
> http://tests.caniuse.com/?feat=css-touch-action with google translator for
> firefox addon.) Could this please be mentioned in the release notes?

Ah, you are talking about this:

https://developer.mozilla.org/en-US/docs/Web/CSS/touch-action

?

I've looked into this, and it looks like work is still ongoing, see these bugs:

https://bugzilla.mozilla.org/show_bug.cgi?id=960316
https://bugzilla.mozilla.org/show_bug.cgi?id=1285685

Comment 24

[Luke]

(In reply to Chris Mills (Mozilla, MDN editor) [:cmills] from comment #23)
> Ah, you are talking about this:
> 
> https://developer.mozilla.org/en-US/docs/Web/CSS/touch-action
> 
> ?
> 
> I've looked into this, and it looks like work is still ongoing, see these
> bugs:
> 
> https://bugzilla.mozilla.org/show_bug.cgi?id=960316
> https://bugzilla.mozilla.org/show_bug.cgi?id=1285685
Yep, that's the one :) Some related work is indeed ongoing, but in my opinion isn't significant here for developers:

bug 960316 is the meta bug for implementing the pointer events specification. Touch-action is defined in the pointer events specification but it's a distinct property with applicability to touch events, most notably the touch-action:manipulation trick for bypassing the 300ms click delay. (In fact even Safari supports this, Firefox is the last.) The actual implementation bug (bug 795567) is long resolved and the meta bug states that touch-action may be shipped before pointer events.

bug 1285685 is the bug for implementing the new values added to the touch-action property in the pointer events level 2 draft spec. In my opinion that shouldn't be a blocker for announcing level 1 support, the current W3C recommendation.

I also point to the intent to ship touch-action (all platforms) that references this bug as the enabler: https://groups.google.com/forum/#!topic/mozilla.dev.platform/6CGjsm1XpD4
Some of these arguments are also stated in the intent to enable touch-action on nightly for FF50:
https://groups.google.com/forum/#!topic/mozilla.dev.platform/SYEzvXJKw9M

Perhaps kats disagrees but it seems like an oversight to me...

Comment 25

[Kartikaya Gupta (email:[e-mail])]

Luke is correct. We should probably add touch-action to the dev notes for 52.

Comment 26

[Chris Mills [:cmills]]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #25)
> Luke is correct. We should probably add touch-action to the dev notes for 52.

OK, cool. Thanks to Luke for the detailed explanation that made sense of it all, and to kats for chiming in ;-)

I have added a note to the release notes under CSS/New features:

https://developer.mozilla.org/en-US/Firefox/Releases/52#New_features

I have pointed to the intent to ship mails fthe full story, rather than to specific bug(s), as I think it makes more sense if you read those.

Comment 27

[Luke]

LGTM, thanks Chris!

Comment 28

[Alecazam]

This doesn't seem to work on Windows Firefox 52.  Setting touch-action:none on the body tag in our css.  We are still seeing browser zoom using touch pinch-zoom, and all touch actions seem to still apply.  The default Firefox touch actions are all fine except the browser zoom.  Firefox doesn't do swipe back/forward actions like Chrome which is good for our web application where we need app pinch-zoom and panning.

Comment 29

[Alecazam]

Also we set meta viewport initial-scale=1 max-scale=1 user-scalable=no, but Firefox never seems to honor that.  Must be a mobile only setting, but would be nice if the desktop honored that as well.

Comment 30

[Kartikaya Gupta (email:[e-mail])]

(In reply to Alecazam from comment #28)
> This doesn't seem to work on Windows Firefox 52.  Setting touch-action:none
> on the body tag in our css.  We are still seeing browser zoom using touch
> pinch-zoom, and all touch actions seem to still apply.  The default Firefox
> touch actions are all fine except the browser zoom.  Firefox doesn't do
> swipe back/forward actions like Chrome which is good for our web application
> where we need app pinch-zoom and panning.

Can you check what it says for "Asynchronous Pan/Zoom" in the graphics section on about:support? Also, if you can post a test page that reproduces the problem for you that would be great. Thanks!

(In reply to Alecazam from comment #29)
> Also we set meta viewport initial-scale=1 max-scale=1 user-scalable=no, but
> Firefox never seems to honor that.  Must be a mobile only setting, but would
> be nice if the desktop honored that as well.

Yeah at the moment we only support that on mobile. Zoom is tricky because there are different kinds of zoom. The zoom on desktop isn't the same as the zoom on mobile (at least in Firefox). Mobile has "continuous zoom" where everything on the page is scaled, but desktop only has text zoom or page zoom, which involves reflowing the content. The different semantics mean that the meta-viewport tag doesn't exactly map to desktop zoom.

Comment 31

[Adrian]

I am seeing the same problem (touch-action: none; not being honored on Windows Firefox 52). If I remember correctly I tested it on Nightly 52 a while back, where this seemed to work. 

Under "Asynchronous Pan/Zoom" it says "none" / "nothing" for me (freely translated from German "nichts").

Example page showing the problem: http://output.jsbin.com/hubulobovo

Comment 32

[Kartikaya Gupta (email:[e-mail])]

(In reply to Adrian from comment #31)
> Under "Asynchronous Pan/Zoom" it says "none" / "nothing" for me (freely
> translated from German "nichts").

This means APZ is off, which is probably because e10s is off as well (check what it says under Multiprocess Windows in about:support). Without APZ we don't support touch events and therefore touch-action will not work either.

Comment 33

[Adrian]

Yes, it says: 0/1 (Disabled by accessibility tools)
What would be the reason for this to happen, and can I somehow work around this?

Comment 34

[Kartikaya Gupta (email:[e-mail])]

This is most likely because the touchscreen gets detected as an accessibility device and enables some of the accessibility codepaths, disabling e10s. This was fixed in Firefox 52 for 32-bit builds, but is still a problem for 64-bit builds. It should be fixed in Firefox 53.

Comment 35

[Adrian]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #34)
> This is most likely because the touchscreen gets detected as an
> accessibility device and enables some of the accessibility codepaths,
> disabling e10s. This was fixed in Firefox 52 for 32-bit builds, but is still
> a problem for 64-bit builds. It should be fixed in Firefox 53.

okay, thanks for the clarification!

Comment 36

[Alecazam]

The Samsung Notebook 9 Pro with high-precision touchpad reports Async Pan-Zoom as none on both 32-bit and 64-bit.  32-bit FF honors touch-action:none and does not perform browser-zoom/double-tap-zoom/swipe from the touchpad (not touchscreen), whereas 64-bit does do all those things (ignoring touch-action).   These are both FF 52, so hopefully 53 works.  

Edge still doesn't honor touch-action either, and Microsoft created the spec.  Only Chrome works right now.  This is the fiddle that I modified from elsewhere to test touch-action support  https://jsfiddle.net/vd4yr00L/8

Comment 37

[Adrian]

(In reply to Alecazam from comment #36)
> The Samsung Notebook 9 Pro with high-precision touchpad reports Async
> Pan-Zoom as none on both 32-bit and 64-bit.  32-bit FF honors
> touch-action:none and does not perform browser-zoom/double-tap-zoom/swipe
> from the touchpad (not touchscreen), whereas 64-bit does do all those things
> (ignoring touch-action).   These are both FF 52, so hopefully 53 works.  
> 
> Edge still doesn't honor touch-action either, and Microsoft created the
> spec.  Only Chrome works right now.  This is the fiddle that I modified from
> elsewhere to test touch-action support  https://jsfiddle.net/vd4yr00L/8

Not sure about your last point. I just successfully tested `touch-action: none;` preventing browser default pinch-to-zoom with the following snippet in Edge and IE11: http://output.jsbin.com/hubulobovo

Comment 38

[Alecazam]

Sorry wrong thread, but Edge is bust with your example.  I still get browser-zoom in Edge on the Samsung, and with my fiddle.  I'm just setting touch-action:none on the body of the fiddle.

Comment 39

[Adrian]

(In reply to Alecazam from comment #38)
> Sorry wrong thread, but Edge is bust with your example.  I still get
> browser-zoom in Edge on the Samsung, and with my fiddle.  I'm just setting
> touch-action:none on the body of the fiddle.

Oh that's interesting. I've been testing it using the touch screen on my Dell Precision M3800 laptop, whereas you have been using your touchpad. Testing it using my laptop's touchpad I get the same results as you.

For completeness sake the browser version used is: Microsoft Edge 38.14393.0.0, Microsoft EdgeHTML 14.14393.

Comment 40

[Kartikaya Gupta (email:[e-mail])]

Most touchpads are not going to be treated as touch devices. Touchpad drivers might have their own gesture detection code and deliver gesture events to the browser/OS, so in those cases touch-action will not apply to that input. Only if the touchpad is delivering WM_TOUCH messages to the browser will touch-action apply. Touchscreens generally produce WM_TOUCH.

Comment 41

[Alecazam]

Okay, I checked and the Razer Stealth does not have a high precision touchpad, and the Samsung Notebook 9 Pro does.  That still does't explain the magnitude differences in reported numbers on wheel event deltaXY in Firefox.  Pinch, for example, sends a huge value on Samsung, and tiny value on Razer.  I'm not sure how people can respond to this much variation in data.  These are both running Windows 10, and are less than a year old.  It's not going to help if touch/gesture convert their own values into these same deltas.


    // On Firefox, on Razer, we can only do this conversion for pans, zoom and pinch-zoom are
    //  not scaled by dpr or numLinesToScroll, but the last zoom event is scaled by numLinesToScroll (big spike).
    //  This happens when the inertia slows on the Razer.  Also seeing 4, 8, 4, 8 at tail end of inertia.
    //  And there are large inertia spikes on Chrome Windows, and cutoff at the tail of inertial movement.

    // Razer, the base units for delta using touchpad are:
    // pan    = 0.2        no dpr, 6x bigger than zoom/pinch
    // pan-w  = 3.0        this is 15x bigger than touchpad
    // zoom   = 0.01666    no dpr, but numLinesToScroll spike at end of pinch (can be 1.666 or higher)
    // zoom-w = 3.0        this is 80x bigger than touchpad     180/60/1 if normalizing on 0.016666
    // pinch  = 0.01666    no dpr, but numLinesToScroll spike at end of pinch

    // these magniutes overlap, making it difficult to detect pinch or actual wheel use
    //   may need to identify from key_down
    
    // Samsung
    // pan    = 0.075   <- differences
    // pan-w  = 3.0
    // zoom   = 0.075   <-
    // zoom-w = 3.0
    // pinch  = 3.0     <-

Comment 42

[Kartikaya Gupta (email:[e-mail])]

So this is way off-topic for this bug now, since it has nothing to do with touch-action. Please direct any follow-ups to this on a new bug if you are having trouble with wheel events. That being said, most likely you need to take the deltaMode into account to normalize the deltaX/deltaY values. https://developer.mozilla.org/en-US/docs/Web/API/WheelEvent/deltaMode

Comment 43

[Alecazam]

Not trying to take things off-topic, but trying to share the results I have.   The differences are more significant than a missing multiplier for deltaMode.  Also the last value on zoom and pinch-zoom uses numLinesToScroll causing a massive spike at the end.  Just pointing to larger problems with event handling in Firefox.  Chrome and Edge have their own problems too.

Comment 44

[Alecazam]

Pointer events are supposed to encompass touch-pad, and touch-action seems to encompass the gestures that the touchpad is defaulting to that are nearly identical to the touchscreen gestures on the same Samsung device.  Indeed Microsoft states that touch-action doesn't affect mouse, keyboard, or touchpad.  I'm assuming there then has to be some identical set of filtering to disable the touchpad, or a lot of event.preventDefault() calls.