1991701 - RFP: review maxTouchPoints and window Touch* properties

Status: RESOLVED FIXED

(Core :: DOM: Events, enhancement)

Description

[Thorin [:thorin]]

first a recap (from memory) - we've had a bit of history with maxTouchPoints

• android: we've always reported something, whatever it is/was and we've settled on 5 now which is all good: matches FPP's cap
• desktop
  • we used to report 0
  • then we decided to report 10 : the idea being possible compat but also that then meant that if someone happened to actually use touch, it wouldn't be significant. If the user didn't use touch they were using their mouse or trackpad, otherwise they used their fingers - big deal. It's also makes touch when detected (and you can't hide it!! trust me!) non-adversarial
  • but there were "issues" (that were going to be solved in Bug 1957658)
    • macs aren't touch capable (yet)
    • devices we spoof as touch can be missing window Touch* properties
  • the end results being that Mac and Linux are now no longer spoofed as touch capable, and I fail to see why that was done to Linux

PieroV and I have done a lot of testing and discussion and would like some changes since Bug 1957658 which was backported to ESR

Proposals

1. dom.w3c_touch_events.enabled
   • defaults: 0=disabled (macOS) 1=enabled 2=autodetect (linux/win/android)
   • when at default Bug 1957658 solves the window Touch properties - mac doesn't factor into it, as it's disabled
     • edit: IDK what happens on mac if the value is 2 - maybe it overrides it
   • the problem is that values 0 and 1 override our desired result depending on the value and platform
   • we would like RFP to ignore the pref value and do the right thing! edit on all platforms including mac
2. linux should be spoofed as touch capable
   • this then re-solves the problem of users actually using touch not being a significant thing, and also makes it benign
   • maybe better compat
   • we think 5 is the best value - but would like some data breakdown
3. windows we would like to change the maxTouchPoints value
   • 10 is really a high end laptop, and yet we report lower end webgl specs etc
   • we think 5 is the best value - but would like some data breakdown

@tom can we have a breakdown on windows and linux maxTouchPoints where it's not 0 to determine the best value

One other thing

• with desktops and pointerevents etc and spoofing touch as mouse, we have "issues" - compat stuff, w.g. gestures not working, or text being selecting instead of scrolling .. touch not working in maps .. and so on (from memory)
• by having all windows and linux users reporting as touch capable (and mac doesn't factor here since they lack touch), the fact that touch is used is no longer significant and is now benign - so we could drop spoofing touch as mouse - this will solve a lot of headaches - am interested in people's thoughts on this: e.g. for a surface user who (probably almost) always uses touch, this might help linkify traffic, but then I think tablets already leak entropy via resolutions and other metrics
  • if we did this, then we could spoof pen as touch which is closer to the truth - piero and I discussed this a little

Comment 1

[Pier Angelo Vendrame]

I activated my old Surface and done some tests.
I found out that it reports 0 touch points on Linux, which not true, it should support 10.
So, I went through code, and I think that reporting the number of touch points is implemented only on Windows and on Android: https://searchfox.org/firefox-main/search?q=%3A%3AGetMaxTouchPoints&path&case=false&regexp=false...
Therefore, 0 might be a good value. However, in some parts of the code there was an assumption that 0 meant that we had to spoof touch disabled.
The standard isn't explicit about that, however it seems to me 0 doesn't violate this:

On platforms where the precise number of touch points is not known, the minimum number guaranteed to be recognized is provided. Therefore, it is possible for the number of recognized touch points to exceed the value of maxTouchPoints.

That said, we could override the value of dom.w3c_touch_events.enabled with hardcoded values when RFP is on, rather than checking RFP only when its value is 2 (the current logic), and set touch enabled everywhere, except on macOS.

FWIW, there's a device with 0 touch points but Touch events: my Surface when I use Wayland.
So, we'd be benign.

---

Windows: I think 10 as a number of touch points is good.
My lay take is that touch is a high end functionality, so it makes sense that once you have it, you have all the 10 points.
But I'd be also curious to see telemetry.

---

Pens: they're platform dependent, tracked in Bug 1953665 😢.
On Windows and Android, pens are treated as touch inputs.
On Linux and macOS, they are treated as mouse.
I'd be up for spoofing them in this way, but I guess we can handle it in Bug 1987929.

Comment 2

[Thorin [:thorin]]

That's interesting about linux

When testing I found that blink desktop (windows, mac, linux) always reports the three window touch properties regardless of maxTouchPoints or touch capability. Safari always reports none on desktop. We could should do the same IMO and remove deprecate the pref

I too am interested in tom's data analysis

Comment 3

[Pier Angelo Vendrame]

I wrote a simple page for checking pointerId and touch events:

<pre></pre>
<script>
const pre = document.querySelector("pre");

function log(t) {
  pre.textContent = t + "\n" + pre.textContent;
}

function dumpPointer(e) {
  log(`${e.type} ${e.pointerId}`);
}

document.addEventListener("pointerdown", dumpPointer);
document.addEventListener("pointerup", dumpPointer);

function dumpTouch(e) {
  const touches = Array.from(e.touches ?? [], t => "" + t.identifier).join(" ");
  log(`${e.type} [${touches}]`);
}

document.addEventListener("touchstart", dumpTouch);
document.addEventListener("touchend", dumpTouch);
</script>

And I'm wondering if the target makes sense.
Without disabling touch events, that information is just equivalency.
I checked on Linux, and I noticed I get incrementing integers in pointerId, and they are always consistent with the touch events. It's consistent with Windows. Android, instead, counts how many fingers are down, and restarts the pointerId count.
FWIW, with RFP on you get those integers too, so as a matter of fact you bypass RFPTarget::PointerId.

Therefore, the PointerId target might be useful only on macOS, but I wonder if macOS even has multi-touch code implemented.
Maybe some graphics tablet could help to test that... But it's hard for me to get to one.
If it isn't helpful, we could drop it.

Short term, it seems that the logic to check whether SPOOFED_MAX_TOUCH_POINTS is 0 or greater won't work, because of the differences between Linux and macOS.

Comment 4

[Pier Angelo Vendrame]

Emilio: when you converted dom.w3c_touch_events.enabled to a static pref, you added this fixme comment.
Have you ever figured out if TouchEvent::PrefEnabled can be used there?
It seems to me it can, and that would help to make RFP override that pref, but maybe I'm missing something 😅.

Comment 5

[Emilio Cobos Álvarez [:emilio]]

I think so, yeah. I just didn't fix it myself because I didn't want to change behavior in that patch.

Comment 6

[Pier Angelo Vendrame]

Perfect, thank you!

Comment 7

[Pier Angelo Vendrame]

Attachment: Bug 1991701 - Re-enable touch on Linux with RFP and remove RFPTarget::PointerId. r?tjr

On Windows, we do not spoof touch-related inputs anymore, and we
pretend touch is always available.
There is no reason not to do the same on Linux, even though the code to
report the number of maxTouchPoints is not implemented there.

Comment 8

[Pier Angelo Vendrame]

Hi Emilio, I left a question as a reply to the comment you left in the review.
Could you please have a look? Thanks in advance!

Comment 9

[Emilio Cobos Álvarez [:emilio]]

Hey, sure, separate bug seems fine. Sorry if you don't re-request review it doesn't show up in the phab dashboard.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/00f1baf7d5af
https://hg.mozilla.org/integration/autoland/rev/414f55d38b69
Re-enable touch on Linux with RFP and remove RFPTarget::PointerId. r=tjr,layout-reviewers,emilio

Comment 11

[agoloman]

Backed out for causing perma failures @test_pointer_event.html.

• Backout link
• Push with failures
• Failure Log

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/56fc255e5729
https://hg.mozilla.org/integration/autoland/rev/56c31a3ddb6a
Re-enable touch on Linux with RFP and remove RFPTarget::PointerId. r=tjr,layout-reviewers,emilio

Comment 13

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/56c31a3ddb6a