1833963 - Ship fractional coordinates of Pointer Events

Status: RESOLVED FIXED

(Core :: DOM: UI Events & Focus Handling, enhancement)

Description

[Hsin-Yi Tsai (she/her) | away [:hsinyi]]

The current spec still defines "long" types for coordinates.

Filing this bug to track interests from web developers (see bug 1234669 comment 2) and the open spec issue https://github.com/w3c/uievents/issues/40.

Comment 1

[bplu4t2f]

Just want to add some additional context and a maybe unpleasant addendum here:

This is needed when a canvas in a high DPI setup is in play, that is, the canvas takes more physical device pixels than it does CSS pixels.
In that case, pointer events must be accurate to device pixels, otherwise some pixels are skipped in input processing, which results in apparent jagged motion when the input coordinates are used to render something in the canvas.

But that is not quite all.
See this discussion: https://github.com/KhronosGroup/WebGL/issues/2460
(there's a whole bunch of related issues mostly from the WebGL team)

Summary: Multiplying CSS pixel coordinates with the devicePixelRatio value will de facto produce off-by-one errors due to rounding issues, depending on circumstances. Any attempt to calculate the number of physical device pixels occupied by an arbitrary canvas given its bounds in CSS pixels, by using devicePixelRatio is doomed to fail with current browser implementations, and can lead to visible moire patterns in the canvas. This led to the creation of device-pixel-border-box in ResizeObserver, which gives the size in actual device pixels.

I am mentioning this because technically, pointer coordinates have a similar problem. Most code I've seen multiplies the pointer event coordinates with devicePixelRatio (I am not aware of any way to get raw device coordinates here - if there is, disregard this comment). As a user, given fractional pointer coordinates, and a fractional device pixel ratio, I would expect the resulting device pixel coordinate to be accurate within the limits of floating point arithmetic, and a rounding error should not occur in this order of magnitude.

Comment 2

[Hsin-Yi Tsai (she/her) | away [:hsinyi]]

This is being handled in bug 1680669 IIUC, while I am not sure if there's anything left even after bug 1680669 is done.

Comment 4

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Let's use this bug to ship the fix of bug 1680669.

Comment 5

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Attachment: Bug 1833963 - Ship fractional coordinates of `PointerEvent` r=smaug!,#dom-core

Although it's tested only 3 weeks in the Nightly channel, it may be okay to
ship it because there is no regression report and the value new value is not
far over 1px.

Depends on D232822

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/dd8dce538c37
Ship fractional coordinates of `PointerEvent` r=smaug

Comment 7

[agoloman]

https://hg.mozilla.org/mozilla-central/rev/dd8dce538c37

Comment 8

[Ryan VanderMeulen [:RyanVM]]

Is this something we should call out in the Fx135 relnotes? Please add a relnote-firefox request if so.

Comment 9

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Release Note Request (optional, but appreciated)
[Why is this notable]: A major improvement of Pointer Events
[Affects Firefox for Android]: Yes, and important for Android
[Suggested wording]: The attribute values which indicate the coordinates of PointerEvent may be fractional values rather than integer. This allows web app developers to handle the events with higher precision coordinates when the target element is transitioned by CSS and/or the viewport is zoomed.

This is applied to untrusted events too, but is not applied to click, auxclick nor contextmenu events which were introduced as MouseEvent (for backward compatibility).
[Links (documentation, blog post, etc)]: https://w3c.github.io/pointerevents/#event-coordinates

Comment 10

[Ryan VanderMeulen [:RyanVM]]

Added to the Fx135 relnotes.