1739598 - Mouse navigation in a specific game is severely impaired in wayland compared to the xwayland

Status: REOPENED

(Core :: Widget: Gtk, defect, P3)

Description

[Bodea Daniel [:danibodea]]

Note

• When the user opens the browser with wayland window manager and loads this game, he will notice that mouse navigation is severely impaired; the speed at which the cursor is moving (in-game) is very slow.

Affected versions

• Nightly v96.0a1

Affected platforms

• Ubuntu 21 + wayland

Steps to reproduce

1. Launch browser with wayland window manager:
   a. Go into the nightly folder.
   b. Right-click and "Open in Terminal".
   c. MOZ_ENABLE_WAYLAND=1
   d. ./firefox
2. Load https://classic.minecraft.net
3. Type in a username and click "Start".

Expected result

• The in-game mouse navigation should be comfortable to use for a normal game-play.

Actual result

• The in-game mouse navigation is severely impaired, in the sense that the speed at which it's moving is much too slow for a normal playing environment.

Regression range

• unknown.

Additional notes

• This issue occurs with "wayland" Window Protocol and not with "xwayland".

Comment 1

[Robert Mader [:rmader]]

Greg, do you have an idea what could be wrong here? And do you think it's a Firefox or compositor issue?

Comment 2

[Jan Alexander Steffens [:heftig]]

The implementation of pointer lock on Wayland provides unaccelerated movement. In the pointer lock demo, notice how the circle moves noticeably slower than an unlocked mouse cursor.

Having unaccelerated movement available to the application is desirable, in principle, as it allows the application to define its own speed and acceleration when mouse movement does not steer a cursor.

However, it is contrary to the spec, which specifies that the movement deltas provided should match the system cursor, acceleration and all.

this specification defines the movement deltas to be taken from how the system mouse cursor moves, which incorporates operating system filtering and acceleration of the mouse movement data

So it also breaks the use case of emulating a confined cursor given just movement deltas, where the locked cursor should move identical to an unlocked one.

Comment 3

[Jan Alexander Steffens [:heftig]]

I suspect using dx_w and dy_w instead of dx_unaccel_w and dy_unaccel_w in relative_pointer_handle_relative_motion will fix this.

https://searchfox.org/mozilla-central/rev/21f0bb70ce5747d18625ad1c67f03ed2af7f9c56/widget/gtk/nsWindow.cpp#9104-9105

Comment 4

[Martin Stránský [:stransky] (ni? me)]

Jan, do you mind to create a patch for it?
Thanks!

Comment 5

[Robert Mader [:rmader]]

Attachment: Bug 1739598 - Use accelerated deltas for Wayland pointer lock, r=stransky

Patch suggested by Jan Alexander Steffens <[e-mail]>

From the spec (https://w3c.github.io/pointerlock/):

this specification defines the movement deltas to be taken from how
the system mouse cursor moves, which incorporates operating system
filtering and acceleration of the mouse movement data

Comment 6

[Robert Mader [:rmader]]

Lets try! https://treeherder.mozilla.org/jobs?repo=try&revision=7960d35b0b5f6a3511f91f30e9e493c894694629

Edit: https://treeherder.mozilla.org/jobs?repo=try&revision=b77e0e909e85bbfc59b5d6863c5df137909c8daa&selectedTaskRun=RXYnqE2rQMi_c3XuvCFYXQ.0 for the double->int change, but as expected it doesn't seem to change anything.

Comment 7

[Robert Mader [:rmader]]

Yep, with the try build from comment 6, the pointer lock demo from comment 2 now behaves as it should for me.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c2039e42047f
Use accelerated deltas for Wayland pointer lock, r=stransky

Comment 9

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/c2039e42047f

Comment 10

[Bodea Daniel [:danibodea]]

It appears that this issue still reproduces with the latest Beta v96.0b5 which contains the fix.
NeedInfo me for more info and/or testing.

Comment 11

[Robert Mader [:rmader]]

Hm, yeah, I somewhat suspected this to not be completely right because we do not take scaling into account. Will revisit, keeping ni active.

Comment 12

[Robert Mader [:rmader]]

Bodea: can you shortly elaborate about your setup? Most notably: do you use buffer scaling (something like 200% in the display settings of the OS)? And do you have any default zoom setting in Firefox set (usually a number like 90% in the url bar)?

Maybe you can also add you about:support for the version you tested? Thanks!

Edit: and do you also see a difference of pointer speed on https://mdn.github.io/dom-examples/pointer-lock/ ?

Comment 13

[Bodea Daniel [:danibodea]]

Attachment: about:support of an affected browser session (wayland)

My setup has 2 monitors (though, this does not affect it); I do not use any scaling (it is 100% by default), also I don't usually use zoom in firefox (unless specifically needed in test process).

I have attached the about:support information from an affected browser session.

Comment 14

[Bodea Daniel [:danibodea]]

The other test page you've linked in your request does not seem to be affected by a similar issue.
Mouse navigation sensitivity appears to be the same in both wayland and xwayland window protocols.

Comment 15

[Robert Mader [:rmader]]

Hm, tested this again and both, https://classic.minecraft.net/ and https://mdn.github.io/dom-examples/pointer-lock/, appear to be affected. Strangly, apparently only when using an actual mouse, not my touchpad. Will look into it.

Comment 16

[Robert Mader [:rmader]]

Greg, I'm a bit stuck on this. Do you see something similar yourself? What confuses me is that things work totally fine for my touchpad but not the mouse. Any ideas?

Comment 17

[greg v [:unrelentingtech/myfreeweb]]

The MDN example has good speed with both mouse and touchpad for me. Minecraft is slow (too low sensitivity) with the touchpad but fine with the mouse. Weird.

The only thing that's been bothering me is not speed related — non-fullscreen things seem to have some downward drift of the cursor when moving only horizontally, it is only slightly noticeable in the MDN example but way more significant in the "grab and drag a number field" feature in Figma. Maybe that's related to the rounding of the center or something. We should try directly passing the deltas to movementX/Y.

Comment 18

[Jan Alexander Steffens [:heftig]]

Yeah, I can confirm that. The pointer lock demo feels good speed-wise but drifts vertically when moving horizontally. Minecraft is too slow. This applies to both touchpad and mouse.

Using Chrome from Flatpak (XWayland), the pointer lock demo moves too quickly but Minecraft feels good.

Epiphany on Wayland seems to be using unaccelerated deltas like Firefox used to. The pointer lock demo feels okay but Minecraft is unusably slow.
Epiphany on XWayland is broken.

Arch Linux, GNOME Shell 41.3, scale 2.

Comment 19

[greg v [:unrelentingtech/myfreeweb]]

Here's some logging where 'PLC' lines are from the GTK nsWindow and the 'cur' lines are from UIEvent::GetMovementPoint:

cur 676,430 last 667,430 delta 9,0
cur 676,345 last 664,344 delta 12,1
cur 676,345 last 664,344 delta 12,1
cur 676,345 last 664,344 delta 12,1
PLC 1329,859 delta 4.937500,0.000000
cur 667,430 last 676,430 delta -9,0
PLC 1329,859 delta 6.753906,0.000000
cur 668,430 last 667,430 delta 1,0
cur 668,345 last 664,344 delta 4,1
cur 668,345 last 664,344 delta 4,1
cur 668,345 last 664,344 delta 4,1
PLC 1329,859 delta 30.132812,0.000000
cur 680,430 last 668,430 delta 12,0
cur 680,345 last 664,344 delta 16,1
cur 680,345 last 664,344 delta 16,1
cur 680,345 last 664,344 delta 16,1
PLC 1329,859 delta 6.496094,0.000000
cur 668,430 last 680,430 delta -12,0

This 1 delta (which seems to be a HiDPI rounding error, doesn't happen to me with Title Bar enabled) that does the drift comes from those extra events that did not come from our relative events. Could it be that we still have absolute events coming too?

Comment 20

[greg v [:unrelentingtech/myfreeweb]]

Nah, that's not it. The drift is because whatever operations inside Firefox transform the event to remove window offsets are not exactly inverse to + mChromeOffset, causing a rounding error to constantly accumulate :/

I can't really reproduce the Minecraft speed problem. (I said it's too slow with touchpad but.. not really, that's just "controlling first person cameras with a touchpad generally is not good".) Maybe that's actually something in the compositor, not under our control.

Comment 21

[Robert Mader [:rmader]]

Possibly related: bug 1644307

Comment 22

[Jan Alexander Steffens [:heftig]]

I just started a session with scale-monitor-framebuffer enabled and the speed of the locked pointer got halved. Now Minecraft is extremely slow again.

PS: Using Chrome from Flatpak (XWayland), the browser is now scaled by the compositor, the pointer lock demo moves at the same speed as the pointer and Minecraft feels too slow, but not as slow as Firefox.

Arch Linux, GNOME Shell 41.3, scale 2.