1417702 - [e10s] movementX and movementY misbehaving

Status: NEW

(Core :: DOM: Events, defect, P3)

Description

[lama12345]

Attachment: canvas.html

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36

Steps to reproduce:

Hello, I ported ioquake3 (via WebAssembly and WebGL) to the browser and I mostly used Chrome for that process. Always now and then I checked if everything works with Firefox and so it did.

But since Firefox Quantum the mouse movement for the FPS engine with locked mouse is messed up. But I don't know 100% if it worked perfect before Quantum, I mostly just checked if the WebGL rendering works and didn't test the ingame mouse movement so much.

The relative coordinates are out of touch with the actual mouse movement. It looks like the mouse pointer is drunk.

I made a video to illustrate the problem:

https://youtu.be/ESJfjyD0Ejc

Since the mouse is locked while gaming, I need to use the relative coordinates to move the 3D FPS player camera and those drunk values make the game unplayable.

You can quickly draw circles here for yourself, to test it on your platform: http://deepdreamers.org/canvas.html


Actual results:

https://youtu.be/ESJfjyD0Ejc


Expected results:

https://youtu.be/ESJfjyD0Ejc?t=44

Comment 1

[Henri Kemppainen]

Thank you for the detailed report!

I don't seem to get the exact same behavior, but there's certainly something wrong with mouse motion in the canvas demo when e10s is enabled (in 56, 57, and more subtly in Nightly, tested on Fedora).  In fact I can't draw circles at all, as the point will rush northwest and off the screen.  With e10s disabled, I can draw circles and it certainly doesn't look as drunk as in your video.

It might be useful to know if disabling e10s has any effect on your end (if it is, then we are probably witnessing the same bug).  If you have a few minutes to spare:

1) Go to about:config.
2) Toggle browser.tabs.remote.autostart.2 *off*.
3) Restart the browser, and verify that about:support says 0/1 (Disabled) for Multiprocess Windows.
4) Then try and draw circles again.
5) Report back.

Thanks again.

Comment 2

[lama12345]

Hello Henri Kemppainen, thanks for your quick reply!

I disabled Multiprocess Windows and it indeed works nearly as good as in Chrome. Only the initial delta value seems to be messed up, but only every 2nd time on pointerlock request.

Also thanks from my end.

Comment 3

[Henri Kemppainen]

Ok, I got a few bits of information.

When the pointer is locked, the browser tries to center the pointer every time you move the mouse.  So it can't escape the window.  Then relative pointer motion is computed against this center point.  The centering is done by synthesizing a native mouse event:

https://hg.mozilla.org/mozilla-unified/file/tip/dom/events/EventStateManager.cpp#l4419

The branch just below prevents the synthesized event from reaching the other side.  Unfortunately there seems to be some offset discrepancy that results in the mRefPoint *not* matching the computed center point.

When I said my pointer veers northwest, this is it.  The synthesized events are passed on as if they were real events, because they're offset from the center point by one pixel along both x and y.

It appears to be no coincidence that the offset is one pixel and so is the size of my window border.  In fact, if I increase my window border size to 5 pixels, then the mouse will veer off (-5, -5) pixels at a time.  And if I run the browser in fullscreen mode, well, this disables the border entirely and the problem goes away.

So far I can't tell why e10s has any effect, nor have I looked to see where the counts go wrong.

Comment 4

[Hsin-Yi Tsai (she/her) [:hsinyi]]

(In reply to Henri Kemppainen from comment #3)
> Ok, I got a few bits of information.
> 
> When the pointer is locked, the browser tries to center the pointer every
> time you move the mouse.  So it can't escape the window.  Then relative
> pointer motion is computed against this center point.  The centering is done
> by synthesizing a native mouse event:
> 
> https://hg.mozilla.org/mozilla-unified/file/tip/dom/events/EventStateManager.
> cpp#l4419
> 
> The branch just below prevents the synthesized event from reaching the other
> side.  Unfortunately there seems to be some offset discrepancy that results
> in the mRefPoint *not* matching the computed center point.
> 
> When I said my pointer veers northwest, this is it.  The synthesized events
> are passed on as if they were real events, because they're offset from the
> center point by one pixel along both x and y.
> 
> It appears to be no coincidence that the offset is one pixel and so is the
> size of my window border.  In fact, if I increase my window border size to 5
> pixels, then the mouse will veer off (-5, -5) pixels at a time.  And if I
> run the browser in fullscreen mode, well, this disables the border entirely
> and the problem goes away.
> 
> So far I can't tell why e10s has any effect, nor have I looked to see where
> the counts go wrong.

Olli or Stone may be able to tell us how e10s impacted.

Comment 5

[Ming-Chou Shih [:stone]]

It seems the mLastRefPoint isn't updated correctly. So far have no idea why it's affected by e10s.

Comment 6

[Ming-Chou Shih [:stone]]

Original implementation synthesizes a mousemove event to move the pointer back to the center of the window when handling a native mousemove event. It sets WidgetMouseEvent::mLastRefPoint to the center of the window. With e10s enabled, the movements of events are accumulated when there are continuous native events. Moreover, coalescing mousemove events changes the positions of events and we have no idea whether the event is synthetic or native.

Comment 7

[Ming-Chou Shih [:stone]]

I had a patch and it's work for me on Linux and Windows. Could you please help me test this build to confirm if the problem is fixed? Thanks.
https://queue.taskcluster.net/v1/task/VwLwRef1Tsa6d8ia1Fl4xw/artifacts/public/build/target.zip

Comment 8

[lama12345]

Hi, thanks alot for all the testing. But now it seems to be even worse.

I made a Video again to compare Chrome/Firefox: https://www.youtube.com/watch?v=Eu5m41prJKM

It seems that at times multiple events hit at the same time with wrong values. And when the devconsole is open, it is completely **** up. Later in the Video I opened the 3D game engine, which shows lots of stuttering. Then I tested it in Chrome, which works fine... then again in Firefox and it wasn't playable at all anymore.

Comment 9

[Ming-Chou Shih [:stone]]

Sorry for replying late. I found another cause of this bug and had a wip. I had tried on Windows10 and it seems better. Wondering could you please test it for me to confirm the wip works? Thanks.

here is the test binary for windows10(x64)
https://queue.taskcluster.net/v1/task/AIXGyKqWSbqG1Ys0m5FdrA/artifacts/public/build/target.zip

Comment 10

[lama12345]

This WIP works way better, just not yet perfect. :)

There are two problems left:

1) Every once in a while there is a wrong event, best seeable when moving the cursor fast
2) Entering or leaving the cursor lock while still moving the mouse causes a very big relative value

Made a short video to illustrate it better: https://youtu.be/UYmeXjsxyks

Comment 11

[Ming-Chou Shih [:stone]]

(In reply to lama12345 from comment #10)
> This WIP works way better, just not yet perfect. :)
> 
> There are two problems left:
> 
> 1) Every once in a while there is a wrong event, best seeable when moving
> the cursor fast
> 2) Entering or leaving the cursor lock while still moving the mouse causes a
> very big relative value
> 
> Made a short video to illustrate it better: https://youtu.be/UYmeXjsxyks

Thanks for your feedback!

Comment 12

[Ming-Chou Shih [:stone]]

Observed some problems of pointerlock with e10s enabled.
1. We may compress mousemove events in IPC level, which makes us no way to know whether they are synthetic or not.
2. Mouse coalescing in content side also break it as #1.
3. The real and synthesized mousemove events aren't necessarily interleaved.
4. OS might merge the request to bring the pointer to centre with the movement triggered by users.

#1, #2 and #3 aren't hard to fix. Still looking for a solution for #4.

Comment 13

[BugBot [:suhaib / :marco/ :calixte]]

The bug assignee didn't login in Bugzilla in the last 7 months.
:hsinyi, could you have a look please?
For more information, please visit auto_nag documentation.