853160 - Mouse cursor can break outside the window bounds while pointer is locked with Pointer Lock API

Status: NEW

(Core :: DOM: Core & HTML, defect)

Description

[David Humphrey (:humph)]

When we first did the Pointer Lock implementation (bug 633602), we did it only for fullscreen elements.  Since then work has been done to implement the rest of the spec, and add support for non-fullscreen elements (bug 737100).  The latter has exposed a bug in the earlier work, namely, when the mouse is locked, it is possible to move it quickly enough to break outside the bounds of the window, causing the mouse cursor to appear.

STR:

* Make your browser window 1/3 of your screen, place it in the centre
* Go to http://www.html5rocks.com/en/tutorials/pointerlock/intro/
* Find the "Click Me!" button, click it
* Allow pointer lock in the door hanger (click "Yes")
* Notice the mouse pointer is locked, and cursor vanishes
* Try moving really fast, notice you can get a mouse pointer outside the browser (i.e., the arrow appears)
* Move back toward the browser, notice the arrow vanish and pointer lock take over again

I get this on my iMac with an Apple mouse, and in a Windows VM.  smaug is unable to reproduce it.  In the fullscreen-only case, you had to have multiple monitors to hit this, so it was basically a non-issue; now in the non-fullscreen case it makes for a jarring experience.

Comment 1

[Justin Dolske [:Dolske]]

I can also reproduce this on OS X 10.8... With the additional excitement that it will thusly trigger the Mission Control hotcorner I have set up.

Comment 2

[(not reading bugmail) Nick Desaulniers [:\n]]

I think this is what I'm seeing in OSX 10.8.5.
http://timeinvariant.github.io/gorescript/play/

Comment 3

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Hi Xidorn, this bug was mentioned in the mid-aurora sign offs by SoftVision team. Is this something we plan to fix in Aurora50? It seems to be such an old bug that it could not possible block the Fx50 release. Right?

Comment 4

[Xidorn Quan [:xidorn] UTC+10]

I don't think it needs to block Fx50.

This is probably not very hard to fix, but need some investigation on different platforms.

Comment 5

[Alexandru Simonca [:asimonca], Release Desktop QA]

Hi Xidorn,
I've been testing Pointer Lock API and saw this issue and investigated it a little. This reproduces only on Mac and Ubuntu for me (not in Windows 10 x64) on the latest Nightly (20160831030224) and the latest Aurora (id: 20160901004001) using this tool: www.media.tojicode.com/q3bsp/ 
Please ni? me if I can be of further help.

Comment 6

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

(In reply to Xidorn Quan [:xidorn] (UTC+10) from comment #4)
> I don't think it needs to block Fx50.
> 
> This is probably not very hard to fix, but need some investigation on
> different platforms.

Gotcha. Thanks!

Comment 8

[Edgar Chen [:edgar]]

(In reply to Alexandru Simonca [:asimonca], Release Desktop QA from comment #5)

I've been testing Pointer Lock API and saw this issue and investigated it a
little. This reproduces only on Mac and Ubuntu for me (not in Windows 10
x64)

I could also reproduce this on Window if the browser window is small enough.

Comment 9

[Edgar Chen [:edgar]]

I guess the issue is because the mouse moves too fast and makes it to be outside the browser before the content process tries to reset its position. After bug 1662587, now the content process will send an IPC to the parent to request a pointer lock, and maybe we could set pinter capture in the parent process while it knows which remote target requests a pointer lock?

Comment 10

[Edgar Chen [:edgar]]

Hmm, I could even reproduce this in non-e10s mode, so even in non-e10s, we don't set the captured content correctly.
(Correct: We do set the captured content correctly in non-e10s mode.)

Comment 11

[Edgar Chen [:edgar]]

My original thought was the issue is caused by the delay of IPC, but it seems not the real root cause given that this could also be reproduced even in non-e10s.

Comment 12

[Edgar Chen [:edgar]]

I am thinking maybe we could also handle eMouseExitFromWidget to reset the mouse position, I could try to this approach.

There is another potential issue on e10s: we don't set the captured content in parent process, so if the mouse is out the XULFrame but inside the browser window, for example, on URL bar or menu bar, we are unable to handle that case.

Comment 13

[Xidorn Quan [:xidorn] UTC+10]

The issue here is that we are just repeatedly trying to move the pointer back to the center of the window, but we never really restrict the pointer within the window area.

IIRC all desktop platforms provide some kind of mouse movement restriction API, for example on macOS we may use CGAssociateMouseAndMouseCursorPosition to forbid the mouse cursor from being moved at all.

We would probably need to hook those APIs in order to fix this issue.

Comment 14

[Edgar Chen [:edgar]]

(In reply to Xidorn Quan [:xidorn] UTC+10 from comment #13)

IIRC all desktop platforms provide some kind of mouse movement restriction API, for example on macOS we may use CGAssociateMouseAndMouseCursorPosition to forbid the mouse cursor from being moved at all.

We would probably need to hook those APIs in order to fix this issue.

Thanks for the comment! I will try this in bug 1255338 which is filed to change the way we implement pointer lock.