1539652 - Add a privileged API to listen for mouseenter and mouseexit-like events for DOM nodes that might have other elements overtop capturing normal events

Status: NEW

(Core :: DOM: Events, enhancement, P3)

Description

[Mike Conley (:mconley) (:⚙️)]

In bug 1535354, I've got a work-in-progress patch that does some pretty harebrained stuff to know when a mouse is over a <video>, in order to show a toggle for Picture-in-Picture.

This is because popular video sites like Vimeo and Twitch oftentimes place elements over top of <video> elements in order to capture clicks. The underlying video never receives mouse events in that case.

See bug 1535354 comment 1 for my harebrained current mechanism - essentially, I'm using mousemove and a non-flushing version of elementsFromPoint.

What I'd love is if I could do something like this instead from privileged script:

for (let video of videos) {
  video.addEventListener("specialmouseenter", this);
  video.addEventListener("specialmouseleave", this);
}

And then the Event system will know that I care about hearing when the mouse cursor enters the rects of those elements - even if there's a bunch of junk on top of them.

Speaking with smaug in IRC, I suspect this would be far more efficient and less glitchy than what I've currently got.

Comment 1

[Mike Conley (:mconley) (:⚙️)]

Out of curiosity, how hard would this be to support, smaug? I've found WidgetQueryContentEvent which seems related, but it's not clear to me if what I'm asking for is a day's work, or a months work, or something in between.

Comment 2

[Olli Pettay [:smaug][[e-mail]]]

(WidgetQueryContentEvent is for IME, a bit different.)

Having some special event shouldn't be too hard. But how is this supposed to work if the video really is overlapped by opacity:1 content or some such?

Comment 3

[Mike Conley (:mconley) (:⚙️)]

(In reply to Olli Pettay [:smaug] from comment #2)

But how is this supposed to work if the video really is overlapped by opacity:1 content or some such?

I think for now, I still want to hear about it.

Having some special event shouldn't be too hard.

Music to my ears! Are you the right person to talk to about making that happen? Or is there somebody else who might be able to help me? I don't want to add more to your plate if I can help it.

Comment 4

[Mike Conley (:mconley) (:⚙️)]

(In reply to Mike Conley (:mconley) (:⚙️) from comment #3)

(In reply to Olli Pettay [:smaug] from comment #2)

But how is this supposed to work if the video really is overlapped by opacity:1 content or some such?

I think for now, I still want to hear about it.

Having dealt with a few of these bugs for the past few weeks, I think I can refine this a bit more:

We're going to want to hear about the mouse intersecting that Element's rect even if an <iframe> is "between" the rect and the mouse cursor, where normally the parent document wouldn't hear about mousemove events.

So ultimately, I want to be able to pass an Element to this API, and have that API monitor the position of an Element, and then be notified if the mouse cursor enters or exits that rect regardless of what's over top of that rect. The API will also need to know if the element moves.

Comment 5

[Mike Conley (:mconley) (:⚙️)]

Attachment: Test case

Comment 6

[Mike Conley (:mconley) (:⚙️)]

In the test case I just posted, notice how the outer box turns green when the mouse enters it (via a mouseenter event), but yellow again when the mouse enters the iframe. This is because the mouseleave event fires, despite the fact that the cursor is still within the bounds of the outer element that has the event listeners on it.

Comment 7

[Mike Conley (:mconley) (:⚙️)]

I suspect to fix bug 1561308 "properly", we're going to need this. :/

Hey smaug, roughly how difficult would it be to introduce this kind of privileged API? Would you be able to coach me through it?

Comment 8

[Olli Pettay [:smaug][[e-mail]]]

Ok, iframes may make this a lot harder, especially in Fission world.
Need to move the hit testing to parent process then.

Or perhaps not a lot harder, but different. You may want to ask kats or botond too.
I could imagine us to be able to add some 'marker' which parent process' hit testing finds, even if it is covered by other layers.