Open Bug 1543805 Opened 6 years ago Updated 5 months ago

New Picture-in-Picture toggle is not detected when underneath a transparent iframe

Categories

(Toolkit :: Picture-in-Picture, defect, P3)

Desktop
All
defect

Tracking

()

Tracking Status
firefox-esr91 --- affected
firefox69 --- disabled
firefox70 --- wontfix
firefox71 --- wontfix
firefox72 --- wontfix
firefox73 --- fix-optional
firefox74 --- fix-optional
firefox75 --- affected
firefox98 --- affected
firefox99 --- affected
firefox100 --- affected

People

(Reporter: mconley, Unassigned)

References

(Blocks 1 open bug)

Details

(Whiteboard: [fidefe-MR1-2022])

Attachments

(6 files, 1 obsolete file)

I suspect nsIDOMWindowUtils.nodesFromRect isn't able to penetrate below iframes.

Attached file Test case (obsolete) —
Attachment #9057662 - Attachment mime type: text/plain → text/html
Attachment #9057662 - Attachment is obsolete: true
Attached file Test case
Attachment #9057663 - Attachment mime type: text/plain → text/html

Hey emilio,

Not sure if you're the right person to ask here... but are we expecting nodesFromRect and friends to not be able to include things behind a transparent iframe?

Flags: needinfo?(emilio)
Blocks: 1545872

So right now we always use IgnoreCrossDoc right now:

https://searchfox.org/mozilla-central/rev/b59a99943de4dd314bae4e44ab43ce7687ccbbec/dom/base/DocumentOrShadowRoot.cpp#260

Which means that we don't look at the internal <iframe> state. Chances are that removing that we'd actually go deep inside the iframe instead, and things would just work in this case.

But this would only be a partial fix, which would only be guaranteed to work for same-origin iframes. In a Fission world we cannot see what that iframe looks like from the content process, so there's nothing helping us there.

So I'm not sure how fixable this is generally.

Flags: needinfo?(emilio)

I don't think I asked my question correct.

I wasn't interested in elements inside of the <iframe> (and you're right - that'd be a disaster with Fission) - I'm interested in elements behind the <iframe>.

HOWEVER

This doesn't matter, because my original understanding of this bug is flawed. Either Gecko has changed, or I just didn't debug correctly the first time. The problem isn't that nodesFromRect isn't returning items behind the <iframe> - the problem is that mousemove events aren't being heard by the listener that we expect them from. Sorry about the noise!

Priority: P3 → P2

So, suppose we have the following DOM:

<body>
<video />
<iframe />
</body>

Suppose that with CSS styling, the iframe has a transparent background, and is placed over top of the video.

At that point, movemove and click events within the rectangle of the <video> will no longer dispatch to the outer document hosting the <video> - so our normal mechanism for detecting the mouse being over the video will not work.

Attaching the mousemove handler to the window root seems like an okay solution, but I don't think that's going to work long-term when <iframe>'s start moving OOP with Fission.

This feels like it's going to be a whack-a-mole. I suspect something like bug 1539652, which introduces a chrome-only API for registering for a notification when the mouse intersects the rect for a particular element, will probably be the best solution. We might need something similar for clicks too.

Priority: P2 → P3
Attached video Twitch

Reproduced on latest Nightly 71 on several twitch streams, attached video for reference.

Blocks: videopip
No longer blocks: 1527926

Build ID 20191124230652
User Agent Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:72.0) Gecko/20100101 Firefox/72.0

The behavior described in Comment 8 is reproducible also on Nightly 72 on macOS 10.13 and Linux platforms.

OS: Unspecified → All
Hardware: Unspecified → Desktop

Marking fix-optional to remove this from weekly triage.

Attached video Nightly74

Current behavior when the PiP toggle is placed higher.

Attached video Twitch_1 Nightly75.mp4

Currently PIP behavior

Comment on attachment 9127541 [details]
Twitch_1 Nightly75.mp4

Currently PIP behavior

Attached image piptube.png

Encountered this while checking with PIP and 75.0b3 on Youtube.
Same cause, right?
Thanks!

Flags: needinfo?(mconley)

(In reply to Cristian Fogel, QA [:cfogel] from comment #16)

Encountered this while checking with PIP and 75.0b3 on Youtube.
Same cause, right?
Thanks!

It depends - what is the problem you were experiencing in this case? That the toggle was still clickable even though there was a semi-transparent popup over it? If so, that's bug 1548046.

Flags: needinfo?(mconley)

Yep, that looks like it.
Thanks for the input!

See Also: → 1662814
Component: Video/Audio Controls → Picture-in-Picture
Version: unspecified → Trunk
Whiteboard: [fidefe-MR1-2022]
Blocks: 1742457

This bug and bug 1761351 may be duplicates.

See Also: → 1761351

The test case in comment 2 may be enough to confirm the reproduction of the issue, however, the PiP toggle DOES appear if the user insists on hovering the area of its appearance and IT DOES LAUNCH if he clicks it, as opposed to the behavior seen in bug 1761351.
This issue still occurs in Nightly v100.0a1 from 2022-03-24, Beta v99.0b8, Release v98.0.2 and ESR v91.7.1esr in Windows 10, Ubuntu 20 and Mac OS 11.

No longer blocks: 1742457
See Also: → 1655802
See Also: → 1705275
See Also: → 1892317
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: