1920097 - PiP should not be autotrigger in the case of short embedded videos

Status: NEW

(Toolkit :: Picture-in-Picture, defect, P3)

Description

[Daniel Bodea [:danibodea]]

Note

• When loading a web page with a short embedded autoplay video and focus a different page, the PiP triggers automatically.

Found in

• Nightly v132.0a1

Affected versions

• Nightly v132.0a1

Tested platforms

• Affected platforms: Windows 10, Ubuntu 22, MacOS 11
• Unaffected platforms: -

Steps to reproduce

1. Enable PiP auto-trigger:
   media.videocontrols.picture-in-picture.enable-when-switching-tabs.enabled = true
2. Load https://www.twicpics.com/blog/the-optimal-ways-to-embed-video-according-to-use-cases#short-autoplay-videos
3. Open a new tab.

Expected result

• A PiP should not auto-trigger for the short embedded video.

Actual result

• PiP will auto-trigger for the short embedded autoplay video.

Regression range

• not a regression, but an issue since the implementation of PiP auto-trigger.

Comment 1

[Mike Conley (:mconley) (:⚙️)]

Attachment: WIP: Bug 1920097 - Use toggle heuristics for autoPiP feature as well.

Comment 2

[Mike Conley (:mconley) (:⚙️)]

Hey padenot,

You had mentioned that there were some heuristics that we could potentially use to more precisely target when it's appropriate to apply the autoPiP feature. In the patch I just posted, I call into the pre-existing logic that we use for offering the toggle.

Besides the following:

1. Checking that the length exceeds our minimum 45s threshold
2. Checking that the video width or height exceeds 140 pixels
3. Checking that the video has an audio track

Can you propose any additional heuristics that might be useful here?

Comment 3

[Paul Adenot (:padenot)]

Here are some ideas, including some things that are best answered by Gecko and not always surfaced easily. If we want to experiment, we can plumb the information through in a way that's easily consumed by the code making the decision about auto-pipping or not:

• The video has almost ended (switching tab during the credits etc.), we probably don't want to pop it out
• The video is looping -- same deal, there's high chance it's a gif-style video. There's probably something to explore around allowing looping once for longer videos that loop (e.g. tiktok / ig reel / yt short type content)
• The video was seemingly watched for some time, it's plausible that we want to pop it out: it was front and center on the focused page and there was no kb/mouse activity for that time. We can simply look at painted frames vs. decoded frames, etc., there, and we have extensive visibility checks and respective duration (invisible duration, visible duration, etc.) in the engine already. Layout can tell us if it was side content or main content but it is likely that this will require some tuning
• The video is heavily overlaid by other things, don't pop it out. This is probably best answered by layout as well, but we don't want to pop out "background video" type stuff. This might be caught by the 45s threshold but maybe not
• The video isn't playing regular media (webrtc, etc.), no automatic pop out
• The video has an audio track but is muted (this takes care of auto-playing muted videos and lots of ads), probably not pop it out, but I could also see users wanting the opposite, especially if there are subtitles. I see that sometimes on news websites, they have a video bottom right with a broadcast, muted + autoplay, but are otherwise a news site.