1598654 - Picture-in-picture video becomes low quality when originating tab is in the background

Status: RESOLVED WORKSFORME

(Toolkit :: Picture-in-Picture, defect, P3)

Description

[Andreas Pehrson [:pehrsons]]

STR:
1 Play a video from https://www.svtplay.se/, e.g., https://www.svtplay.se/video/21895142/al-pitcher-pa-paus/al-pitcher-pa-paus-sasong-1-avsnitt-4
2 Enable picture-in-picture
3 Change to another tab so the originating video is in the background
4 Wait for a bit, seems to trigger after ~2.5 minutes

Expected:
Video quality remains as sharp as before

Actual:
Video quality degrades

5 Switch back so the originating video's tab is in the foreground

Actual:
Video quality more or less instantly improves

Comment 1

[Mike Conley (:mconley) (:⚙️)]

I suspect there are a number of sites that lower video quality / quality of service when the tab is not visible.

We should probably make it so that the originating tab thinks it's still in the foreground.

Comment 2

[Dennis Schubert [:denschub]]

Just for the record: This is an issue when watching Twitch as well. Given Twitch is rather large, I figured I'd mention it explicitly, in case this fact has an influence on the priority. :)

Comment 3

[Jean-Yves Avenard [:jya]]

I wonder what criteria the site could use: do they check that the entire page is in the background, or can they check if just the media element is.
I'm unsure.

Comment 4

[Alastor Wu [:alwu]]

If we stop sending the visibilitychange event for document, then this issue won't happen on svtplay.se. So apparently that site would automatically lower their video quality after the page becomes invisible after a while.

But not to dispatch visibilitychange won't be a solution for us. I think there are two things we can do, (1) using an extension to block visibilitychange for those sites, and ask people to install that if they get this problem. We've done the similar way for Fennec before which was used to prevent Youtube stopping media playback when the page is on the background. (2) discuss if we could have new API to let website know that the part of page is still visible via PIP mode or discuss how to modify the spec of visibility change to let it fit in with PIP mode.

---

According to [1], The Document is at least partially visible on at least one screen, so it seems to me that if the video is still showing via PIP mode, then the document should be considered as visible. So I would move this bug to DOM to see what do they think.

[1] https://www.w3.org/TR/page-visibility/#visibility-states-and-the-visibilitystate-enum

Comment 5

[Olli Pettay [:smaug][[e-mail]]]

Tweaking visibilitychange event firing doesn't sound totally unreasonable to me.
How would one know if there is a PIP video for the document?

The spec link is obsolete. Never use /TR/ specs, if there is something newer available.
Latest one is https://w3c.github.io/page-visibility

The spec contradicts itself a bit
https://w3c.github.io/page-visibility/#dfn-visibility-state
visible: The Document is at least partially visible on at least one screen.

https://w3c.github.io/page-visibility/#dom-document-visibilitystate
Return "visible" if: Any of the doc's viewport contents are observable to the user

First part talks about any parts of the Document, latter only about viewport. What does viewport even mean in PIP case?

How does this work in other browsers? Do they change visibilitychange handling when there is an active PIP?

Comment 6

[Emilio Cobos Álvarez (:emilio)]

An issue with that is that we also use the page visibility to throttle animation frame callbacks and so on, so we'd be running the page at full speed if we consider it visible...

Comment 7

[Olli Pettay [:smaug][[e-mail]]]

sure. That might not be too unexpected.
But we could also internally still consider the tab to be hidden. That might complicate the implementation a bit, hopefully not too badly.
Anyhow, consistency with other browsers is rather important here.

Comment 8

[Mike Conley (:mconley) (:⚙️)]

Attachment: Bug 1598654 - Keep Picture-in-Picture originating tab rendering. r?dthayer!

Comment 9

[Alastor Wu [:alwu]]

Thank you for explanation. So it sounds like we should use other internal step to fix this issue, not change the page visibility.

Comment 11

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Comment 5 and comment 9 seem to draw different conclusions.

Return "visible" if: Any of the doc's viewport contents are observable to the user

The way I read this, a part of the doc's viewport's contents is clearly observable through PiP.

Therefore, treating a tab with active PiP as in the foreground for the duration of the PiP seems not just like a reasonable argument, but perhaps the most web compatible option, since Firefox's PiP is otherwise totally UA driven, and it would otherwise be hard to fault a web page for pausing when it learns it's no longer visible. Detecting situations like this seems like the point of the visibility API.

How does this work in other browsers? Do they change visibilitychange handling when there is an active PIP?

AFAIK other browsers do not offer PiP. It looks like Chrome has a plug-in. I haven't played with it or checked how it deals with visibilitychange.

Comment 12

[Andreas Pehrson [:pehrsons]]

(In reply to Jan-Ivar Bruaroey [:jib] (needinfo? me) from comment #11)

AFAIK other browsers do not offer PiP. It looks like Chrome has a plug-in. I haven't played with it or checked how it deals with visibilitychange.

Chrome has the PiP API. The MDN page for that API has an example video where PiP can be toggled. Having that PiP window visible and a simple visibilitychange observer on the page document.addEventListener("visibilitychange", () => console.log(document.visibilityState)); shows that they do change visibilityState when switching tabs while PiP is active.

Comment 13

[Mike Conley (:mconley) (:⚙️)]

This should be fixed in Nightly 100 now with bug 1755748 fixed. pehrsons, can you confirm?

Comment 14

[Andreas Pehrson [:pehrsons]]

I couldn't reproduce this in a pre-100 version, including the Nightly from when I filed this. I'll attribute thid to svtplay changes.
Maybe we could verify with Twitch instead, per comment 2.

Comment 15

[Ania]

I tested this with a few Twitch videos on macOS 11.6 and Windows 10 in the Nightly 100.0a1 (2022-03-17), and can no longer reproduce it. Twitch videos played with the originating tab in the background for 3-10 minutes without losing quality.