1770429 - Picture on picture subtitles don't update unless firefox is visible

Status: VERIFIED FIXED

(Toolkit :: Picture-in-Picture, defect, P1)

Description

[luis.pabon]

Attachment: broken recording please ignore

Steps to reproduce:

In sway 1.7 (wayland), Ubuntu 22.04 and FF nightly 102.0a1 (2022-05-18)

1. Open firefox nightly
2. Go to youtube, open a vid like this one https://www.youtube.com/watch?v=TWP0YpfvGvM and enable closed captions
3. Activate PIP
4. Open some other app side by side on same workspace and give it focus
5. Subtitles still work
6. Switch to tabbed mode and ensure the main firefox window is not visible
7. Subtitles don't update anymore
8. Switch to the tab with firefox, making it visible
9. Subtitles now catch up

I've uploaded a screengrab showing just this scenario.

Actual results:

Subtitles on PIP window stop updating if the main FF window is not visible

Expected results:

Subtitles continue to update

Comment 1

[luis.pabon]

The issue is also present on FF beta (101).

I am unable to turn off the wayland backend in Firefox and make it run in Xwayland though to check it's a wayland thing or what. Unsetting MOZ_ENABLE_WAYLAND then launching firefox doesn't seem to have an effect and it still loads in wayland mode.

Comment 2

[luis.pabon]

Attachment: recording.m4v

For some reason the first vid I uploaded just won't play on firefox with a mp4 mime type. Atttaching the same thing as m4v

Comment 3

[Release mgmt bot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Toolkit::Picture-in-Picture' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 4

[Molly Howell (she/her) [:mhowell]]

Hmm. This looks like it ought to be the same thing as bug 1755748, but that one only fixed the tab being blurred, not the whole app. Mike, do you know if there's something similar we could do at the app level that might address this, or is it not going to be that easy?

Comment 5

[Mike Conley (:mconley) (:⚙️)]

Hm... this might be related to window occlusion handling in AsyncTabSwitcher. The platform fires "window occlusion" changes. We handling occlusion state changes here: https://searchfox.org/mozilla-central/rev/1c391443f770eddc7cde9e52dba5ef50cc233c06/browser/modules/AsyncTabSwitcher.jsm#855-857

and we do seem to try to handle the PiP case in shouldDeactivateDocShell here: https://searchfox.org/mozilla-central/rev/1c391443f770eddc7cde9e52dba5ef50cc233c06/browser/modules/AsyncTabSwitcher.jsm#926

If this is easily reproducible (and I expect it is), it'd be worth checking to see if that's working properly. If it is, we might have an issue deeper in the platform.

Comment 6

[luis.pabon]

I can reproduce it every time on my system.

Comment 9

[kpatenio]

Attachment: Bug 1770429 - set PiP originating window as active when window is occluded

Comment 11

[pokechu022]

Based on my experience on bug 1810380, it seems like if the issue also doesn't happen if the tab is not selected (i.e. if the tab with the video is not the selected tab, it behaves correctly). So switching to a different tab is a workaround. I'm not sure if that's consistent with the patch or not.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1da22d92c592
set PiP originating window as active when window is occluded r=mconley,pip-reviewers

Comment 13

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/1da22d92c592

Comment 14

[Bodea Daniel [:danibodea]]

This fix has been tested on Windows 7, 10, 11, Ubuntu 22 and Mac OS 11 and 12 with Nightly v111.0a1 from 2022-02-02.

We've tested the following scenarios:

1. Open a new profile.
2. Load 2 youtube videos with subs:
   2.a generic/loaded subs
   2.b auto-generated subs
3. Launch PiP and activate subs
4. minimize the window and check that subs are not delayed in PiP
5. put another window over the Fx window and check that subs are not delayed in PiP
6. send the PiP/ main window to another desktop and check that subs are not delayed in PiP
7. focus another tab on the Fx browser and check that subs are not delayed in PiP

All scenarios and operating systems suggest that the issue was fixed except for one situation: on Mac OS (both 11 and 12) the subtitles are still displayed with a delay if the main window is minimized (and only in this situation of the ones tested).

This being said, this report is partially fixed. Considering that the fix appears to be complete on the other operating systems, I suggest we close this report, reopen duplicate bug 1786258 and attempt to fix the remaining issue there. If you think this is an acceptable procedure, let me know to make the necessary changes. Thank you!

Comment 15

[kpatenio]

Thanks :danibodea - I agree, let's use Bug 1786258 for MacOS.

Comment 16

[Bodea Daniel [:danibodea]]

I will verify this report based on comments 14 and comment 15 and reopen bug 1786258.