1827583 - youtube.com - Media playback not detected, breaking media notifications, media controls, fullscreen landscape, background playback and PiP.

Status: VERIFIED FIXED

(GeckoView :: Media, defect, P1)

Description

[Calin Tanase]

From github: https://github.com/webcompat/web-bugs/issues/120786.

Note: Feel free to move the issue if the Component/Product is wrong.

<!-- @browser: Firefox Mobile 111.0 -->
<!-- @ua_header: Mozilla/5.0 (Android 13; Mobile; rv:109.0) Gecko/111.0 Firefox/111.0 -->
<!-- @reported_with: unknown -->
<!-- @public_url: https://github.com/webcompat/web-bugs/issues/120786 -->

URL: http://youtube.com

Browser / Version: Firefox Mobile 111.0
Operating System: Android 13
Tested Another Browser: Yes Other

Problem type: Something else
Description: picture-in-picture is enabled for Firefox android app, but PiP functionality stopped working after having used it successfully earlier today
Steps to Reproduce:
I've had picture-in-picture mode enabled for several months with zero issues and use it frequently. today, after previously having used picture-in-picture successfully earlier, it suddenly no longer works for Firefox on Android. I'll put a video in fullscreen but when I click away from the app it goes to running in the background with audio only, as if I hadn't put the video in fullscreen mode

<details>
<summary>Browser Configuration</summary>
<ul>
<li>None</li>
</ul>
</details>

From webcompat.com with ❤️

Change performed by the Move to Bugzilla add-on.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Since nightly and release are affected, beta will likely be affected too.
For more information, please visit auto_nag documentation.

Comment 2

[Kestrel]

PiP not working is a symptom of media playback not being detected which also breaks media notifications, media controls and background playback. It is reproducible on Fenix 97 so it is not related to any recent updates. Clearing all cookies or using private tabs will sometimes get it working.

Comment 7

[BugBot [:suhaib / :marco/ :calixte]]

The product::component has been changed since the backlog priority was decided, so we're resetting it.
For more information, please visit auto_nag documentation.

Comment 8

[robsmith11]

For anyone else searching, this bug seems to be the reason why the "Video Background Play Fix" is no longer working with YouTube.

Comment 9

[Chris Peterson [:cpeterson]]

This bug sounds like a regression in Firefox 111 (or earlier).

Comment 10

[Chris Peterson [:cpeterson]]

QA, is this bug reproducible in the GeckoView Example app? If so, can you please try to bisect the regression using mozregression?

Comment 11

[Chris Peterson [:cpeterson]]

Jonathan says that he recently saw this bug in Focus and the Reference Browser, so we know this is not a Fenix-specific bug.

Comment 12

[Jonathan Almeida [:jonalmeida]]

Can anyone here confirm this is happening on a non-Youtube site as well? Rob maybe?

Comment 13

[gkulcsar]

I can reproduce it on Youtube, Google Podcasts, srf.ch/audio. Can't reproduce it on SoundCloud and some more exotic (local, non-English language) sites.

Comment 14

[Jan Henning [:JanH]]

Could bug 1796359 be in any way related? (Not that it has much more clues about why and what, but for completeness's sake…)

Comment 17

[Kestrel]

It also breaks landscape orientation in fullscreen mode, instead staying in portrait orientation (Bug 1828346).

Comment 18

[miralobontiu]

This issue is reproducible on youtube.com, on the latest Fenix Nightly 114.0a1 from 4/20, and Beta 113.0b5. Tested with Xiaomi Mi8 Lite (Android 10), and Google Pixel 6 (Android 13):

• PiP is working properly on vimeo.com, cnn.com, bbc.com.
• It is not reproducible on RC 112.1.0 on the Pixel device.
• It is reproducible on RC 112.1.0 on the Xiaomi device.
• In the Geckoview Example app the PiP mode is not supported.

Comment 19

[Ryan VanderMeulen [:RyanVM]]

This smells more like a change on the YouTube end of it's reproducing in older releases now that were previously OK. Have we tried reaching out on the discuss list yet?

Comment 20

[Chris Peterson [:cpeterson]]

(In reply to Ryan VanderMeulen [:RyanVM] from comment #19)

This smells more like a change on the YouTube end of it's reproducing in older releases now that were previously OK. Have we tried reaching out on the discuss list yet?

Not yet, but I'm working with Jonathan and the webcompat team to create a summary of the problem to send to YouTube.

Comment 21

[Chris Peterson [:cpeterson]]

Though comment 13 says this bug is also reproducible on Google Podcasts and https://srf.ch/audio, but perhaps they all reuse some common Google library?

Comment 22

[Jan Henning [:JanH]]

Though is the e.g. srf.ch issue a fresh one, or is it just that nobody ever reported it until now?

Comment 23

[Jonathan Almeida [:jonalmeida]]

(In reply to gkulcsar from comment #13)

I can reproduce it on Youtube, Google Podcasts, srf.ch/audio. Can't reproduce it on SoundCloud and some more exotic (local, non-English language) sites.

To add to JanH's question, could you try an older build of Fenix like version 102.2.1 from the list of previous releases: https://archive.mozilla.org/pub/fenix/releases/

I suggested 102.2.1 because we've also received reports of this bug starting to happen from a third party consider of GeckoView who are currently using this version

Comment 24

[Jonathan Almeida [:jonalmeida]]

With a debug build of Fenix 113, I'm able to see the broken and working states inconsistently with Youtube. When initially in a working state, I was able to clear app data and try again to see the broken state (no media events).

Comment 25

[eclaudiu64]

I also noticed this error on websites that do not belong to youtube. The problem seems to be general in many streaming players. I noticed the problem on the following site, selecting the StreamSB server: https://www1.divxfilmeonline.net/superwho-2021/ and on this website I noticed these errors: https://rds.live/travel-channel/
But after I clear the app data it goes back to normal and I can put the video in full screen auto and pip.

Comment 26

[gkulcsar]

(In reply to Jonathan Almeida [:jonalmeida] from comment #23)

(In reply to gkulcsar from comment #13)

I can reproduce it on Youtube, Google Podcasts, srf.ch/audio. Can't reproduce it on SoundCloud and some more exotic (local, non-English language) sites.

To add to JanH's question, could you try an older build of Fenix like version 102.2.1 from the list of previous releases: https://archive.mozilla.org/pub/fenix/releases/

I suggested 102.2.1 because we've also received reports of this bug starting to happen from a third party consider of GeckoView who are currently using this version

I can confirm that installing 102.2.1 fixes the problem with Youtube and srf.ch!
I also checked a Google Podcast, started playback, media controls were not shown first. Then I stopped playback, switched to another tab, switched back, restarted playback and the media controls appeared.
HTH

Comment 27

[gkulcsar]

I can confirm that installing 102.2.1 fixes the problem with Youtube and srf.ch!
I also checked a Google Podcast, started playback, media controls were not shown first. Then I stopped playback, switched to another tab, switched back, restarted playback and the media controls appeared.
HTH

Scrap that. Once I'm logged in (to Firefox and to Google / Youtube), installed uBO and the background playback fix addons, the problem is there again.

Comment 28

[BugBot [:suhaib / :marco/ :calixte]]

The bug is marked as tracked for firefox112 (release), tracked for firefox113 (beta) and tracked for firefox114 (nightly). We have limited time to fix this, the soft freeze is in 10 days. However, the bug still isn't assigned.

:cpeterson, could you please find an assignee for this tracked bug? If you disagree with the tracking decision, please talk with the release managers.

For more information, please visit auto_nag documentation.

Comment 29

[Jonathan Almeida [:jonalmeida]]

(In reply to gkulcsar from comment #27)

I can confirm that installing 102.2.1 fixes the problem with Youtube and srf.ch!
I also checked a Google Podcast, started playback, media controls were not shown first. Then I stopped playback, switched to another tab, switched back, restarted playback and the media controls appeared.
HTH

Scrap that. Once I'm logged in (to Firefox and to Google / Youtube), installed uBO and the background playback fix addons, the problem is there again.

Could you possibly test after each of those steps so we can narrow down what state change it might be that is causing this for you? e.g. is it signing-in, uBO, background playback addon.

Comment 30

[Chris Peterson [:cpeterson]]

I asked our YouTube contacts if YouTube made any changes that might have caused these problems. They are investigating.

Comment 31

[Andrew]

Possible related issue: around the same time this appeared, playback became quite compromised at higher speeds when the screen is off. At 1.0 speed, or with firefox open, there are no audio issues. At 1.5 speed with firefox minimized (or even the lock screen on), massive audio clipping is introduced. This applies across multiple bluetooth headsets, and is an issue that I remember cropping up for a few months some years back.

Comment 33

[Ilya Basin]

Meanwhile it works in embedded youtube, at least for me. Try this web page https://basinilya.github.io/wisdom/yt-embed2.html

Comment 34

[BugBot [:suhaib / :marco/ :calixte]]

This is a reminder regarding comment #28!

The bug is marked as tracked for firefox112 (release), tracked for firefox113 (beta) and tracked for firefox114 (nightly). We have limited time to fix this, the soft freeze is in 3 days. However, the bug still isn't assigned.

Comment 35

[[:owlish] 🦉 PST]

Attachment: passing-no-fission.txt

Comment 36

[[:owlish] 🦉 PST]

Comment on attachment 9331005 [details]
passing-no-fission.txt

disregard, added by mistake

Comment 37

[Jonathan Almeida [:jonalmeida]]

I've been able to reproduce this in Focus 112.0.2-20230424110519 by only starting a new private browsing session a few times to get into the broken and working states.

It's easy to know if you're in a buggy state: you don't see a media notification when a video is playing.

Since the STR are not consistent, here are two media profiles with the same video:

No media events are received; four interactions (pause, play, fullscreen, fullscreen exit): https://share.firefox.dev/3nk8tTw
Media events are received; four interactions (pause, play, fullscreen, fullscreen exit): https://share.firefox.dev/3LIaMYP

Comment 38

[BugBot [:suhaib / :marco/ :calixte]]

This is a reminder regarding comment #28!

The bug is marked as tracked for firefox112 (release), tracked for firefox113 (beta) and tracked for firefox114 (nightly). We have limited time to fix this, the soft freeze is in 2 days. However, the bug still isn't assigned.

Comment 39

[Jonathan Almeida [:jonalmeida]]

denschub, if you get the time, could you possibly take a look at the profiles in comment 37 or pass it along to someone else to confirm that media events are being received?

Comment 40

[BugBot [:suhaib / :marco/ :calixte]]

This is a reminder regarding comment #28!

The bug is marked as tracked for firefox113 (beta) and tracked for firefox114 (nightly). We have limited time to fix this, the soft freeze is in a day. However, the bug still isn't assigned.

Comment 41

[Sergio Villar]

I've checked again some minutes ago and I'm now again receiving the media events in Wolvic. Can anyone confirm that it is the same for Fenix?

Comment 42

[mehmet.gunacti]

(In reply to Sergio Villar from comment #41)

I've checked again some minutes ago and I'm now again receiving the media events in Wolvic. Can anyone confirm that it is the same for Fenix?

Just tried on the latest FF Nightly (114.0a1 B. 2015948515), opened and started playing a video on m.youtube.com, minimized to homescreen, and still no notification showed up.

Comment 44

[Alastor Wu [:alwu]]

I can reproduce this issue on Android Nightly (115.0a1, Build #2015951011) as well, but not on the Android Firefox Release (113.1.0). In addition, I can't reproduce this issue if I use the default html player, eg. this. When using that player, I can see the media notification, and the PiP also works as expected.

(In reply to Jonathan Almeida [:jonalmeida] from comment #37)

It's easy to know if you're in a buggy state: you don't see a media notification when a video is playing.

From what I understand, the media statuses are exposed to GeckoView by using this interface. There is a middle layer useing this GeckoViewMediaControl to propagate events to GeckoView. And this media session in GeckoView would further notify those events to the application (eg. Fenix).

As I mentioned above, on Youtube, I can't see the media notification, but I can see the media notification on my test player. I can confirm that in both cases, Gecko dispatched media status events correctly (youtube case / default html player), which indicates the issue might happen on GeckoView side, or the Fenix side.

Comment 45

[Jonathan Almeida [:jonalmeida]]

(In reply to Alastor Wu [:alwu] from comment #44)

(In reply to Jonathan Almeida [:jonalmeida] from comment #37)

It's easy to know if you're in a buggy state: you don't see a media notification when a video is playing.

From what I understand, the media statuses is exposed to GeckoView by using this interface. There is a middle layer useing this GeckoViewMediaControl to propagate events to GeckoView. And this media session in GeckoView would further notify those events to the application (eg. Fenix).

That is correct. When NOT in the broken state, I am able to see the debugger hit points in the media session layer in GeckoView, and when in the broken state, that no longer happens. I would deduce that that particular layer is working.

For the GeckoViewMediaControl.sys.mjs, the changelog shows the module was converted to an ESM module in 2023-03-17, and before that there have been no changes to that since 2021-12-11 which might be suspect, but from comment 27, it seems like the bug was still appearing in 102 all of a sudden (after account login and cookie/storage changes).

Nothing in the MediaController.webidl logs look suspicious or recent either.

(clearing old NI too)

Comment 46

[BugBot [:suhaib / :marco/ :calixte]]

This is a reminder regarding comment #28!

The bug is marked as tracked for firefox113 (release) and tracked for firefox114 (beta). We have limited time to fix this, the soft freeze is in 14 days. However, the bug still isn't assigned.

Comment 47

[BugBot [:suhaib / :marco/ :calixte]]

This is a reminder regarding comment #28!

The bug is marked as tracked for firefox113 (release) and tracked for firefox114 (beta). We have limited time to fix this, the soft freeze is in 13 days. However, the bug still isn't assigned.

Comment 48

[Makoto Kato [:m_kato]]

When this issue occurs, since MediaController is never activated, MediaSessionDelegate won't be used.

alwu, why doesn't MediaController dispatch activated event?

Comment 49

[Alastor Wu [:alwu]]

We did send activated event for the media controller, see this profiled result.

Comment 50

[Makoto Kato [:m_kato]]

(In reply to Alastor Wu [:alwu] from comment #49)

We did send activated event for the media controller, see this profiled result.

I guess that YouTube has published another mobile version.

When I test it last week, no activate event.

05-20 01:41:30.001 21780 21815 I Gecko   : [Child 21780: Main Thread]: D/MediaControl HTMLMediaElement=79f59934c0, NotifyMediaState from state='paused' to state='played'
05-20 01:41:30.001 21780 21815 I Gecko   : [Child 21780: Main Thread]: D/MediaControl ContentMediaController=79e6eb3e40, Notify media played in BC 5
05-20 01:41:30.001 21780 21815 I Gecko   : [Child 21780: Main Thread]: D/MediaControl ContentMediaController=79e6eb3e40, Notify media became audible in BC 5
05-20 01:41:30.006 21727 21759 I Gecko   : [Parent 21727: Main Thread]: D/MediaControl MediaStatusManager=79e5c4f190, UpdateMediaPlaybackState played for context 5
05-20 01:41:30.006 21727 21759 I Gecko   : [Parent 21727: Main Thread]: D/MediaControl MediaPlaybackStatus=79e5c4f298, Update playback state 'played' for context 5
05-20 01:41:30.006 21727 21759 I Gecko   : [Parent 21727: Main Thread]: D/MediaControl MediaStatusManager=79e5c4f190, SetGuessedPlayState : 'playing'
05-20 01:41:30.006 21727 21759 I Gecko   : [Parent 21727: Main Thread]: D/MediaControl MediaStatusManager=79e5c4f190, UpdateActualPlaybackState : 'playing'
05-20 01:41:30.006 21727 21759 I Gecko   : [Parent 21727: Main Thread]: D/MediaControl MediaController=79e5c4f100, Id=5, Cancel deactivation timer
05-20 01:41:30.006 21727 21759 I Gecko   : [Parent 21727: Main Thread]: D/MediaControl MediaPlaybackStatus=79e5c4f298, Update audible state 'audible' for context 5
05-20 01:41:30.006 21727 21759 I Gecko   : [Parent 21727: Main Thread]: D/MediaControl AudioFocusManager=79e68b8510, Controller 5 grants audio focus
05-20 01:41:30.006 21727 21759 I Gecko   : [Parent 21727: Main Thread]: I/MediaControl MediaControlKeyManager=79e68b8200, playbackState=playing
05-20 01:41:30.006 21727 21759 I Gecko   : [Parent 21727: Main Thread]: D/MediaControl MediaController=79e5c4f100, Id=5, Dispatch event playbackstatechange
05-20 01:41:30.027 21780 21815 I Gecko   : [Child 21780: Main Thread]: D/MediaControl ContentMediaController=79e6eb3e40, Notify declared playback state  'playing' in BC 5
05-20 01:41:30.027 21727 21759 I Gecko   : [Parent 21727: Main Thread]: D/MediaControl MediaStatusManager=79e5c4f190, SetDeclaredPlaybackState from paused to playing

But when I test this today, I cannot reproduce this. As long as I test, I could reproduce this on last week's m.youbute.com.

Does anyone have reproduced site now? Jonathan?

Comment 51

[Jonathan Almeida [:jonalmeida]]

Yes, I can still reproduce this with Focus 113.0 by going to the same YouTube video and cleearing the session until the bug reproduces. You can do this with Fenix private browsing mode too as it's quicker that way than clearing cache in a normal browsing session.

Comment 52

[Kestrel]

Youtube is intermittent (90% fails) but it is always reproducible on podcasts.google.com.

Comment 53

[Makoto Kato [:m_kato]]

Thanks, Janothan and Kestrel.

I guess that MediaController is re-created after we enable MediaSessionDelegate. It means that we don't listen current controller.

Comment 54

[Pascal Chevrel:pascalc]

We are in our last week of beta, but given the number of duplicates and the fact that it impacts a top site, I would take an uplift in our RC next week or in our planned dot release.

Comment 55

[Makoto Kato [:m_kato]]

GeckoViewMediaController doesn't handle process switching. So if it occurs, the browser is switched. Then, MediaController (browser.browsingContext.mediaController) is re-created and GeckoViewMediaController doesn't attach newer one.

So I think that we has to handle this event (ex. onInitBrowser).

Comment 56

[Makoto Kato [:m_kato]]

https://treeherder.mozilla.org/jobs?repo=try&revision=c45094dfa87c5c0ef2b51a3e4fb8350c100258fe

Comment 57

[Sergio Villar]

(In reply to Makoto Kato [:m_kato] from comment #55)

GeckoViewMediaController doesn't handle process switching. So if it occurs, the browser is switched. Then, MediaController (browser.browsingContext.mediaController) is re-created and GeckoViewMediaController doesn't attach newer one.

So I think that we has to handle this event (ex. onInitBrowser).

Of course the bug needs to be fixed, but I wonder if process switching is something that could be disabled via about: flags.

Comment 58

[Makoto Kato [:m_kato]]

Attachment: Bug 1827583 - Set event listener to current media control after switching process. r=#geckoview-reviewers!

This issue is a timing issue of loading content at first time.

Although GeckoViewMediaControl registers event listeners for
MediaController when enabling MediaSessionDelegate, if process
switching occurs (content is loaded later at first time), browsingContenxt
and MediaController are re-created.

It means that we has to register newer MediaController.

To detect destroying current browser, I will add onDestroyBrowser to
GeckoViewModule again (This was removed at bug 1672262).

Comment 59

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fcef8b674ed2
Set event listener to current media control after switching process. r=geckoview-reviewers,amejiamarmol,jonalmeida

Comment 60

[Makoto Kato [:m_kato]]

(In reply to Sergio Villar from comment #57)

(In reply to Makoto Kato [:m_kato] from comment #55)

GeckoViewMediaController doesn't handle process switching. So if it occurs, the browser is switched. Then, MediaController (browser.browsingContext.mediaController) is re-created and GeckoViewMediaController doesn't attach newer one.

So I think that we has to handle this event (ex. onInitBrowser).

Of course the bug needs to be fixed, but I wonder if process switching is something that could be disabled via about: flags.

Unfortunately, GeckoView doesn't support no-e10s mode. It depends on e10s (This occurs on fission and non-fission too)

Comment 61

[Sandor Molnar]

https://hg.mozilla.org/mozilla-central/rev/fcef8b674ed2

Comment 62

[Makoto Kato [:m_kato]]

Comment on attachment 9336031 [details]
Bug 1827583 - Set event listener to current media control after switching process. r=#geckoview-reviewers!

Beta/Release Uplift Approval Request

• User impact if declined: When video becomes fullscreen mode, Firefox doesn't set correct orientation (If video is landscape, Firefox should be landscape) and doesn't set screen wakelock. Also, it means that this cannot enter PiP screen mode even if fullscreen.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Low. This changes that we add event listener when "gBrowser" is created, instead of enabling Media module.

This issue is Android only. No effect for Desktop.

• String changes made/needed:
• Is Android affected?: Yes

Comment 63

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:m_kato, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
• If no, please set status-firefox114 to wontfix.

For more information, please visit BugBot documentation.

Comment 64

[Makoto Kato [:m_kato]]

beta cycle is ended, so I am already requesting to uplift this to 114.

Comment 65

[Pascal Chevrel:pascalc]

Comment on attachment 9336031 [details]
Bug 1827583 - Set event listener to current media control after switching process. r=#geckoview-reviewers!

Low risk and high value (multiple duplicates, affects a top site), patch approved for our RC3

Comment 66

[Pascal Chevrel:pascalc]

https://hg.mozilla.org/releases/mozilla-release/rev/e72fea541812

Comment 67

[Delia Pop]

Verified fixed on Nightly 115.0a1, latest Nightly 116.0a1 from 06/06 and RC 114.0 build 3 on Focus and Fenix with Samsung Galaxy A53 5G (Android 13) and Lenovo tablet M10 (Android 10).