1565689 - centralized media control service in chrome process

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P2)

Description

[Alastor Wu [:alwu]]

In the past, we use AudioChannelService to manage all media within one process, we did play, pause, stop those media and managed audio focus all in AudioChannelService.

However, since e10s, we have had multiple content processes and will have more in Fission. AudioChannelService no longer has an ability to manage media together which might started in different processes.

Therefore, we will implement a new service in chrome process which can manage media among different processes. In the first step, we would start from supporting play, pause and stop which we already have supported on Android by using the media control interface or managing audio focus among tabs.

In the future, we can extend the service to support hardware media key, using media key to control all (or one) tabs at the same time, or support something like having a media control in desktop. Like what Chrome did [1].

[1] https://www.theverge.com/2019/7/6/20684353/google-chrome-global-media-controls-play-button-pause-experimental-test-canary

Comment 1

[Alastor Wu [:alwu]]

Attachment: Media control + audio focus (draft)

Here is my current draft, in case that someone has an interest about it.
Will add some tests and split patches later.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug is a enhancement, but please change it back in case of error.

Comment 3

[Alastor Wu [:alwu]]

It's actually a defect, because we have this feature long time ago but it has been broken since we switch to e10s.

Comment 4

[Alastor Wu [:alwu]]

Attachment: Bug 1565689 - part1 : implement MediaController and MediaControlService.

In order to have a centralized audio control in the parent process, we create two new classes here.

• MediaController
  MediaController is a class used to control certain amount of media in the content process. Every controller corresponds to a browsing context.
  For example, TabMediaController would correspond to the top level browsing context, which mean it can control all media in the specific tab.

• MediaControlService
  As there might be multiple tabs playing audio, so there would be multiple controllers. MediaControlService is a place to manage all of them, you can access specific controller through MediaControlService by providing controller ID.
  Everytime a controller becomes active, which means there is a media starts in corresponding browsing context, then controller would be added into the list of the MediaControlService. And it would be removed from the list when the media in corresponding browsering context stopped.

Comment 5

[Alastor Wu [:alwu]]

Attachment: Bug 1565689 - part2 : implement AudioFocusManager.

In order to support audio competing among different tabs, we implement a new class AudioFocusManager.

AudioFocusManager is used to assign the audio focus to different requester and decide which requester can own audio focus when audio competing happens.
When the audio competing happens, the last request would be a winner who can still own the audio focus, and all the other requesters would lose the audio focus.
Now MediaController is the onlt requester, it would request the audio focus when it becomes audible and revoke the audio focus when the controller is no longer active.

Comment 6

[Alastor Wu [:alwu]]

Attachment: Bug 1565689 - part3 : use static pref to control audio competing.

We don't want to enable audio competing by default, so hide this feature behind a static pref.

Comment 7

[Alastor Wu [:alwu]]

Attachment: Bug 1565689 - part4 : notify controller about media active state and audible state changed from content processes.

We implement some helpful functions in MediaControlUtils which can be used to notify controller when media starts/stops playing or become audible/inaudible.

For now, we can temporarily notify these changes in AudioChannelService where we have already known when media has these kinds of status changing.

Comment 8

[Alastor Wu [:alwu]]

Attachment: Bug 1565689 - part5 : control media from chrome process.

This patch implements how to use MediaController to control corresponding media in content processes.

Comment 9

[Alastor Wu [:alwu]]

Attachment: Bug 1565689 - part6 : add gtests.

Comment 10

[Alastor Wu [:alwu]]

Attachment: Bug 1565689 - part7 : add browser test.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0de7b1db3ef9
part1 : implement MediaController and MediaControlService. r=baku
https://hg.mozilla.org/integration/autoland/rev/ba665f96330d
part2 : implement AudioFocusManager. r=baku
https://hg.mozilla.org/integration/autoland/rev/177f2b26c892
part3 : use static pref to control audio competing. r=baku
https://hg.mozilla.org/integration/autoland/rev/faaa4e9ba8d3
part4 : notify controller about media active state and audible state changed from content processes. r=baku
https://hg.mozilla.org/integration/autoland/rev/4f508252b015
part5 : control media from chrome process. r=farre,baku
https://hg.mozilla.org/integration/autoland/rev/278340adf69c
part6 : add gtests. r=baku
https://hg.mozilla.org/integration/autoland/rev/36ea91bf28f7
part7 : add browser test. r=baku

Comment 12

[Bogdan Tara[:bogdan_tara | bogdant]]

Backed out 7 changesets (Bug 1565689) for browser chrome failures

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=linux%2Cx64%2Casan%2Cmochitests%2Ctest-linux64-asan%2Fopt-mochitest-browser-chrome-e10s-4%2Cm%28bc4%29&fromchange=7168c4343dfa94e897d110b25cd62e5b655568e5&tochange=82db0cb5779d84929e6ffed33c7b879cf04394ec

Backout link: https://hg.mozilla.org/integration/autoland/rev/82db0cb5779d84929e6ffed33c7b879cf04394ec

Failure log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=260017370&repo=autoland&lineNumber=3658

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=260024667&repo=autoland&lineNumber=10440

Comment 13

[Bogdan Tara[:bogdan_tara | bogdant]]

I think this was also introduced by these changes:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=260017559&repo=autoland&lineNumber=30849

Comment 14

[Alastor Wu [:alwu]]

Have updated the fix, waiting for the try result and will push them again if all results are good.

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1d27d083e974
part1 : implement MediaController and MediaControlService. r=baku
https://hg.mozilla.org/integration/autoland/rev/1ca26f550f4b
part2 : implement AudioFocusManager. r=baku
https://hg.mozilla.org/integration/autoland/rev/d3db3285e933
part3 : use static pref to control audio competing. r=baku
https://hg.mozilla.org/integration/autoland/rev/d9f7fcbef3ee
part4 : notify controller about media active state and audible state changed from content processes. r=baku
https://hg.mozilla.org/integration/autoland/rev/9014276e070a
part5 : control media from chrome process. r=farre,baku
https://hg.mozilla.org/integration/autoland/rev/49c41a9238f9
part6 : add gtests. r=baku
https://hg.mozilla.org/integration/autoland/rev/9ebf6f8a4fda
part7 : add browser test. r=baku

Comment 16

[Andreea Pavel [:apavel]]

Backed out for assertion failure at MediaControlUtils.h

Push that started the failures https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&revision=9ebf6f8a4fda6267d3f44fd3cd775474e1fb1002

Push with failures https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedJob=260250811&resultStatus=testfailed%2Cbusted%2Cexception&revision=562d312e70e2dfebbfb32fc6194bb24f51d321d6

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=260250811&repo=autoland&lineNumber=23184

Backout: https://hg.mozilla.org/integration/autoland/rev/e0029fe751e0fea1ee69549a9414d26dac1548f4

Comment 17

[Alastor Wu [:alwu]]

From the push result [1], although there are some orange failures, but all of them seem not related with my change, and I didn't see any assertion or suspious failure this time.

[1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=41d891a3d473e706243a9d3131a7ea70e79f620c&selectedJob=260625544

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f6383bdf9dfd
part1 : implement MediaController and MediaControlService. r=baku
https://hg.mozilla.org/integration/autoland/rev/c1a2ecc4098e
part2 : implement AudioFocusManager. r=baku
https://hg.mozilla.org/integration/autoland/rev/2642920cf299
part3 : use static pref to control audio competing. r=baku
https://hg.mozilla.org/integration/autoland/rev/5790ef015c5b
part4 : notify controller about media active state and audible state changed from content processes. r=baku
https://hg.mozilla.org/integration/autoland/rev/d138720d49a9
part5 : control media from chrome process. r=farre,baku
https://hg.mozilla.org/integration/autoland/rev/bd22cb43dc59
part6 : add gtests. r=baku
https://hg.mozilla.org/integration/autoland/rev/8016ae836490
part7 : add browser test. r=baku

Comment 19

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/f6383bdf9dfd
https://hg.mozilla.org/mozilla-central/rev/c1a2ecc4098e
https://hg.mozilla.org/mozilla-central/rev/2642920cf299
https://hg.mozilla.org/mozilla-central/rev/5790ef015c5b
https://hg.mozilla.org/mozilla-central/rev/d138720d49a9
https://hg.mozilla.org/mozilla-central/rev/bd22cb43dc59
https://hg.mozilla.org/mozilla-central/rev/8016ae836490