1489278 - Show doorhanger when web audio is trying to autoplay

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, enhancement, P3)

Description

[Alastor Wu [:alwu]]

We should show the doorhanger for web audio as well when the web audio tries to autoplay, and then we would allow it to autoplay if getting user's approval.

Comment 1

[Alastor Wu [:alwu]]

The timing of showing doorhanger for media element is when audible autoplay is trying to start playing. However, it seems to me that the timing of showing doorhanger doesn't fit in well in the design of web audio, because web audio doesn't have the entry point for "starting playback".

Although AudioContext has resume()/suspend() functions which are not used to indicate the playback start of end, more like to indicate that the web audio engine has started.

The resume() will change AudioContext into `running` state which means Audio Context now starts processing input, but it doesn't mean that we have fed AudioContext any audible input yet. 

---

So, the question is "When should we show the doorhanger for web audio?"

1. When AudioContext is created

From the purpose of doorhanger, it violates the timing of showing the doorhanger. Creating audio context doesn't mean that the web audio is trying to play audible sound. 

From user pespective, it would cause lots unnecessary doorhanger, it would make user feel confused. 


2. Just before the destination node get the audible input

It's a good timing to show doorhanger, but we don't want user to hear any sound before user approve the doorhanger. Therefore, we must suspend AudioContext and resume AudioContext after get the user's approval

That causes two issues,

(1) AudioContext was changed from "running" -> "suspended" -> "running"
We have two extra state transitions here, and it apparently is not a spec-defined behavior. Not sure whether it would break any sites. I also don't like this redundant transition.

(2) Audio leaking 
I haven't tested that, but I guess the audio might leak in this situation. We handle input buffer in MSG thread, but if we want to suspend AudioContext, we need to go back to mainthread to call AudioContext.suspend() and wait for the operation sending to MSG thread. Within this period, the audio input might has been sent to audio device.

---

If possible, I would prefer to keep AudioContext in `suspened` state in the beginning, and then show doorhanger when AudioContext's destination node has connected with some input nodes which have called start or is sending audilbe input to the destionation node.

Paul, could you give me any suggestion here?

Thank you.

Comment 2

[Alastor Wu [:alwu]]

In addition, here is a second questions, 
"should we automatically resume AudioContext which was suspended by blocking-autoplay when user activate the page?"

Comment 3

[Paul Adenot (:padenot)]

The spec is pretty clear. We need to not do the initial "suspended" -> "running" transition, and show the door-hanger instead.

Then, when a uses accepts the door-hanger, this transition happens.

Alternatively, the door-hanger hides if `resume()` is called from a touch/click handler, and the AudioContext switches to "running".

We can then implement other behaviour if we decide this is not sufficient.

Comment 4

[Alastor Wu [:alwu]]

Attachment: Bug 1489278 - part1 : show doorhanger when create AudioContext

In the ideal situation, sites should create AudioContext only when sites are going to produce
sound, so we would show doorhanger to ask users whether they want to allow autoplay.

We delay the AudioContext's state transition from `suspended` to `running` until
(1) user click 'Allow' button in doorhanger
(2) user interact with sites, and then AudioContext calls resume() again

Comment 5

[Alastor Wu [:alwu]]

Attachment: Bug 1489278 - part2 : add test.

Comment 6

[Alastor Wu [:alwu]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=82bc3ad65fcdb84e9ee605f7e40812dda71a8c82

Comment 7

[Paul Adenot (:padenot)]

Comment on attachment 9008240 [details]
Bug 1489278 - part1 : show doorhanger when create AudioContext

Paul Adenot (:padenot) has approved the revision.

Comment 8

[Paul Adenot (:padenot)]

Comment on attachment 9008241 [details]
Bug 1489278 - part2 : add test.

Paul Adenot (:padenot) has approved the revision.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4cf5ee8d6112
part1 : show doorhanger when create AudioContext r=padenot

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2477110ccfba
part2 : add test. r=padenot

Comment 11

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/4cf5ee8d6112
https://hg.mozilla.org/mozilla-central/rev/2477110ccfba