1405827 - Add an API to allow fine-grained control over media autoplay

Status: NEW

(WebExtensions :: General, enhancement, P5)

Description

[Stephan Sokolow]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36
Build ID: 20170802111520

Steps to reproduce:

Currently, toggling media.autoplay.enabled programmatically, as described in bug 1391501, is only useful as a user-triggered operation, because it's effects are unpredictable when multiple tabs are loading in parallel.

It also has the downside of potentially breaking sites which are designed on the assumption that autoplay or play() cannot be vetoed.

A better solution for general use would be to grant extensions the ability to set a media autoplay policy in a more fine-grained scope (such as per-tab, per-document, or per-video-element) by registering a handler for an event which is guaranteed to fire before autoplay takes effect and JavaScript execution begins.

Ideally, this would happen as late as possible in the process (eg. DOM completion or perhaps even after enough of the video has been loaded for basic parameters to be known) but it could still be very useful with as little as just the URL of the top-level window.

In that case, I envision it being useful for implementing something akin to NoScript or uMatrix, but for controlling autoplay rather than resource loading/execution.

The specifics I propose would be:

1. Always start a document out with the preference-defined behaviour.

2. By registering a handler for the correct event, the extension is guaranteed the opportunity to customize the value before it takes effect. (In either direction, so there's no reason for an extension to ask the user to change the preference.)

3. With tab-scoped behaviour that only has access to the top-level URL, the following behaviour would become possible:

   a. WebRTC sites like appear.in get automatically allowed, so I don't have to resort to using Chromium for them as I do now.

   b. YouTube gets autoplay disabled, so it continues to require me to click Pause and then Play to start playback. (Deferring autoplay until the tab is first focused is insufficient because I often middle-click a bunch of entries in the recommended sidebar and come back to them, intending to refresh my memory of the channel name... only to startle and then curse when they start making noise... possibly after I've already switched away if things are slow that day.)

   c. Video advertising within pages is blocked by uMatrix, regardless of whether I've allowed autoplay.

4. The setting would be a yes/no/inherit value with child elements (eg. iframes) starting out set to "inherit" so extensions need only set the policy at the coarsest level of granularity necessary.

Comment 1

[Caitlin Neiman [:caitmuenster]]

Hi Stephen, this has been added to the agenda for the October 17 WebExtensions APIs triage meeting. Would you be able to join us? 

Here's a few things about the triage to keep in mind: 

* We normally spend 5 minutes per bug
* The more information in the bug, the better
* The goal of the triage is to give a general thumbs up or thumbs down on a proposal; we won't be going deep into implementation details

Relevant Links: 

* Wiki for the meeting: https://wiki.mozilla.org/Add-ons/Contribute/Triage
* Meeting agenda: https://docs.google.com/document/d/1oUFGD57_NGbtV5y8k_yIS3GN8pFO3M_K1SWQhzlq6Ho/edit#heading=h.hhpni8ijl0wx
* Vision doc for WebExtensions: https://wiki.mozilla.org/WebExtensions/Vision

Comment 2

[Stephan Sokolow]

I can try, but I can't guarantee anything today as, over the last few days, I've been waking at or after 18:00 UTC.

(I'll need a few days to see if I can push that back enough to be coherent at the requisite time.)

Also, just to avoid the mistake spreading further, my name is the "ph" spelling of "Stefan", not "Steven". (You're not the first to make that mistake and you certainly won't be the last.)

Comment 3

[Caitlin Neiman [:caitmuenster]]

Thanks, Stephan -- I'll keep that in mind. :) No worries if you're not able to attend; it looks like you have provided a lot of information in the bug and we can needinfo you if any questions come up.

Comment 4

[Stephan Sokolow]

That might be better. I always worry about not having time to properly consider my responses in a real-time context.

Comment 5

[Caitlin Neiman [:caitmuenster]]

The WebExtensions group approved this design, but most of the implementation work will need to be done by the media team. Bob will reach out to them to get their feedback.

Comment 6

[Bob Silverberg [:bsilverberg]]

Alastor, would you mind commenting on this bug, and whether it's something that you could see the media team supporting? We feel that it would be an acceptable addition to the WebExtensions API, but imagine that most of the work would have to be done by the media team to provide an event for which WebExtensions APIs could listen.

Comment 7

[Alastor Wu [:alwu]]

I must to say sorry I'm not familiar how the Web Extension works, so not sure whether my understanding is correct.

First, shortly to describe when to check the pref and stop autoplay.

1. If video is added "autoplay" keyword, we check the pref when video is getting enough data
2. If script calls video.play(), we check the pref when the play() is invoking

And then stop to play the video if pref is off (media.autoplay.enabled=false).

(In reply to Stephan Sokolow from comment #0)
> A better solution for general use would be to grant extensions the ability
> to set a media autoplay policy in a more fine-grained scope (such as
> per-tab, per-document, or per-video-element) by registering a handler for an
> event which is guaranteed to fire before autoplay takes effect and
> JavaScript execution begins.

"autoplay takes effect" means you want a new event prior we do those checkings? Or just want a event handler to watch the the value change of pref?

And how to change the pref scope to make it per-tab or per-element? I thought its effect is global.

Comment 8

[Stephan Sokolow]

What I propose is to store a per-scope boolean or tri-state value at whatever level of granularity this is implemented for and have that sit in between the pref and "autoplay" or "video.play()".

Suppose we call it scope.autoplay_allowed, it would work like this:

1. Whenever a new scope is created (eg. navigating a top-level tab, DOMComplete, or what have you), scope.autoplay_allowed is set to the value of the pref.
2. If a video is added with the "autoplay" keyword, check scope.autoplay_allowed when enough video data has arrived.
3. If a script calls video.play(), check scope.autoplay_allowed.

As for whether a new event is needed, that would depend on the scope it's implemented for.

If there's one boolean per top-level document, then something existing like DOMComplete can be sufficient as long as it's an event that's guaranteed to be emitted after #1 but before #2 or #3. (Use case: Set scope.autoplay_allowed based on the value of window.location or the result using document.querySelector to inspect the page's contents.)

If there's one tri-state per document at any level (eg. each IFRAME has its own scope.autoplay_allowed with the third value meaning "inherit parent's value"), then the only significant difference from the "one per top-level document" case is providing a documented, reliable way to catch the insertion of new iframes and bind to their handlers before the event is emitted. (Use case: Allowing things like WebRTC widgets iframes and video player iframes in the same page with different policies.)

If there's one tri-state per audio/video element, then the design can only come into its own if there's a reliable way to trigger an event handler based on what the audio/video element is trying to play. (eg. some kind of DOMReady equivalent which fires on a per-element basis just before #2 and #3 are evaluated.)

Any one of those would be enough for my current use cases, but it'd be nice to have the "one tri-state per document at any level" option so I don't have to trust that player widgets will disable autoplay when embedded in other sites where I may want to allow the local use of media.

Comment 9

[Mike Conca [:mconca]]

Comment 5 notes that this feature request was approved.  Updating the whiteboard tag.