Closed Bug 659285 Opened 12 years ago Closed 8 years ago

Extend media.autoplay.enabled to provide a way to disable untrusted play() invocations

Categories

(Core :: Audio/Video, enhancement)

Product:
Core
Component:
Audio/Video
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla41
Tracking Flags:
Tracking Status
firefox41 --- fixed
fennec 40+ ---

People

(Reporter: a.nielsen, Assigned: rbarker)

References

(
URL
)

Details

(Keywords: dev-doc-needed, user-doc-needed)

Attachments

(1 file, 9 obsolete files)

0001-Bug-659285-Extend-media.autoplay.enabled-to-provide-a-way-to-disable-untrusted-play-invocations-15060509-ecec545.patch
1.67 KB, patch
Details | Diff | Splinter Review 
User-Agent:       Mozilla/5.0 (X11; Linux x86_64; rv:6.0a1) Gecko/20110523 Firefox/6.0a1
Build Identifier: Mozilla/5.0 (X11; Linux x86_64; rv:6.0a1) Gecko/20110523 Firefox/6.0a1

HTML5 videos crash Firefox on my PC 100% of the time, so obviously I would like to prevent them from playing.  I found the media.autoplay.enabled option in about:config, set it to false, yet videos (e.g. those from YouTube) will play automatically and thus crash Firefox shortly after the tab is loaded.

Lacking any other way of disabling HTML5 videos, I think this option should be extended to work with any audio/video option.

Reproducible: Always

Steps to Reproduce:
1. Set media.autoplay.enabled to false in about:config
2. Visit a web site with an autoplay video (e.g. https://www.getup.org.au/campaigns/climate-action-now/video/fund-solutions-not-pollution)
3. Observe video starts without warning and there's no way of preventing that

Actual Results:  
Video started playing, followed by browser crash as per bug 650601.

Expected Results:  
Video would not play until I clicked play, and thus I could avoid a crash

No Flash plugin installed
Do you have links to crash reports for the crashes? It'd be great to find and fix those. They should show up if you enter about:crashes in the address bar.
Chris, the info about the crashes is in bug 650601.

I think this bug is about the autoplay pref not affecting scripted play() calls on pageload...
Ah, a scripted play() could explain it.  Well thanks to some help the crash bug is now fixed, but it would still be nice to have videos respect my preference for not starting playback automatically.

I guess then any play() calls should do nothing (if autoplay is false), unless they were triggered in response to a mouse click/key press?
The YouTube HTML5 Experiment plays videos by calling play() directly, they don't rely on autoplay, so that's why setting media.autoplay.enabled to false isn't stopping the video playing on YouTube.

Disabling play() from script when media.autoplay.enabled=false would break custom controls, so I don't think we want to do that, and it doesn't sound like it's worth the effort determining whether a play() call is running in a keypress or mouse event handler.
Status: UNCONFIRMED → RESOLVED
Closed: 12 years ago
Resolution: --- → WONTFIX
So exactly what in hell is the poor user supposed to do if the site he visits decides to play media with a scripted call, right over the media that I WANT to play on MY computer right in the middle of a party or other event? It has now become completely impossible to look up anything on the Internet while I am using the computer to play music. Are we returning to the days when a separate computer will be required for every different task I want to perform? Might it be possible to add another option that allow us to prevent playback even by a scripted call? I don't mind changing the option depending on whether or not I'm playing music on the computer. Thanks.
(In reply to freonpsandoz from comment #6)

NoScript can be configured to act as a secure FlashBlock-alike for various kinds of embedded media, HTML5 video/audio tags included, without needing to operate in javascript-whitelisting mode.
I could go on a rant on why your policy on this is stupid... so I will.

Not having the option to block 'play()' from being programmatically invoked means that $marketing_droid is going to decide that their $amazing_most_awesomest_video_ever is going to use it. The end result obviously being that setting media.autoplay.enabled=false is _never_ going to do anything. 

I know it would be more work to fix the video player controls so this preference actually does what you'd expect, but in its current form, the preference is worse than useless. You _will_ see more people bi**h and moan about this in the future as HTML5 video gets more widely adopted and more and more people try this option and realise that it does absolutely f**k all, I _guarantee_ it.

For now, the only reasonable workaround is to disable the media.* options altogether if you want a browsing experience where you are in control over whether sound blares from your speakers at random. 

Marking this as WONTFIX is an insult to your users and essentially declaring defeat in the arms race against idiots who think annoying users with sounds and moving pictures is a good idea.
I think this would be a useful enhancement, so reopening with a more accurate summary.  Unfortunately I won't have time to personally work on this for the foreseeable future, but I'd be happy to review and shepherd patches.
Severity: normal → enhancement
Status: RESOLVED → REOPENED
Ever confirmed: true
Resolution: WONTFIX → ---
Summary: media.autoplay.enabled=false doesn't prevent autoplay → Extend media.autoplay.enabled to provide a way to disable untrusted play() invocations
> Extend media.autoplay.enabled to provide a way to disable untrusted play() invocations

I barely have time to touch codebases I'm intimately familiar with these days, so I can't provide a patch, but that summary makes me think that the best solution is probably to mimic the ad-blocker's solution for popups.

In other words, limiting play() to some variation on "only as a result of direct user action".
Consider also the effect on bandwidth cap and overage charges when some random <video> starts filling its play buffer before you even know its there. 

A few extensions are trying to address the issue with hit-or-miss results -- they only work on youtube, and then only until youtube updates their scripts. 

Flash manipulation extensions have been much more effective.  The way <video> and play() work seems to be harder for extension authors to manipulate gracefully than <object> and <embed>.  Substituting the tag in DOM causes breakage in page JS for instance. ... I may try my hand at an extension...
Is there any reason why the code for blocking media.play() wouldn't look the same as the code for blocking window.open()?
I have to agree with comment 12. To me, code for switching on whether an event was user-triggered seems like code which would ideally be a generalized, reusable component.

(And if that were solved by some kind of system of annotating events, it might also provide a clue for how to implement "anything is middle-clickable, nothing automatically opens new tabs" where Javascript URLs can reliably be forced to treat navigation by window.location and window.open as equivalent and controlled by whether the user middle-clicked.)
How about a new 'super-title bar':  YouTube would like to play a video. Allow Y/N? [x] remember this
Also media.autoplay.enabled should be OFF by default!  I know you guys are proud of the HTML5 video support.  Well done.  Now turn off autoplay.
(In reply to sam from comment #15)
> Also media.autoplay.enabled should be OFF by default!  I know you guys are
> proud of the HTML5 video support.  Well done.  Now turn off autoplay.

It's not that simple. Just simply turning it off by default risks breaking HTML5 video's usability for building multimedia applications like games and driving developers back to Flash or users to other browsers.
For flash video and adobe pdf plugin I have had great success by setting plugins.click_to_play to true in about:config ( see https://blog.mozilla.org/addons/2012/10/11/click-to-play-coming-firefox-17/ ).  All the stupid auto-play video and PDF is dead until I click on the screen, then it loads it.  Extremely pleasing, try it out.

Now we need the same thing for HTML5 "big media" like video and audio that no longer uses plugins.  The concept of a "click to play" is very satisfying for big media like huge videos and annoying audio.

A browser is not a tool for web site owners.  It is a tool for users.  It's my agent to view the web, not someone else's advertising tool to stream junk.

To yield 100% control to web site owners to peddle junk, consume your bandwidth, blare noise, is not something we want for Firefox.  To prove this point, just check out how many add-ons are created to fix the browser's lack of support for what users want (ghostery fixes spying, adblock plus fixes noise and junk, noscript fixes remote control of your browser until you say so, etc).

Users really want control, and since plugin control is now ebbing, it is really important to replace the loss of control.
This is a major deal for me. It's enough that I'm wary of using HTML5 video
at all. If it comes down to using a sane, open standard with an open codec,
or having to use Flash, I'll stick with Flash if that's the only way I can
disable autoplay.
I note that this is an issue which I sense began many versions ago (I am on Ver 26.0, now), particularly with media company websites such as Yahoo or ABC.  I don't have an autoplay problem with YouTube.

This makes me wonder if the effect is due to HTML5 issues or something else -- though I don't know which version brought on HTML5 support.

I have had Adobe Flash installed, which makes me wonder if the video is driven by HTML5 controls, or relies on old flash driven issues.   The video clearly appears in a Flash box, and is tabbed by Adblock.  

Neither Safemode enabled, nor Addons disabled (removing Adblock, Ghostery and HTTPSeverwhere), remove the problem.   Settings for media.autoplay.enabled and plugins.click_to_play do not affect the problem.

Is my experience with this problem landed within this bug, or within a new bug ?
An easy way to achieve this is to make HTMLMediaElement.play() only work in a user generated event handler when the pref is turned on. This is what we do for fullscreen.

This means script can only successfully play a media if the script is running in an event handler like a mouse click or key press handler, i.e. in response to a user action. This can be achieved like we do for fullscreen, just by only honouring the script call if nsEventStateManager::IsHandlingUserInput() returns true.

We should probably also fire events as if we successfully started playing, and then were immediately paused by the user, so that third party controls don't get their state mixed up.
Chris's comment 20 from January, 2014 sounds like a great idea.  Sadly, I can't program this sort of change; is there anyone who can pick up this bug and crack it?
I managed to implement what Chris described (comment 20). Firing the play, playing, and pause events doesn't appear to be enough to "fake" a successful call to play(), though. The Vimeo player will call play() in a loop with my modifications, for example. I assume I need to set the media state to paused, but it looks like calling HTMLMediaElement::Pause() will force the media to load (which I'd like to avoid). Any pointers? This is my first attempt at modifying the Firefox code base, so I'd appreciate any help.
Further testing reveals that YouTube's HTML5 player doesn't call play() from a trusted event handler, even when clicking the play button. So, I think since the spec doesn't say anything about this behavior (that I can see), implementing it in a way that doesn't completely break existing, widely-used media players is just not possible. :(
What about a tri-state setting where the compromise mode blocks play() until the page has received at least one user-triggered click event, then allows it from any source?

That wouldn't stop things like video ads which keep trying to play, but it would at least help common benign stuff like middle-clicking a bunch of YouTube links and expecting them to not auto-play.
That sounds workable. I assume I should set that flag in nsPresShell::HandleEventInternal, but where is the best place to store it so it can be retrieved from HTMLMediaElement::Play?
What would be the three different states' definitions that this variable holds?

Would two separate variables work better?  I'm a strong believer in normalization, as it would simplify the coding later, if different parts of the code set and clear, and or trigger decision paths -- would save having to determine the state of the variables other [hidden normalized] meaning all over the place.
0: No restrictions
1: Compromise restriction
2: Hard restriction

That allows an "if truthy value" check for quick bypassing of the code when off, an intuitive ordering, and the least trouble for someone who mistakes it for an integer-form boolean and sets it to 1.
As additional example you can see this link:

https://www.paypal.com/it/webapps/mpp/home

In the main page there is a video that autostarts!
Another example here: http://www.avclub.com/tvclub/elementary-grand-experiment-204741

Advertisement on the right-hand side is an auto-playing html5 video, auto-plays even with this preference set to 'false'.

Firefox users far more power than safari 7 on my mac because of this, open up any page like the above example and power usage skyrockets because of auto-playing video.
Bug reproducible on FF 33 as well.
I'd rather disable html5 entirely than have autoplay possible, so that's what I've done. This is not an enhancement request, this is a broken feature.
Disabling autoplay should work like plugins.click_to_play and prevent the whole element from loading without user interaction. NoScript does this perfectly fine and for now everyone who want's to disable html5 video autoplay is is forced to use it or a similar plugin.
While waiting for this to arrive, the 'Flash Control' extension is a workable option for this (works for flash & html5 audio/video).
Blocks: 1130637
Alice why was bug 1129562 duped to this?
(In reply to Aaron Train [:aaronmt] from comment #36)
> Alice why was bug 1129562 duped to this?

The websites call .play() themselves.
Assignee: nobody → snorp
tracking-fennec: --- → 38+
FF36 (beta) on android also affected.

I just cannot believe this bug is almost 4 years old bug...
Flags: needinfo?(snorp)
See Also: → 980939
One relatively simple and workable option would be to treat it like popups, with per-site permissions. If the user has disabled autoplay (which should have a UI control in Options) then ignore the autoplay attribute and if the page calls .play() put up an infobar or doorhanger telling the user that the page would like to play media, with allow-once and allow-always options.

I assume we would not make this the default option so I'd prefer an easily ignored infobar to an attention-grabbing doorhanger (again, like popups). Of course we'd also have to add this to the about:permissions controls, and I guess an "Exceptions" button next to the checkbox (like popups) would fit what we currently use since about:permissions is still in limbo in terms of discoverability.
With media.autoplay.enabled = false, both videos and ads still play automatically.

Also, user comments on liveleak.com videos contain user avatars, in which some play sound as you hover over them.  I am not sure if that should be considered automatic.
This is now impacting a huge number of people, as Facebook is now enabling (multiple) preloading, autoplaying HD <video> clips in the newsfeed.  Pausing a clip doesn't pause the preload and, strangely, even switching to a different page seems not to.  (This may be a bug, as the preload then goes on and on.)

My daily download usage has spiked dramatically, and I've had a lot of trouble recently with Firefox becoming unresponsive - have yet to establish if the two are connected.

The obvious solution would be to treat <video> like Flash: have a video icon top-left of the infobar when the page contains a valid <video> tag.  Content would be enabled by default, but clicking on this would present the user with per-site blocking options.  (There would then be a dark gray video placeholder with an 'enable' option.)  The advantages with this are (a) it would always work, no matter what future call methods were employed, (b) it would prevent all <video> preloads for the page, including those added by updating feeds, and (c) it would provide per-site control with a simple and familiar 'right there' GUI control.

This needs to be a priority - it can't wait another 4 years(!), or even another 4 months, really.  For more reasons this needs to be a priority see my firefox-dev comment here: https://mail.mozilla.org/pipermail/firefox-dev/2015-February/002721.html

Is there any native method of disabling <video>?  If not, that is an extraordinary omission for this 'encoder experiment'!
tracking-fennec: 38+ → 40+
Assignee: snorp → rbarker
As several comments have mentioned, I think we plan to handle this similar to popups by requiring user action for .play().
Flags: needinfo?(snorp)
I just want a click-to-play option for all Audio/Video. I don't want any Audio or Video to play unless I click a Play button or something.

I don't have the problem of html5 crashing my linux box though.
Comment on attachment 8592960 [details] [diff] [review]
0001-Bug-659285-Extend-media.autoplay.enabled-to-provide-a-way-to-disable-untrusted-play-invocations-15041512-b7032d2.patch

Who should I ask to review this. I'm sure this isn't correct, but it seems to work for me on both Fennec and Desktop so it is at least a starting point.
Attachment #8592960 - Flags: feedback?(snorp)
Comment on attachment 8592960 [details] [diff] [review]
0001-Bug-659285-Extend-media.autoplay.enabled-to-provide-a-way-to-disable-untrusted-play-invocations-15041512-b7032d2.patch

Review of attachment 8592960 [details] [diff] [review]:
-----------------------------------------------------------------

Looks like a good start. There will be many edge cases, though, so we should try to figure those out. The imgur 'gifv' videos, for instance, have no player controls. We should detect if that is the case and play it on tap/click or something.
Attachment #8592960 - Flags: feedback?(snorp) → feedback+
I applied these 4 patches and rebuilt. And it crashes when opening a youtube video page. (e10s build on Linux)
Actually crash isn't cause by patch. Sorry for the noise.
New patch restricts functionality to Fennec.
Attachment #8605461 - Attachment is obsolete: true
Attachment #8605462 - Attachment is obsolete: true
Attachment #8605463 - Attachment is obsolete: true
Attachment #8605930 - Flags: review?(wjohnston)
(In reply to Randall Barker [:rbarker] from comment #55)

> New patch restricts functionality to Fennec.


Why only Fennec? Desktop users are annoyed by it too.
Comment on attachment 8605931 [details] [diff] [review]
0002-Bug-659285-Part-2-Extend-media.autoplay.enabled-to-provide-a-way-to-disable-untrusted-play-invocations-15051411-1e28227.patch

I made the changes to the web-platform test in order to get them to pass. I believe they started to fail because the current tests appear to be racey in nature. It seems that the changes I've made to the video element when no controls are specified increase initialization time so that the onload will fire before the video element has had time to emit an error event. I've tried to modify the test so that the events have time to propagate before the test is ended. I modified one of the tests because a blank white div is no longer equivalent to what a video element shows when there is an error. Here is a try run with these changes applied: https://treeherder.mozilla.org/#/jobs?repo=try&revision=4b7fe37e47f3
Attachment #8605931 - Flags: review?(james)
(In reply to Hubert Figuiere [:hub] from comment #57)
> (In reply to Randall Barker [:rbarker] from comment #55)
> 
> > New patch restricts functionality to Fennec.
> 
> 
> Why only Fennec? Desktop users are annoyed by it too.

It doesn't work with e10s. Rather than block Fennec, I want to try and land this while I keep working on desktop support. Youtube is also broken on desktop with the old patch.
Comment on attachment 8605930 [details] [diff] [review]
0001-Bug-659285-Part-1-Extend-media.autoplay.enabled-to-provide-a-way-to-disable-untrusted-play-invocations-15051411-906d0c3.patch

Review of attachment 8605930 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/html/HTMLMediaElement.cpp
@@ +2209,5 @@
>  void
>  HTMLMediaElement::Play(ErrorResult& aRv)
>  {
> +#if defined(MOZ_WIDGET_ANDROID)
> +  if (!IsAutoplayEnabled() && !EventStateManager::IsHandlingUserInput() && !nsContentUtils::ThreadsafeIsCallerChrome()) {

Why does this not work in e10s Desktop Firefox? The fullscreen code uses both EventStateManager::IsHandlingUserInput() and nsContentUtils::IsCallerChrome() and it works in e10s Desktop Firefox.

HTMLMediaElement::Play() should only be called on the main thread, so you should not need to call the threadsafe variant if IsCallerChrome.
(In reply to Chris Pearce (:cpearce) from comment #60)
> Comment on attachment 8605930 [details] [diff] [review]
> 0001-Bug-659285-Part-1-Extend-media.autoplay.enabled-to-provide-a-way-to-
> disable-untrusted-play-invocations-15051411-906d0c3.patch
> 
> Review of attachment 8605930 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: dom/html/HTMLMediaElement.cpp
> @@ +2209,5 @@
> >  void
> >  HTMLMediaElement::Play(ErrorResult& aRv)
> >  {
> > +#if defined(MOZ_WIDGET_ANDROID)
> > +  if (!IsAutoplayEnabled() && !EventStateManager::IsHandlingUserInput() && !nsContentUtils::ThreadsafeIsCallerChrome()) {
> 
> Why does this not work in e10s Desktop Firefox? The fullscreen code uses
> both EventStateManager::IsHandlingUserInput() and
> nsContentUtils::IsCallerChrome() and it works in e10s Desktop Firefox.
> 
> HTMLMediaElement::Play() should only be called on the main thread, so you
> should not need to call the threadsafe variant if IsCallerChrome.

This part works fine in e10s. The issue is the place where I was listening for the MozAutoplayMediaBlocked no longer receives the event. I assume it is because the listener is in a different process in e10s? The old patch also broke youtube on desktop when autoplay was disabled which was the bigger issue. I'm still looking into the desktop issues but don't want to block Fennec while I figure out how to get it working on desktop. I also wanted the code reviewed as I was working on unfamiliar parts of gecko and didn't want to get too far down the wrong path.
For sending messages in e10s, you need to handle the message in browser/base/content/tab-content.js and send it across to the chrome process using messageManager, and then you need to listen on that message in the browser chrome code. For an example, see the "MozExitedDomFullscreen" message:

http://mxr.mozilla.org/mozilla-central/search?string=MozExitedDomFullscreen
Comment on attachment 8605931 [details] [diff] [review]
0002-Bug-659285-Part-2-Extend-media.autoplay.enabled-to-provide-a-way-to-disable-untrusted-play-invocations-15051411-1e28227.patch

Review of attachment 8605931 [details] [diff] [review]:
-----------------------------------------------------------------

You should get a media peer to review changes to media tests. :karlt would be good in this case, as he's got experience with the web platform tests.

::: testing/web-platform/tests/html/semantics/embedded-content/media-elements/loading-the-media-resource/resource-selection-invoke-set-src.html
@@ -8,5 @@
>  async_test(function(t) {
>    var v = document.querySelector('video');
>    v.onloadstart = t.step_func(function() { t.done(); });
>    v.setAttribute('src', '');
> -  window.onload = t.step_func(function() { assert_unreached(); });

Changing this to a setTimeout seems like you're working around a bug.

Changes to the web platform tests automatically get pushed upstream so that other browser vendors get them too, so we need to ensure we're making responsible changes to these test otherwise that privilege may be revoked. So please don't add code to work around a Firefox bug here, we need to fix the underlying bug.

And why is this even necessary? media.autoplay.enabled should not be disabled by default in the test harness, so how can this code be hit?
Attachment #8605931 - Flags: review?(james) → review?(karlt)
(In reply to Chris Pearce (:cpearce) from comment #63)
> Comment on attachment 8605931 [details] [diff] [review]
> 0002-Bug-659285-Part-2-Extend-media.autoplay.enabled-to-provide-a-way-to-
> disable-untrusted-play-invocations-15051411-1e28227.patch
> 
> Review of attachment 8605931 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> You should get a media peer to review changes to media tests. :karlt would
> be good in this case, as he's got experience with the web platform tests.
> 
> :::
> testing/web-platform/tests/html/semantics/embedded-content/media-elements/
> loading-the-media-resource/resource-selection-invoke-set-src.html
> @@ -8,5 @@
> >  async_test(function(t) {
> >    var v = document.querySelector('video');
> >    v.onloadstart = t.step_func(function() { t.done(); });
> >    v.setAttribute('src', '');
> > -  window.onload = t.step_func(function() { assert_unreached(); });
> 
> Changing this to a setTimeout seems like you're working around a bug.
> 
> Changes to the web platform tests automatically get pushed upstream so that
> other browser vendors get them too, so we need to ensure we're making
> responsible changes to these test otherwise that privilege may be revoked.
> So please don't add code to work around a Firefox bug here, we need to fix
> the underlying bug.
> 
> And why is this even necessary? media.autoplay.enabled should not be
> disabled by default in the test harness, so how can this code be hit?

The issue is that the error messages for video elements were turned off when the video element does not have controls. This was done to speed up FxOS (see Bug 821695). As part of the work to enforce the autoplay preference when scripts invoke play, we wanted to have a click to play button and to display the standard error messages that are displayed when the video has controls. So the error screen will now display even when autoplay is not disabled. Creating the elements to show error messages seem to have changed the timing of the tests so that before where they would mostly pass, they now mostly fail.
Comment on attachment 8605931 [details] [diff] [review]
0002-Bug-659285-Part-2-Extend-media.autoplay.enabled-to-provide-a-way-to-disable-untrusted-play-invocations-15051411-1e28227.patch

>Bug 659285 - Part 2, Extend media.autoplay.enabled to provide a way to disable untrusted play() invocations
>
>Update tests to pass when error message is displayed for video with no controls.

Given this patch is updating web platform tests, the second line is a good
commit message.

However, the comment on the first line has little to do with the change, so
please remove the first comment and replace with the second.

> <title>Reference for WebVTT rendering, when media cannot be played (404 error), subtitles are not displayed</title>
> <style>
> html { overflow:hidden }
> body { margin:0 }
>-.video {
>-    display: inline-block;
>-    width: 1280px;
>-    height: 720px;
>-    position: relative
>+::cue {
>+    font-family: Ahem, sans-serif;
>+    color: green
> }
> </style>
>-<div class="video"></div>
>+<video width="1280" height="720" autoplay>
>+    <source src="videonotfound.webm" type="video/webm">
>+    <source src="videonotfound.mp4" type="video/mp4">
>+</video>
>+</html>

Replacing the div with video to make the test makes sense in light of [1].
Is the cue pseudo-element css required, given there are no cues?

> <script src="/resources/testharnessreport.js"></script>
> <div id=log></div>
> <video></video>
> <script>
> async_test(function(t) {
>   var v = document.querySelector('video');
>   v.onloadstart = t.step_func(function() { t.done(); });
>   v.setAttribute('src', '');
>-  window.onload = t.step_func(function() { assert_unreached(); });
>+  setTimeout(t.step_func(function() { assert_unreached(); }), 100);
> });

Adding a timeout duration is pretty much never the right fix.
Here the onload I assume is testing that loadstart is received on v before
load on window.

It seems the test is assuming that t.done() will synchronously unload the page.

I don't know whether there is good reason why t.done() should do this
synchronously, but I doubt it.  James may know the answer to this.

Either the test should remove the assumption, or the harness should support it
(if it is a requirement), or the test should be disabled.

I haven't looked at all the other onload/timeout changes, but I suspect the
timeout is not the right solution.

[1] http://www.w3.org/TR/2015/WD-html51-20150417/semantics.html#the-video-element:the-video-element-20
"the user agent may provide messages to the user (such as "buffering", "no
video loaded", "error", or more detailed information) by overlaying text or
icons on the video or other areas of the element's playback area,"
Attachment #8605931 - Flags: review?(karlt) → review-
(In reply to Karl Tomlinson (ni?:karlt) from comment #65)
> It seems the test is assuming that t.done() will synchronously unload the
> page.
> 
> I don't know whether there is good reason why t.done() should do this
> synchronously, but I doubt it.

Actually, it is explicitly documented that the test will not finish until "The load event on the document has fired".

http://testthewebforward.org/docs/testharness-library.html#determining-when-all-tests-are-complete
Comment on attachment 8605930 [details] [diff] [review]
0001-Bug-659285-Part-1-Extend-media.autoplay.enabled-to-provide-a-way-to-disable-untrusted-play-invocations-15051411-906d0c3.patch

Review of attachment 8605930 [details] [diff] [review]:
-----------------------------------------------------------------

Seems fine to me. You'll need a peer for this file to review too though. I think dolske and jaws both are, so whichever gets to it first I guess. They may want this on Desktop as well (although I'd say we can do that in a follow up...)

::: toolkit/content/widgets/videocontrols.xml
@@ +1857,5 @@
> +
> +              for each(let element in this.controlListeners)
> +                element.item.removeEventListener(element.event, element.func, false);
> +
> +              delete this.controlListeners;

The devtools guys have told me that deleting is a bad thing before (i.e. it ruins Shapes in the JS engine which prevents it from optimizing property access). There aren't that many of these objects that it matters, but probably better to just set to null or [] if you're done with something.

@@ +1864,5 @@
> +            hasError : function () {
> +              return (this.video.error != null || this.video.networkState == this.video.NETWORK_NO_SOURCE);
> +            },
> +
> +            updateErrorText : function () {

We really should probably pull this method into a super class and let everyone inherit from it, but these controls are complex enough, I'm starting to think maybe this is better.
Attachment #8605930 - Flags: review?(jaws)
Attachment #8605930 - Flags: review?(dolske)
The media playback team have been discussing this issue, and we think a better approach would be to delay playback start until the tab has been made visible at least once.
(In reply to Chris Pearce (:cpearce) from comment #68)
> The media playback team have been discussing this issue, and we think a
> better approach would be to delay playback start until the tab has been made
> visible at least once.

Sounds like a good start for sure, but doesn't fully address the issue for many of us. Most html5 videos I encounter are either unexpected or undesirable - I load a page in the currently active tab by clicking on a link, and a video begins playing out of nowhere. This happens on many news sites (cnn, yahoo news, ...), for example.

Might I suggest your (cpearce) approach as the default, with an easily accessible 'never ever autoplay ever' setting for the rest of us? I won't sleep easy until that setting exists on desktop firefox somewhere.
Comment on attachment 8605930 [details] [diff] [review]
0001-Bug-659285-Part-1-Extend-media.autoplay.enabled-to-provide-a-way-to-disable-untrusted-play-invocations-15051411-906d0c3.patch

Review of attachment 8605930 [details] [diff] [review]:
-----------------------------------------------------------------

::: mobile/android/themes/core/content.css
@@ +304,5 @@
>    -moz-binding: url("chrome://global/content/bindings/videocontrols.xml#touchControls");
>  }
>  
> +/* display error and click to play when autoplay is blocked for videos */
> +video:not([controls]) > xul|videocontrols {

I expect this to cause issues with sites that have custom controls and a custom click-to-play overlay.
Attachment #8605930 - Flags: review?(jaws) → review-
I agree with Comment #69. I find autoplay very annoying even for the current tab: it slows down the machine, it makes noise (when audio output is on), it increases the network consumption very much (which matters when one is connected via the mobile network).
Doing both, behind separate prefs, is fine with me.
(In reply to Jared Wein [:jaws] (please needinfo? me) from comment #70)
> Comment on attachment 8605930 [details] [diff] [review]
> 0001-Bug-659285-Part-1-Extend-media.autoplay.enabled-to-provide-a-way-to-
> disable-untrusted-play-invocations-15051411-906d0c3.patch
> 
> Review of attachment 8605930 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: mobile/android/themes/core/content.css
> @@ +304,5 @@
> >    -moz-binding: url("chrome://global/content/bindings/videocontrols.xml#touchControls");
> >  }
> >  
> > +/* display error and click to play when autoplay is blocked for videos */
> > +video:not([controls]) > xul|videocontrols {
> 
> I expect this to cause issues with sites that have custom controls and a
> custom click-to-play overlay.

You will only see the click to play overlay if the video has no controls and it tries to autoplay so I'm not sure how it would have a custom click-to-play overlay.
+1 to having it disabled all the time. I don't want a video to start playing just because I view a tab. Not only bandwidth is an issue. I don't want to see html5 video advertisement either. And I don't want to be searching for the place some strange audio is coming from. Autoplay is always a very bad idea.
Agreed. I put up with a weird, irritating, and buggy NoScript experience in order to prevent autoplay in <video> elements.

Specifically...
* I have NoScript set to block "embeddings" even when JavaScript is allowed
* Because of how sites use the video tag, I have to manually "temporarily allow all of this type for this domain" to make it play
* That causes a page reload (but I've configured NoScript to only reload the current tab when rules change)
* When I finish watching, I revoke temporary permissions

That means that, for want of this feature, every single time I want to view a video, I have to:
1. Load the page
2. Select an entry from a second-level drop-down menu
3. Wait for the page to reload
4. Watch
5. Select "Revoke Temporary Permissions" from a first-level drop-down menu

(It also means that, for anything application-like which uses audio or video such as the appear.in WebRTC service, I'm likely to just use Chrome since NoScript's "Block embeddings even when JavaScript is allowed" feature has no "permanently allow" option of its own.)

I've taken to using a custom youtube-dl wrapper script because this is such a hassle that it's easier to watch YouTube by this means:
1. Triple-click the location bar (I'm on Linux, so that puts it in the selection)
2. Ctrl+Click my clipboard manager's icon
3. Select "Watch via youtube-dl+mplayer" from the actions popup
4. Do something else while I wait because I was too lazy to parallelize download and playback in my script
(In reply to Randall Barker [:rbarker] from comment #73)
> (In reply to Jared Wein [:jaws] (please needinfo? me) from comment #70)
> > Comment on attachment 8605930 [details] [diff] [review]
> > 0001-Bug-659285-Part-1-Extend-media.autoplay.enabled-to-provide-a-way-to-
> > disable-untrusted-play-invocations-15051411-906d0c3.patch
> > 
> > Review of attachment 8605930 [details] [diff] [review]:
> > -----------------------------------------------------------------
> > 
> > ::: mobile/android/themes/core/content.css
> > @@ +304,5 @@
> > >    -moz-binding: url("chrome://global/content/bindings/videocontrols.xml#touchControls");
> > >  }
> > >  
> > > +/* display error and click to play when autoplay is blocked for videos */
> > > +video:not([controls]) > xul|videocontrols {
> > 
> > I expect this to cause issues with sites that have custom controls and a
> > custom click-to-play overlay.
> 
> You will only see the click to play overlay if the video has no controls and
> it tries to autoplay so I'm not sure how it would have a custom
> click-to-play overlay.

Can you make that explicit in the CSS here?

video:not([controls])[autoplay]

I expect this to break sites that try to use video as a background image. http://www.destinythegame.com/ for example. They're being good citizens (using WEBM instead of GIF, and I think destiny actually falls back to static images on mobile) so I feel sad not just breaking their UI, but potentially making it really ugly. I wonder if we'd actually be better to just implement click-to-play with no UI, but we can let UX think about that.
Because I am demanding and horrible, I'd love it to know if it is possible to extend about:permissions to allow per site/domain autoplay? Ideally, it would work exactly like per-site plugin permissions. Despite adding steps, I think it would be the best compromise for all use cases. Something like this:

- Show a flashing "media LEGO brick" in the location bar for any page with autoplay media with the same options for Plugins.
- Users who disable media.autoplay can still whitelist well-behaved sites like https://www.elitedangerous.com/ and http://www.destinythegame.com/
- Default behavior is autoplay on tab visible on all platforms, because it is a great idea. However, users can still blacklist misbehaving sites. 

Should I file this as a separate bug? Thanks, all.
(In reply to Wesley Johnston (:wesj) from comment #76)
> I wonder if we'd actually be better to
> just implement click-to-play with no UI, but we can let UX think about that.

We need some UX input here. I would prefer that we not alter the UI but we probably need some way to let users know that there is a video available to play (doorhanger, infobar, etc).
Flags: needinfo?(philipp)
List of media will be ideal. I may not want to allow *all* media on a site. Also I don't want to let a site drain my connection and produce noises just because it is a good citizen. I may allow these if I decide so.
(In reply to Alexander from comment #79)
> List of media will be ideal. I may not want to allow *all* media on a site.
> Also I don't want to let a site drain my connection and produce noises just
> because it is a good citizen. I may allow these if I decide so.

I agree with you, especially on mobile. I was thinking more like this:

Default behavior
- Autoplay on tab visible
- Location bar media brick uses the permissions system to Block/Continue Allowing

Users who explicitly disable all autoplay (media.autoplay.enabled = False)
- All media that requests autoplay is now CtP
- Location bar media brick uses the permissions system to Allow/Allow and Remember

A list of all media with fine-grained options would make the most sense in the existing Media section of the Page Info Window. There is room for separate play/pause, stop, and mute buttons near the "Save As..." button. I think that would be super great -- but also lots of work (and a separate bug).
Removes UI changes from patch. No longer Android only.
Attachment #8605930 - Attachment is obsolete: true
Attachment #8605931 - Attachment is obsolete: true
Attachment #8605930 - Flags: review?(wjohnston)
Attachment #8605930 - Flags: review?(dolske)
I'm moving the UI and test changes to Bug 1166961. This bug will only be for blocking the autoplay video when started from a script.
Blocks: 1166961
Maybe we should only block playback (with this pref) for videos which have an audio track? Then good citizens who use <video> as a brackground image won't be blocked.
(In reply to Chris Pearce (:cpearce) from comment #83)
> Maybe we should only block playback (with this pref) for videos which have an audio track?

This could be OK if in practice, videos without an audio track are short. Otherwise there's the network bandwidth problem.

Still, users may not like videos because videos distract them. Isn't it similar to the case where image.animation_mode is set to "none"? Should this pref have an effect on the behavior of such videos?

> Then good citizens who use <video> as a brackground image won't be blocked.

Even for good citizens, one may not like some behaviors for various reasons.
(In reply to Chris Pearce (:cpearce) from comment #83)
> Maybe we should only block playback (with this pref) for videos which have
> an audio track? Then good citizens who use <video> as a brackground image
> won't be blocked.

This will also make the logic much more complicated since when blocking a video started by script the media info has not been loaded yet and we don't know if it has audio or not. The element would need to cache that the video was blocked, go ahead and load it, then test to see if it has audio and then bail out if it does.
Attachment #8610691 - Flags: review?(cpearce)
Blocks: 821695
Comment on attachment 8610691 [details] [diff] [review]
0001-Bug-659285-Extend-media.autoplay.enabled-to-provide-a-way-to-disable-untrusted-play-invocations-15052610-a046f30.patch

Review of attachment 8610691 [details] [diff] [review]:
-----------------------------------------------------------------

On desktop, this breaks the YouTube player, but you can still play by mashing the pause/play buttons for a bit.

I'm really worried by this change. It cripples video players with custom controls playback, which is most of them, and if someone turned this on and forgot about it they'd likely blame Firefox for being broken on YouTube.

We really need a permission prompt here to approve media playback on a page, don't you think?
I am really worried by data plan on my smartphone. And I am really worried by annoying noise what unwanted videos cause, and wake up my niece. This problem (the autoplay) exists on major sites too, for example nfl.com.

Of course nobody wants to brake anyting. But for Christ sakes: this bug is 4 years old.
Just solve it. Just make a solution, a possibility to disable it.
Even in the hidden depth of about:config. Even force the user to confirm it 3 times, if you care about stupid users.

Just please: make a solution. Please make it available first, then polish it.

Thanks
(In reply to Chris Pearce (:cpearce) from comment #88)
> Comment on attachment 8610691 [details] [diff] [review]
> 0001-Bug-659285-Extend-media.autoplay.enabled-to-provide-a-way-to-disable-
> untrusted-play-invocations-15052610-a046f30.patch
> 
> Review of attachment 8610691 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> On desktop, this breaks the YouTube player, but you can still play by
> mashing the pause/play buttons for a bit.
> 

From what I can tell, the youtube video player doesn't actually wait for the playing event to toggle the control when they autostart via a script. The pause button is displayed even though the video doesn't start. Pressing it once will bring up the play button. Pressing again will start the video. I would hope webcompat could reach out to youtube? I can't image this would be terribly hard to fix in the player.

> I'm really worried by this change. It cripples video players with custom
> controls playback, which is most of them, and if someone turned this on and
> forgot about it they'd likely blame Firefox for being broken on YouTube.
> 

It only cripples the player if it tries to autoplay which is what this is supposed to block. If the player doesn't try and autoplay, the user should never see it. I'm not sure what we can do for users that toggle preferences in about:config and then forget. I would assume that when we allowed the autoplay attribute to be blocked initially it broke some players?

> We really need a permission prompt here to approve media playback on a page,
> don't you think?

I disagree. The point of blocking autoplay is prevent the user from being annoyed by autoplaying media on pages they visit. A permission prompt would only be replacing one annoyance with another. The click to play overlay which I moved to 1166961 will at least give an indication that there is a video to play.
Comment on attachment 8610691 [details] [diff] [review]
0001-Bug-659285-Extend-media.autoplay.enabled-to-provide-a-way-to-disable-untrusted-play-invocations-15052610-a046f30.patch

Review of attachment 8610691 [details] [diff] [review]:
-----------------------------------------------------------------

My concern is that the user can flip this pref and end up in a state where it's not obvious how to get out of, i.e. videos don't play. But since you're planing on adding a click-to-play overlay in Fennec, which is the only place that this pref is exposed in the browser UI, then we can run with this since there will be a potential way for users to get out of the "Firefox is broken for no obvious reason" state.

Does the "click-to-play" button appear on <video> elements that don't use our built-in controls? These are the <video>s I'm worried about most. What about when the <video> has another transparent element overlaid on top of it so click events are blocked to the videocontrols?
Attachment #8610691 - Flags: review?(cpearce) → review+
(In reply to Chris Pearce (:cpearce) from comment #91)
> Comment on attachment 8610691 [details] [diff] [review]
> 0001-Bug-659285-Extend-media.autoplay.enabled-to-provide-a-way-to-disable-
> untrusted-play-invocations-15052610-a046f30.patch
> 
> Review of attachment 8610691 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> My concern is that the user can flip this pref and end up in a state where
> it's not obvious how to get out of, i.e. videos don't play. But since you're
> planing on adding a click-to-play overlay in Fennec, which is the only place
> that this pref is exposed in the browser UI, then we can run with this since
> there will be a potential way for users to get out of the "Firefox is broken
> for no obvious reason" state.
> 

While fennec currently passes youtube videos to the android youtube app, I understand we are looking at playing them in-browser in the near future so getting youtube to work when autoplay is block will probably need to be a priority.

> Does the "click-to-play" button appear on <video> elements that don't use
> our built-in controls? These are the <video>s I'm worried about most. What
> about when the <video> has another transparent element overlaid on top of it
> so click events are blocked to the videocontrols?

The click-to-play overlay is only displayed when an autoplay attempt is blocked from either the autoplay attribute or from a script not initiated by the user, and the <video> element does not have the controls attribute. If there is an element covering the video element, obviously the click-to-play overlay may not be visible or even clickable. This actually happens on youtube autoplay videos. I'm not certain their is good solution for this case.
Keywords: checkin-needed
(In reply to Randall Barker [:rbarker] from comment #92)
> The click-to-play overlay is only displayed when an autoplay attempt is
> blocked from either the autoplay attribute or from a script not initiated by
> the user, and the <video> element does not have the controls attribute. If
> there is an element covering the video element, obviously the click-to-play
> overlay may not be visible or even clickable. This actually happens on
> youtube autoplay videos. I'm not certain their is good solution for this
> case.

What about some combination of these ideas?

1. Include an address bar allow/deny token as with plugins but don't start with its dropdown expanded.
2. Display a click-to-play overlay over each video and make it appear above everything that may be covering up the video element.
3. Include a dismiss X in the overlay
4. When the overlay from #3 is dismissed, animate it flying into the address bar token from #1 as a cue for how to access the dismissed setting.
5. Include some mechanism for setting "start the overlay dismissed" for a given domain.
I would like to suggest you another option:
I agree with you about the possibility of blocking every unwanted media, but I would like to choose whether a site is allowed at all for me.

As an example:
I activate the option autoplay.enabled: false! I go to youtube and I see that every content is paused ... I decided that whenever I will come back to youtube I want to use it without a blocking, so I click on an icon with the right button and choose "allow YOUTUBE always"!

Is it possible?

Moreover: How can I test your script? Do I have to compile all the firefox software?
sorry had to back this out for bustage like https://treeherder.mozilla.org/logviewer.html#?job_id=10467410&repo=mozilla-inbound
Flags: needinfo?(rbarker)
Fix build breakage caused by Bug 1165515 landing.
Attachment #8610691 - Attachment is obsolete: true
Flags: needinfo?(rbarker)
Keywords: checkin-needed
Depends on: 1165515
https://hg.mozilla.org/mozilla-central/rev/a2ed39faf480
Status: REOPENED → RESOLVED
Closed: 12 years ago → 8 years ago
status-firefox41: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla41
So we wait until ff 41? Now we are at 38 and advertising is already catching up with html5.
If you want you can install ff41:
https://developer.mozilla.org/en-US/Firefox/Releases/41

Otherwise, you can simply wait ... You have waited for 3 years ... so 3-4 months are not too much! ;-P
(In reply to teo from comment #102)
> Otherwise, you can simply wait ... You have waited for 3 years ... so 3-4
> months are not too much! ;-P

Except that not everyone waited 3 years: web sites are evolving, and new problems appear even though the problem potentially existed before. For me, the problem became more critical less than 3 months ago.
(In reply to Vincent Lefevre from comment #103)
> (In reply to teo from comment #102)
> > Otherwise, you can simply wait ... You have waited for 3 years ... so 3-4
> > months are not too much! ;-P
> 
> Except that not everyone waited 3 years: web sites are evolving, and new
> problems appear even though the problem potentially existed before. For me,
> the problem became more critical less than 3 months ago.

If you don't want to switch to a more development-oriented release channel (I run Aurora exclusively with no issues), then your only other option is to install NoScript and set the embed controls to your liking.
(In reply to Chris Pearce (:cpearce) from comment #83)
> Maybe we should only block playback (with this pref) for videos which have
> an audio track? Then good citizens who use <video> as a brackground image
> won't be blocked.

While I respect your input, this is second time you contrary to quite clear user request for option to block html5 autoplay entirely come with suggestion of some partial blocking and in effect of something else instead, which, coming from what I assume is Mozilla dev, I find extremely distrubing (if sadly unsurprising), second only to explanation you provide. Whoever puts autoplaying video on web page most definitely is not a good citizen and even if he was, good citizens do not use Firefox, users do.
(In reply to maildir from comment #105)
> (In reply to Chris Pearce (:cpearce) from comment #83)
> > Maybe we should only block playback (with this pref) for videos which have
> > an audio track? Then good citizens who use <video> as a brackground image
> > won't be blocked.
> 
> While I respect your input, this is second time you contrary to quite clear
> user request for option to block html5 autoplay entirely come with
> suggestion of some partial blocking and in effect of something else instead,
> which, coming from what I assume is Mozilla dev, I find extremely distrubing
> (if sadly unsurprising), second only to explanation you provide. Whoever
> puts autoplaying video on web page most definitely is not a good citizen and
> even if he was, good citizens do not use Firefox, users do.

To be fair, it's IS consistent with the decision to force users to turn to extensions if they want click-to-play for animated GIFs. (Sadly, there's no extension I can find that does the same for the spec violation that is animated PNG.)
Bugzilla is not a discussion forum. If you need to discuss Mozilla development please use the appropriate list https://lists.mozilla.org/listinfo
Restrict Comments: true
Depends on: 1173844
Depends on: 1173848
Random thought: I wonder if this pref would work better if, instead of simply blocking the play, it made the media element get stuck at the loadedmetadata state (i.e., as if the user is on a slow connection, and the video is simply stalled / loading slowly).

As far as the page is concerned, it's still automatically playing and the state of the media element is as it normally expects it to be. The browser's UI (TBD) for allowing an autoplay would simply un-stall the media. There still might be risk of the page's player breaking if it has its own timeout, but otherwise it wouldn't even know that the browser is suppressing the autoplay.
(In reply to Justin Dolske [:Dolske] from comment #108)
> Random thought: I wonder if this pref would work better if, instead of
> simply blocking the play, it made the media element get stuck at the
> loadedmetadata state (i.e., as if the user is on a slow connection, and the
> video is simply stalled / loading slowly).

I think YouTube detects this and helpfully fallsback to Flash in this case...
Depends on: 1178858
Randall, would you mind summarizing what your patch does and doesn't do? There were a lot of ideas thrown around here and it's a bit hard to follow the implementation. 

I'm asking for doc purposes and also potential follow-ups for some of the ideas mentioned here (also @cpearce).
Flags: needinfo?(rbarker)
Flags: needinfo?(cpearce)
docs: see comment 111 and followups

(sorry for bugspam)
Keywords: dev-doc-needed, user-doc-needed
OS: Linux → Unspecified
Hardware: x86_64 → Unspecified
The current solution that has landed in 41 prevents the media element play() function from actually starting the media playback unless there is or has been human interaction when play() is invoked where media.autoplay.enabled = false.
Flags: needinfo?(rbarker)
(In reply to Florian Bender from comment #111)
> I'm asking for doc purposes and also potential follow-ups for some of the
> ideas mentioned here (also @cpearce).

I am working on a patch to block media playback until the page in which the audio/video element is contain is visible. This would be behind a pref (to start with at least). This way background tabs don't surprise you and starting playing media when you're not interacting with them.
Flags: needinfo?(cpearce)
Depends on: 1196726
Depends on: 1182329
Blocks: 1209957
OMG, what has happened with the patch - has it landed yet? 

I am on Ffx 42, autoplay disabled, no flash, all plugins deactivated, click-to-play-per-element addon installed.
Firefox crashes on every youtube page after causing massive data traffic. 

I do not want autoplay of anything ever. Horrible experience. 
I want a browser, not a video center.
(In reply to Richard Z. from comment #115)
> OMG, what has happened with the patch - has it landed yet? 
> 
> I am on Ffx 42, autoplay disabled, no flash, all plugins deactivated,
> click-to-play-per-element addon installed.
> Firefox crashes on every youtube page after causing massive data traffic. 
> 
> I do not want autoplay of anything ever. Horrible experience. 
> I want a browser, not a video center.

The patch has landed, and this bug is closed. If you have a problem, please open a new report. If you have links to your crashes (from about:crashes) those can be very helpful. Thanks.
See Also: → 1277813
You need to log in before you can comment on or make changes to this bug.