Closed Bug 1483703 Opened Last year Closed Last year

Don't block the autoplay media element without audio track

Categories

(Core :: Audio/Video: Playback, enhancement, P2)

enhancement

Tracking

()

RESOLVED FIXED
mozilla63
Tracking Status
firefox63 --- fixed

People

(Reporter: alwu, Assigned: alwu)

References

(Blocks 1 open bug)

Details

Attachments

(5 files)

Because the media element without audio track is non-audible, we should allow it to autoplay like muted media.
Priority: -- → P3
Hi, Chris, 

I think we should block play() if the media element hasn’t loaded metadata yet.

Although you think it’s racy (from bug1464922 comment0), but I think it’s not, because of following reasons. 

From the threads perspective, changeReadyState() and Play() are both running in main thread, so it won’t race.

From the depending relationship perspective, we have already made the result of play promise depending on the value of different attributes, such as mute, volume and user activation flag. Therefore, it seems reasonable to me to make play() adding one more dependency, ready state.

—

My idea is that, 

(1) if media element have not loaded metadata when it requests play(), start to load the resource of media element, and reject the play promise.

(2) if media element which had loaded metadata and no audio track requests play, reject play promise.

How do you think?

Thank you!
Depends on: 1464922
Flags: needinfo?(cpearce)
Priority: P3 → P2
The cases I'm worried about:

{
    // case 1
    let v = document.createElement("video");
    v.src = "http://pearce.org.nz/video/320x240.ogg";
    v.play();
    document.body.appendChild(v);
}
{
    // case 2
    let v = document.createElement("video");
    v.src = "http://pearce.org.nz/video/320x240.ogg";
    v.addEventListener("loadedmetadata", () => v.play(), { once: true});
    document.body.appendChild(v);            
}

A lot of sites do case 1. By the time the site calls play(), we won't have reached loadedmetadata yet, and so the play would be blocked because we don't know that the video doesn't have an audio track yet. But in case 2, we'll have reached loadedmetadata by the time we call play(), and so we'll know that 320x240.ogg doesn't have an audio track, and so the play will not be blocked.

Basically, I think we should try to make both these cases work.
Flags: needinfo?(cpearce)
Alwu and I discussed this, and decided that if we've reached loadedmetadata at the time that play() is called we'll allow autoplay if the media doesn't have an audio track, but we won't inject a "wait until we've reached loadedmetadata" step.

So potentially we'll get different results if play() is called before loadedmetadata than if it's called after, but at least we'll fix the apple.com case.
Duplicate of this bug: 1479986
We would allow media without audio track to autoplay after it had loaded the metadata.

If media hasn't loaded metadata yet, we would treat it as audible media and then block it.
Add two telemetry scarlar,

"MEDIA_BLOCKED_NO_METADATA" records how many media which was blocked because it hadn't loaded metadata yet.
"MEDIA_BLOCKED_NO_METADATA_ENDUP_NO_AUDIO_TRACK" records how many media which was blocked because it hadn't loaded metadata and ended up for being no audio track.

By collecting those data, we can know the proportion of media which should be autoplay but was blocked because of lacking metadata.
Since we don't block media without audio track anymore, the original telemetry scalar becomes useless.

We need to change its meaning in order to know the number of allowed autoplay without audio track.
Comment on attachment 9002132 [details]
Bug 1483703 - part1 : allow media without audio track to autoplay.

Chris Pearce (:cpearce) has approved the revision.
Attachment #9002132 - Flags: review+
Comment on attachment 9002134 [details]
Bug 1483703 - part2 : add telemetry for the media which was blocked before loading metadata and ended up being without audio track.

Chris Pearce (:cpearce) has approved the revision.
Attachment #9002134 - Flags: review+
Comment on attachment 9002135 [details]
Bug 1483703 - part3 : modify test.

Chris Pearce (:cpearce) has approved the revision.
Attachment #9002135 - Flags: review+
Comment on attachment 9002138 [details]
Bug 1483703 - part4 : modify current telemtry scalar because we won't block media without audio track anymore.

Chris Pearce (:cpearce) has approved the revision.
Attachment #9002138 - Flags: review+
Attached file Data review request
Attachment #9002593 - Flags: review?(francois)
Comment on attachment 9002593 [details]
Data review request

1) Is there or will there be **documentation** that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes, in Scalars.yaml.

2) Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, telemetry setting.

3) If the request is for permanent data collection, is there someone who will monitor the data over time?**

Not permanent.

4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under?  **

Category 1.

5) Is the data collection request for default-on or default-off?

Default ON, only in pre-release channels.

6) Does the instrumentation include the addition of **any *new* identifiers** (whether anonymous or otherwise; e.g., username, random IDs, etc.  See the appendix for more details)?

No.

7) Is the data collection covered by the existing Firefox privacy notice?

Yes.

8) Does there need to be a check-in in the future to determine whether to renew the data?

No, telemetry alerts are enough.
Attachment #9002593 - Flags: review?(francois) → review+
Comment on attachment 9002138 [details]
Bug 1483703 - part4 : modify current telemtry scalar because we won't block media without audio track anymore.

François Marier [:francois] has approved the revision.
Attachment #9002138 - Flags: review+
Pushed by alwu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/cd6a42517bb9
part1 : allow media without audio track to autoplay. r=cpearce
hmm, need to wait the patch2 getting r+, and then I could push other patches.
Comment on attachment 9002134 [details]
Bug 1483703 - part2 : add telemetry for the media which was blocked before loading metadata and ended up being without audio track.

François Marier [:francois] has approved the revision.
Attachment #9002134 - Flags: review+
The comment17 I pushed is the wrong version which became closed and I can't update it again. 
still trying to find the way to reopen and update it...
Pushed by alwu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/71ad474cdeeb
part1 : allow media without audio track to autoplay. r=cpearce
Pushed by alwu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/3326c0e920d4
part2 : add telemetry for the media which was blocked before loading metadata and ended up being without audio track. r=cpearce,francois
Pushed by alwu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/bb9759881cd7
part4 : modify current telemtry scalar because we won't block media without audio track anymore. r=cpearce,francois
You need to log in before you can comment on or make changes to this bug.