1269830 - After calling endOfStream and encountering a buffer gap, we fire ended

Status: RESOLVED INVALID

(Core :: Audio/Video: Playback, defect)

Description

[Dylan Dove]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.86 Safari/537.36

Steps to reproduce:

Visit this link: https://s3.amazonaws.com/_bc_dml/firefox-mse/index.html





Actual results:

Notice the gaps in the buffer as made evident by the visual highlights. After we play into the buffer gap, we are sent to the end of the video, although there is still data in the buffer 10 seconds ahead of us. 




Expected results:

According to MSE spec, we are meant to enter a 'waiting' state after encountering a gap in the buffer, so that we can attempt to close the gap. Since we have called endOfStream, we fire 'ended' and seek to the end of the video.

Comment 1

[Loic]

I tested after bug 1268868 in Nightly, it doesnt change anything.

Comment 2

[Jean-Yves Avenard [:jya]]

(In reply to Dylan Dove from comment #0)

> According to MSE spec, we are meant to enter a 'waiting' state after
> encountering a gap in the buffer, so that we can attempt to close the gap.

Which spec would that be ?
These are the MSE spec as we go by. https://w3c.github.io/media-source/index.html

MSE doesn't provide any rules in regards to playback, it only handles readyState and buffered management.

> Since we have called endOfStream, we fire 'ended' and seek to the end of the
> video.

Calling endOfStream indicates that the media source can't expect any new data to arrive.
The state of the mediasource is now "ended".

The definition of the "waiting" event is per spec:
https://html.spec.whatwg.org/multipage/embedded-content.html#event-media-waiting
"Playback has stopped because the next frame is not available, but the user agent expects that frame to become available in due course. "

the user agent doesn't expect that the frame will become available in due course anymore; it's been notified that no more data will arrive, the mediasource state is "ended".

Unfortunately, there's no consideration for this case in the media element spec.

So while I agree that "ended" is probably not the best event to fire ; per the spec, it's as good as any. Playback has ended and will not progress further.

The "stalled" event would be another solution, however the definition of "stalled" is "The user agent is trying to fetch media data, but data is unexpectedly not forthcoming. "
and "networkState is NETWORK_LOADING. " which is never true with MSE. So that too is unsuitable.

What's certain to me however, is that firing "waiting" would be against the spec.

You may take that up with W3C directly and when the ambiguity is resolved, reopen a new issue.

In the mean time, I believe this bug is invalid.

Comment 3

[Jean-Yves Avenard [:jya]]

to summarise: don't call endOfStream if you know you need to add more data for playback to be uninterrupted.

Comment 4

[Brandon Casey]

To me ended does not seem like a good event to choose here

endOfStream as per the spec "signals the end of the stream" to me this means that we have fetched all of the data that we are going to. As we have a gap in the buffer we know that it will not be filled. So the question is what event do we fire?

ended: (this is currently what is fired)
In the spec ended is defined as "Playback has stopped because the end of the media resource was reached." and media resource is defined as "The term media resource is used to refer to the complete set of media data". So when the gap occurs we are not yet at the end of a media resource as there is more data in the media resource that is buffed after the gap. This means that ended should not yet be fired because we have yet to reach the end of the media resource.

Comment 5

[Jean-Yves Avenard [:jya]]

(In reply to Brandon Casey from comment #4)
> To me ended does not seem like a good event to choose here
> 
> endOfStream as per the spec "signals the end of the stream" to me this means

end of stream means just that, the stream has ended, so no more data will be added.
it puts the mediasource in "ended" state.

This is further confirmed as if you were to call appendBuffer again, the mediasource state will move back to "open"


> that we have fetched all of the data that we are going to. As we have a gap
> in the buffer we know that it will not be filled. So the question is what
> event do we fire?

that I agree with, and unfortunately, there's no real provision for that in the spec. "stalled" it IMHO, the most closely related.

What I'm certain is that "waiting" would be the wrong event to fire. We've specifically told the mediasource that no more data will be added. We are no longer waiting for anything.

> 
> ended: (this is currently what is fired)
> In the spec ended is defined as "Playback has stopped because the end of the
> media resource was reached." and media resource is defined as "The term
> media resource is used to refer to the complete set of media data". So when
> the gap occurs we are not yet at the end of a media resource as there is
> more data in the media resource that is buffed after the gap. This means
> that ended should not yet be fired because we have yet to reach the end of
> the media resource.

Regardless, endOfStream is called wrongly if you intend to add more data, or if you know data is missing in the stream.

So we're trying to add handling for a condition that shouldn't happen in the first place.

Comment 6

[Dylan Dove]

A good analogy my friend made: 
"If you were designing an airbag, and the specifications said to make the airbag explode, but did not specify the strength of the explosion, why would you make the explosion so large that it would destroy the whole car?" 

I think this definitely applies to this situation. While we have reached the end of the stream, and we don't expect any more data to be sent, we still can find ways of recovering playback. The point is that by firing ended, we are not enabling developers to solve the problem, and instead of being able to graciously recover, we are crashing the whole player.

May I suggest an improvement: Since the spec does not specify what event to call when we reach the end of a buffered region, why don't we just not fire an event. I believe this, at least in the short term, will solve many problems, and we can leave it up to the discretion of the developers on how to handle a situation like this.

Comment 7

[Jean-Yves Avenard [:jya]]

How about you don't call endOfStream if there are more data to be added and you have gaps in your buffered range ahead of currentTime?

If your player is crashing, the issue is in your player.