1280023 - [MSE] Investigate different eviction algorithms

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P3)

Description

[Jean-Yves Avenard [:jya]]

In bug 1279885, comments were made about how data is evicted in the mse sourcebuffer.

There are currently two policies in use:
1- Evict past data already played (e.g prior currentTime)
2- Evict future data if it's far enough from currentTime (currently set starting from the end up to 30s away from currentTime).

This is similar logic to what chrome was doing and was expected by YouTube dash player (as per their MSE conformance test "20. DASHLatency" and "29. VideoBufferSize", where data is appended until a discontinuity in the buffered range is found at which case it is assumed data was evicted and the buffer size has been calculated.

Because we can almost always evict data, the QuotaExceededError exception will almost never be fired.

Actual usage has shown that almost no DASH player out there are actually listening to errors emitted by the source buffer; worse some (like YouTube) used to make those errors fatals. If QuotaExceededError was fired, then playback would stop.
However, evicting data in the future, has often be the cause for stalls as most DASH player don't actually check the buffered range to determine if eviction has occurred (unlike what the MSE specs state). So it would be good to remove future eviction.

From the description of bug 1279885, it appears that other browsers are now properly emitting QuotaExceededError, so dash player are more likely to handle those properly now.

So this bug is about investigating new eviction policy.
The one I have in mind is never evicting contiguous data. So if there's already a discontinuity in the buffered range in the future, than we can evict the data after that discontinuity.
Otherwise, leave it alone and don't evict anything and fire QuotaExceededError instead.

This would prevent for any discontinuity to ever be created because data got evicted just after having been added.

Comment 1

[xqq]

You said that if there's a discontinuity in the buffered range in the future, the data after discontinuity can be evicted. As far as I remember in the past experiments during the development of my player, this policy is also adopted by Chrome. Ranges after discontinuity may be evicted if leading data is still being played. This may needs to be verified by source code of chromium/media.

I also observed that hls.js is always monitoring for QuotaExceededError for appendBuffer() call, handles this exception, and report BUFFER_FULL to upper-layer. In older version of hls.js (later 2015), hls.js swallows the exception and defers the buffer for future append.

https://github.com/dailymotion/hls.js/blob/master/src/controller/buffer-controller.js#L328

Comment 2

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Mass change P2 -> P3

Comment 3

[Jean-Yves Avenard [:jya]]

Attachment: Bug 1280023: [MSE] P1. Never evict data in the media segment currently being played.

Review commit: https://reviewboard.mozilla.org/r/77620/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/77620/

Comment 4

[Jean-Yves Avenard [:jya]]

Attachment: Bug 1280023: [MSE] P2. Drop audio size threshold to 10MB.

Following bug 1301307, exception is properly thrown when buffer is full, giving the opportunity to the JS player to adjust accordingly. Confirmed to work with YouTube with an audio threshold of only 1MB.

Review commit: https://reviewboard.mozilla.org/r/77622/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/77622/

Comment 5

[Jean-Yves Avenard [:jya]]

It would be great if you could test those changes in Nightly in a couple of day to see if it's now behaving like you expect.

Thank you

Comment 6

[Jean-Yves Avenard [:jya]]

Watching a long youtube video, we can see the eviction at hand, working perfectly:
https://www.youtube.com/watch?v=eyU3bRy2x44
	blob:https://www.youtube.com/dcd1ec65-00b0-fe46-bb99-00789b9ca5b3
	currentTime: 1103.456
	Quality: 100% (total:33261 dropped:9 corrupted:0)
	Buffered ranges: [(959.761, 1296.296)]
		SourceBuffer 0
			start=959.761 end=1296.401
		SourceBuffer 1
			start=295.296 end=1296.296
	Internal Data:
	audio decoder: opus audio decoder
	audio frames decoded: 55291
	audio state: ni=0 no=0 ie=0 demuxr:0 demuxq:0 tt:-1.000000 tths:-1 in:55291 out:55291 qs=0 pending:0 waiting:0 sid:216
	video decoder: blank media data decoder
	hardware video decoding: disabled
	video frames decoded: 33258 (skipped:0)
	video state: ni=0 no=0 ie=0 demuxr:0 demuxq:0 tt:-1.000000 tths:-1 in:20104 out:20104 qs=0 pending:0 waiting:0 sid:215
	Dumping data for demuxer 1a7fa0000:
		Dumping Audio Track Buffer(audio/webm; codecs=opus): - mLastAudioTime: 1105.821000
			NumSamples:16832 Size:10059988 NextGetSampleIndex:7303 NextInsertionIndex:16832
			Buffered: ranges=[(959.761000, 1296.401000)]
		Dumping Video Track Buffer(video/webm; codecs=vp9) - mLastVideoTime: 1104.170000
			NumSamples:30000 Size:99840846 NextGetSampleIndex:24242 NextInsertionIndex:30000
			Buffered: ranges=[(295.296000, 1296.296000)]
	

10MB of audio or whereabout, and 100MB of video. evictions have occurred at different time, here always prior currentTime.

After seeking to an area prior the start of the buffered range:
	Internal Data:
	audio decoder: opus audio decoder
	audio frames decoded: 60516
	audio state: ni=0 no=0 ie=0 demuxr:0 demuxq:0 tt:-1.000000 tths:-1 in:333 out:333 qs=0 pending:0 waiting:0 sid:220
	video decoder: ffvpx video decoder
	hardware video decoding: disabled
	video frames decoded: 36455 (skipped:0)
	video state: ni=0 no=0 ie=0 demuxr:0 demuxq:0 tt:-1.000000 tths:-1 in:160 out:159 qs=1 pending:0 waiting:0 sid:215
	Dumping data for demuxer 1a7fa0000:
		Dumping Audio Track Buffer(audio/webm; codecs=opus): - mLastAudioTime: 212.181000
			NumSamples:2500 Size:1495144 NextGetSampleIndex:609 NextInsertionIndex:2500
			Buffered: ranges=[(200.001000, 250.001000)]
		Dumping Video Track Buffer(video/webm; codecs=vp9) - mLastVideoTime: 210.544000
			NumSamples:1650 Size:5309682 NextGetSampleIndex:310 NextInsertionIndex:1650
			Buffered: ranges=[(200.200000, 255.256000)]

Here YouTube called sourceBuffer.remove(0.000000, 1370.970000). They cleaned the entire sourcebuffer, so can't test how the behaviour would work.

Note that we now pass the YouTube eviction test and http://www.w3c-test.org/media-source/mediasource-appendbuffer-quota-exceeded.html which we used to fail

Comment 7

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Comment on attachment 8789338 [details]
Bug 1280023: [MSE] P1. Never evict data in the media segment currently being played.

https://reviewboard.mozilla.org/r/77620/#review75882

Comment 8

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Comment on attachment 8789339 [details]
Bug 1280023: [MSE] P2. Drop audio size threshold to 10MB.

https://reviewboard.mozilla.org/r/77622/#review75884

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/16ee06bc3479
[MSE] P1. Never evict data in the media segment currently being played. r=gerald
https://hg.mozilla.org/integration/autoland/rev/d441ac0b35ae
[MSE] P2. Drop audio size threshold to 10MB. r=gerald

Comment 10

[xqq]

I will take a test after the Nightly build released.

Comment 11

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/16ee06bc3479
https://hg.mozilla.org/mozilla-central/rev/d441ac0b35ae

Comment 12

[xqq]

Seems that QuotaExceededError will be throwed if buffer full and no data can be evicted;

but QuotaExceededError is still being throwed if current playback position near the end of buffered range !!!

I means that, although QuotaExceededError has been throwed during past appendBuffer call, when playback position goes by (e.g. near the end), data in the front of a continuous buffered-range can be evicted safely. Then appendBuffer operation should be success and no exception should be throwed.

Comment 13

[Jean-Yves Avenard [:jya]]

Please don't change the status. open another bug if you think there's another problem.

However, I don't see how the situation you describe could happen...

If currentTime is close to the end, it means that data can be evicted at the start. That we can't evict data in the future doesn't mean that no data at all can be evicted.

And this is the first data that will be evicted; we always attempt to evict data at the front, prior currentTime.
https://dxr.mozilla.org/mozilla-central/source/dom/media/mediasource/TrackBuffersManager.cpp#423

QuotaExceededError is only thrown if no data could be evicted *and* the sourcebuffer size is too big (100MiB for video, 10MiB for audio)