1394234 - [MSE] remove doesn't remove partial frames.

Status: RESOLVED INVALID

(Core :: Audio/Video: Playback, enhancement)

Description

[Jean-Yves Avenard [:jya]]

remove doesn't remove a partial frame.

in the Coded Frame Removal Algorithm, https://w3c.github.io/media-source/index.html#sourcebuffer-coded-frame-removal
step 3.
"Remove all media data, from this track buffer, that contain starting timestamps greater than or equal to start and less than the remove end timestamp. 
I strongly suggest that you read the latest MSE spec to learn how MSE works, what it does, how it does it, and its limitations:
https://w3c.github.io/media-source/index.html

Say in our source buffer we have a frame with a start time of 29 and a end time of 31.

We call remove with the arguments 0 and 30.

https://dxr.mozilla.org/mozilla-central/source/dom/media/mediasource/TrackBuffersManager.cpp#582

This calls TrackBuffersManager::FrameRemoval:
https://dxr.mozilla.org/mozilla-central/source/dom/media/mediasource/TrackBuffersManager.cpp#1856

Which use the removal algorithm as specified in the Coded Frame Processing algorithm where a frame is to be removed if overlapped with a new frame only if:

"14. Remove existing coded frames in track buffer:
  //  If highest end timestamp for track buffer is not set:
  //   Remove all coded frames from track buffer that have a presentation timestamp greater than or equal to presentation timestamp and less than frame end timestamp.
  //  If highest end timestamp for track buffer is set and less than or equal to presentation timestamp:
  //   Remove all coded frames from track buffer that have a presentation timestamp 
greater than or equal to highest end timestamp and less than frame end timestamp"

However, in the code we read:
TimeInterval sampleInterval =
      TimeInterval(sample->mTime, sample->GetEndTime());
    if (aIntervals.Contains(sampleInterval)) {

a partial frame will never be contained in aIntervals, and as such will not be removed.

This is incorrect, only the start time is to be used to determine if a frame is removed or not.

Comment 1

[Jean-Yves Avenard [:jya]]

All good. the code behaves as per spec.

If the RemoveFrames behave as described in the description, the way we calculate the interval passed to RemoveFrames means that partial frames are removed as per spec. We remove all frames until the next keyframe, or until the end.

frames that are partially contained but with the start time not within the interval, won't be removed.
frames fully contained in the interval will be removed
frames that are partially contained with the start time within the interval, but not the end time will be removed.