Bugzilla

The behaviour change from Bug 1540573 was to change the default preload action from "none" to "metadata" when we're on Wifi. When the preload action is "metadata", we'll suspend the download sometime soon after loading the first frame.

The error we see is NS_ERROR_NOT_INITIALIZED. What happens is when we load the page the media loads to HAVE_METADATA and since it's not playing, then goes dormant and we shutdown the decoder. When you hit play, the media comes out of dormant, and seeks to its previous position. We create a new decoder, wrapped by a MediaChangeMonitor. The first frame out of the HLSDemuxer is invalid (according to H264::GetFrameType()), and isn't a keyframe, so the MediaChangeMonitor doesn't have the extradata it needs to create a decoder. The frames which come out of the HLSDemuxer are in AnnexB format (the GeckoHlsVideoRenderer is prepending the initData as expected), and the Java side never set the out of band mExtraData fields.

The problem is that the HLSDemuxer underneath uses ExoPlayer in Java. We have a custom renderer which accepts the still compressed frames and forwards them out to the C++ demuxer. ExoPlayer's seek function doesn't seek to the keyframe before the seek target, it's an accurate seek. So when you seek, there's nothing to ensure the HLSDemuxer gives you a key frame. Additionally, the samples ExoPlayer outputs don't have a duration. So the GeckoHlsVideoRenderer queues samples output by ExoPlayer until it can calculate a duration for them, and only releases samples once they have a duration. The seek code doesn't clear that queue; meaning after a seek, the HlsDemuxer will output samples from the previous playback position for a while.

The GeckoHlsPlayer (Java) gets a callback when ExoPlayer detects a discontinuity, so we can clear the GeckoHlsVideoRenderer's queue of samples there in order to fix that. That however causes us to basically seek the video stream to the keyframe after the seek target. This is particularly jarring in the case where we're loading a new video, as we'll be showing the first frame, go dormant, and the seek to come out of dormant will advance us to the first keyframe after the start time. i.e., we'll skip ahead the first time a user plays. The same thing will happen any time we pause mid-stream long enough to go dormant.

Probably the best thing to do here is change the seek to seek to a point some arbitrary distance behind the seek target, clear the GeckoHlsVideoRenderer's queue of samples, and demux forwards until we hit a keyframe. Only once both conditions are true, should the HlsDemuxer emit samples after a seek.

The behaviour change from Bug 1540573 was to change the default preload action from "none" to "metadata" when we're on Wifi. When the preload action is "metadata", we'll suspend the download sometime soon after loading the first frame.

The error we see is NS_ERROR_NOT_INITIALIZED. What happens is when we load the page the media loads to HAVE_METADATA and since it's not playing, then goes dormant and we shutdown the decoder. When you hit play, the media comes out of dormant, and seeks to its previous position. We create a new decoder, wrapped by a MediaChangeMonitor. The first frame out of the HLSDemuxer is invalid (according to H264::GetFrameType()), and isn't a keyframe, so the MediaChangeMonitor doesn't have the extradata it needs to create a decoder. The frames which come out of the HLSDemuxer are in AnnexB format (the GeckoHlsVideoRenderer is prepending the initData as expected), and the Java side never set the out of band mExtraData fields.

The problem is that the HLSDemuxer underneath uses ExoPlayer in Java. We have a custom renderer which accepts the still compressed frames and forwards them out to the C++ demuxer. ExoPlayer's seek function doesn't seek to the keyframe before the seek target, it's an accurate seek. So when you seek, there's nothing to ensure the HLSDemuxer gives you a key frame. Additionally, the samples ExoPlayer outputs don't have a duration. So the GeckoHlsVideoRenderer queues samples output by ExoPlayer until it can calculate a duration for them, and only releases samples once they have a duration. The seek code doesn't clear that queue; meaning after a seek, the HlsDemuxer will output samples from the previous playback position for a while.

The GeckoHlsPlayer (Java) gets a callback when ExoPlayer detects a discontinuity, so we can clear the GeckoHlsVideoRenderer's queue of samples there in order to fix that. That however causes us to basically seek the video stream to the keyframe after the seek target. This is particularly jarring in the case where we're loading a new video, as we'll be showing the first frame, go dormant, and the seek to come out of dormant will advance us to the first keyframe after the start time. i.e., we'll skip ahead the first time a user plays. The same thing will happen any time we pause mid-stream long enough to go dormant.

Probably the best thing to do here is change the seek to seek to a point some arbitrary distance behind the seek target, clear the GeckoHlsVideoRenderer's queue of samples, and demux forwards, dropping frames until we hit a keyframe.