Closed Bug 1294154 Opened 3 years ago Closed 3 years ago

[MSE] The infinite search if in the buffer there aren't enough frames.

Categories

(Core :: Audio/Video: Playback, defect, P3)

48 Branch
x86_64
Windows 7
defect

Tracking

()

RESOLVED FIXED
mozilla51
Tracking Status
firefox51 --- fixed

People

(Reporter: CoolCmd, Assigned: jya)

References

()

Details

Attachments

(2 files)

Attached file testcase
User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:48.0) Gecko/20100101 Firefox/48.0
Build ID: 20160726073904

Steps to reproduce:

load testcase and open console



Actual results:

after second appendBuffer(), seek started ("seeking" event fired), but never ended ("seeked" event never fired).

as i understand, Firefox needs certain amount of buffered frames to complete seek. after second appendBuffer(), only 1 frame buffered 2 seconds long (first 59 frames not begin with key frame and thus ignored).

after third appendBuffer(), there enough frames to complete seek, but FF not do that. Chrome, for example, resumes and completes seeking.

unfortunately, JS do not know, how many frames contain buffer at any point. so playback just never start... this situation quite often happens during live streaming with variable frame rate (some frames are lost during sending video through network, etc).


Expected results:

Firefox must resume and complete seeking after third appendBuffer().
Component: Untriaged → Audio/Video: Playback
OS: Unspecified → Windows 7
Product: Firefox → Core
Hardware: Unspecified → x86_64
Assignee: nobody → jyavenard
Firefox doesn't need a certain amount of frames to complete seeking. Just one will do
Flags: needinfo?(jyavenard)
segment 0 is the init segment.
segment 1 contains:
1.m4v: 69 frames, none of them being marked as keyframe. 
followed by 6 frames, first one being a keyframe 844.911000 to 845.119455
As per spec: https://w3c.github.io/media-source/index.html#sourcebuffer-coded-frame-processing, at start need random access point flag is true, so all frames prior a keyframes are dropped.
So the video track buffered range is [844.911000, 845.119455)
1.m4a: 118 audio frames, from 842.024666 to 844.542000, so the audio buffered range is [842.024666,844.542000)

The buffered range is the intersection of the audio and video track is [(844.911, 845.119455)]
The script then seek to 844.911.

Last week, I would have closed this bug as Invalid, as per spec then:
https://www.w3.org/TR/media-source/#mediasource-seeking
"The media element looks for media segments containing the new playback position in each SourceBuffer object in activeSourceBuffers.
If one or more of the objects in activeSourceBuffers is missing media segments for the new playback position
        If the HTMLMediaElement.readyState attribute is greater than HAVE_METADATA, then set the HTMLMediaElement.readyState attribute to HAVE_METADATA."

there isn't anything to seek to in the audio track. so seeking can't complete.

However, since https://github.com/w3c/media-source/issues/147 got closed last week (bug that we opened against the spec), we now have clarifications that seeking must complete if the target is in the buffered range.
Status: UNCONFIRMED → RESOLVED
Closed: 3 years ago
Flags: needinfo?(jyavenard)
Resolution: --- → DUPLICATE
Duplicate of bug: 1297036
Jean-Yves Avenard, no, this bug is NOT duplicate of 1297036:
1. testcase 1294154 contain ONLY video.
2. buffer HAVE video data to complete seek.
3. endOfStream() was NEVER called.

Please check testcase again.

>Firefox doesn't need a certain amount of frames to complete seeking. Just one will do
OK, forget about 'amount of frames'. Just check why seek cannot complete.
Flags: needinfo?(jyavenard)
If you followed the bug that has been linked to. Or try with a current Firefox Nightly you would have seen that the issue has been corrected. 

All the explanation on why has been provided in comment 2. There's nothing more to add
Flags: needinfo?(jyavenard)
And btw; your samples contain audio. It is NOT video only
Jean-Yves Avenard, i install Firefox Nightly 51.0a1 20160830030201.
Bug 1297036 fixed, but this bug 1294154 NOT.
Testcase 1294154 have different JavaScript file and different segments. Those segments don't have audio and have different frame timestamps.
My apologies, you named all your zip files the same across various bugs, as a result i used the wrong file..

works for me on linux. will test on windows later...
not working on windows only.. fine on mac and linux
Status: RESOLVED → REOPENED
Ever confirmed: true
Resolution: DUPLICATE → ---
The issue is that WMF returns an invalid frame when all the decoder was given is a single frame.
The sample returns always have a pts of 0 and a duration of 1/30.

The coming up patch, detect store the time and duration of the last sample and when draining detects if only one frame was given.

I'm not sure if there's a better way, but this will do for now
Comment on attachment 8787593 [details]
Bug 1294154: Handle case where WMF is unable to determine time and duration.

https://reviewboard.mozilla.org/r/76310/#review74672

This ia pretty sad, but I guess it should work fine.
Attachment #8787593 - Flags: review?(matt.woodrow) → review+
Pushed by jyavenard@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/316b7259747b
Handle case where WMF is unable to determine time and duration. r=mattwoodrow
https://hg.mozilla.org/mozilla-central/rev/316b7259747b
Status: REOPENED → RESOLVED
Closed: 3 years ago3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla51
You need to log in before you can comment on or make changes to this bug.