Closed Bug 1185886 Opened 9 years ago Closed 9 years ago

Dash Live streams don't work in Firefox with MSE

Categories

(Core :: Audio/Video, defect)

Product:
Core
Component:
Audio/Video
Version:
39 Branch
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla42
Tracking Flags:
Tracking Status
firefox42 --- fixed

People

(Reporter: kbagga, Assigned: jya)

References

(Blocks 1 open bug)

Details

Attachments

(2 files)

P1. Ensure DrainComplete() is called when draining.
779 bytes, patch
cpearce
: review+
Details | Diff | Splinter Review
P2. Don't unnecessarily drain decoders when there's no frames pending.
1.40 KB, patch
cpearce
: review+
Details | Diff | Splinter Review 
User Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.132 Safari/537.36

Steps to reproduce:

1) Enable all config flags in Firefox to enable Media Source extensions and also set media.mediasource.whitelist to true.
2) Goto the dash.js latest reference player @ http://dashif.org/reference/players/javascript/1.4.0/samples/dash-if-reference-player/index.html
3) In the Stream text box, input url: http://54.201.151.65/livesim/testpic_2s/Manifest.mpd
4) Vod Streams like '' play fine
5) Live streams play back fine on latest chrome and IE 11 on Win 8.1



Actual results:

Live streams don't play back


Expected results:

Live streams should play. I would be glad to provide more inputs if required
Summary: Dash Live streams don't work in firefox → Dash Live streams don't work in firefox with Media Source extensions
I can't see anything wrong with our implementation at first glance.

We're only receiving the init segment for both streams (audio and video).

the "loadedmetadata" event is properly fired.

Not sure what more this dash.js player is waiting on to start feeding more data.
As much I debugged dash.js, the flow is:
1) For live streams set duration to POSITIVE INFINITY
2) Seek to the live edge time which is finite number. We seek to the point before appending the data for that time assuming that when media is appended for that time, playback will start.
3) After this seek, we depend on seeking event from the video model for the code to start. This event is not getting triggered in Firefox.
4) I tried sending a fake event in dash.js so that the workflow gets started, but then I see buffered ranges as negative. I left it at this point.

I can setup some demo page to isolate these 2 issues at least if required. That will ease out the dash.js complexity for your debugging
I'm just in the middle of debugging the DASH-IF player. (gosh, it's huge).

I came to the same conclusion: they are waiting on the seeking event to be fired.

Unfortunately, our state machine will only start seeking once the first frame has been decoded (so it knows what the start time is).

But DASH-IF will only feed data once seeking has started ; we're not going anywhere...
> Unfortunately, our state machine will only start seeking once the first frame has been decoded (so it knows what the start time is).

This doesn't sound quite right - currentTime appears to be updated to reflect the seek?
Depends on: 1185972
That's issue one. If I go past this by faking a seeking event in our code for it all to get going for Firefox, I see buffered ranges as negative. You think a locally modified code will help you?
Alternative manifest which does not cause negative timestamps (but still doesn't play): http://dash.bidi.int.bbc.co.uk/e/pseudolive/bbb/client_manifest.mpd
@David, did you try the stream by generating a 'seeking' event in dash.js? Just confirming how did you get past that
Pretty much - in the console I did:
document.querySelector(".dash-video-player video").dispatchEvent(new CustomEvent("seeking"))
Exactly what I have been doing. Thanks for the confirmation
After fixing the seeking event, stream still doesn't play.

The audio stream pts on that original stream are completely non-sensical, and more importantly not aligned with the video streams at all.

e.g. audio segment start at -99676888809129us
while video starts at: 2805022778145us
Thanks for the fix. I will check about the original stream. As it plays well on other browsers I assumed there's nothing wrong with the stream.
Do you have a patch with the fix which we can give a try?
I'll post the patch once I've written some regressions tests for it.
Hopefully finish tonight otherwise first thing tomorrow.

In regards to the video not starting, Chrome does a lot of stuff to massage the timestamps ; this can easily lead to A/V sync issues I've found (like if you have video timestamps starting at 0, and audio ts starting at 1, it plays the audio immediately and report the buffered range as starting from 0, leading to a 1s delay in A/V sync)

Here as the audio time is starting well before 0, it's not within the sourcebuffer.appendWindowStart and sourcebuffer.appendWindowEnd (http://w3c.github.io/media-source/index.html#widl-SourceBuffer-appendWindowEnd)

As per spec: Coded Frame Processing (http://w3c.github.io/media-source/index.html#sourcebuffer-coded-frame-processing)
step 8:
"If presentation timestamp is less than appendWindowStart, then set the need random access point flag to true, drop the coded frame, and jump to the top of the loop to start "

The frames are therefore dropped, and the resulting audio buffered range is empty.

As we have no audio to play, and the mediasource has been setup with audio: playback will stall. This is all 100% per spec.

If you enable our new MSE stack (hopefully the default in tomorrow's nightly), our implementation is now almost exactly per spec, the only discrepancies are the lack of support for streams append, and the tracks information.

You can test the new MSE in the current nightly, by setting the preference media.mediasource.format-reader to true.
Thanks for the detailed response. I will check the stream and the spec and get back to you if I have any doubts.
Let me know when the patch is ready. 
Thanks Again!
With http://dash.bidi.int.bbc.co.uk/e/pseudolive/bbb/client_manifest.mpd and patches from bug 1185972; seeking starts but never completes, so playback stall..

got it working once.. investigating...
Blocks: MSE, mse-everywhere
Status: UNCONFIRMED → NEW
Component: Untriaged → Video/Audio
Ever confirmed: true
Product: Firefox → Core
Summary: Dash Live streams don't work in firefox with Media Source extensions → Dash Live streams don't work in Firefox with MSE
Getting some good results on our test platform with bitdash player.
http://198.50.191.222/DASH/bitdash/ (not permanent/no audio!) 

IE, Chrome and FireFox playback for more than 2 mins so far.
2015-07-22 Nightly build.

No playback with Dash.js
http://198.50.191.222/DASH/dashjs/
IE is fine with that one, Chrome is temperamental.
@JackTrail
I don't know whether you are using the latest development version of dash.js but were a couple of issues with SegmentTimeline in v1.4 so worth pulling the latest and rebuilding.
(In reply to David Evans from comment #17)
> @JackTrail
> I don't know whether you are using the latest development version of dash.js
> but were a couple of issues with SegmentTimeline in v1.4 so worth pulling
> the latest and rebuilding.

I have pulled the latest and compiled but same result of no video playback, just buffers.
kbagga has also alerted the dash.js devs of this problem with Firefox:

https://github.com/Dash-Industry-Forum/dash.js/issues/649
status-firefox42: --- → affected
Assignee: nobody → jyavenard
I've opened an issue for dash.js:
https://github.com/Dash-Industry-Forum/dash.js/issues/665

Summary: seeking into the stream prior any data being loaded as likely not going to work.

Now, that doesn't explain why http://dash.bidi.int.bbc.co.uk/e/pseudolive/bbb/client_manifest.mpd doesn't start (it does specify a length in the init segment, so initial seeking will work)
If no samples were ever passed to the H264 wrapper, no decoder would have been created.

Doh !
Attachment #8639008 - Flags: review?(cpearce)
First patch needs to be uplifted. Wonder if that's what causes test_eme_playback.html to stall.

BBC streams now play just fine (though lots of stalls as bandwidth isn't that great here, and dash.js appears to be overly optimistic: sometimes it takes a fair while before it starts rebuffering)
Attachment #8639008 - Flags: review?(cpearce) → review+
Comment on attachment 8639009 [details] [diff] [review]
P2. Don't unnecessarily drain decoders when there's no frames pending.

Review of attachment 8639009 [details] [diff] [review]:
-----------------------------------------------------------------

Should we do this for Flush too?
Attachment #8639009 - Flags: review?(cpearce) → review+
https://hg.mozilla.org/mozilla-central/rev/91de55bfe09a
https://hg.mozilla.org/mozilla-central/rev/6681786341a9
Status: NEW → RESOLVED
Closed: 9 years ago
status-firefox42: affected → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla42
Is this available in the nightly builds? Also is any flag required to test this?
It should be in Nightly.

No need for special flags but the default
@ Jean-Yves, this Dash live stream still doesn't work for me in Nightly 42 build 2014-07-31. I see the following three issues:

1. After loading the mpd from comment 0, the video doesn't autoplay. This is a known issue.

2. If I press the play button, the player just stalls. The player is busy doing something, though, because it logs console messages about streaming audio and video:

GET XHR http://54.201.151.65/livesim/testpic_2s/V300/719182535.m4s [HTTP/1.1 200 OK 144ms]
loaded video:Media Segment:1438365070 (200, 0ms, 166ms)  Debug.js:115:17
GET XHR http://54.201.151.65/livesim/testpic_2s/A48/719182535.m4s [HTTP/1.1 200 OK 73ms]
[video] Getting the request for time: 1438365072  Debug.js:115:17
[video] Index for time 1438365072 is 719182535  Debug.js:115:17
[video] SegmentTemplate: 1438365070 / 4591964833.795  Debug.js:115:17
[video] Buffered Range: 3618058.778145 - 3618310.778145  Debug.js:115:17
loaded audio:Media Segment:1438365070 (200, 0ms, 441ms)  Debug.js:115:17
[audio] Getting the request for time: 1438365072  Debug.js:115:17
[audio] Index for time 1438365072 is 719182535  Debug.js:115:17
...

3. If I press the pause button, a "<video> pause" message is logged to the console, but the streaming log messages don't stop in Firefox, but they do in Chrome.
Status: RESOLVED → REOPENED
Flags: needinfo?(jyavenard)
Resolution: FIXED → ---
The video is not per spec.
As per comment 13; the audio timestamps ever huge and negative ; causing audio to be dropped.
That FF stalls is completely per spec.

Note that the BBC stream works well on mac or linux/ffmpeg ; for windows you could face bug 1189776. An int64 overflow causes decoding to be aborted.
Status: REOPENED → RESOLVED
Closed: 9 years ago → 9 years ago
Flags: needinfo?(jyavenard)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: