Closed Bug 1100000 Opened 9 years ago Closed 8 years ago

[WebM] youtube video freeze and/or audio disappearance html5

Categories

(Core :: Audio/Video: Playback, defect, P5)

Product:
Core
Component:
Audio/Video: Playback
Version:
36 Branch
Platform:
x86_64
Windows 7
Type:
defect

Tracking

()

Status:
RESOLVED WORKSFORME

People

(Reporter: raysatiro, Assigned: ajones)

References

(Blocks 1 open bug)

Details

Attachments

(1 file, 1 obsolete file)

Avoid calling DecodeAudioData multiple times
2.74 KB, patch
bholley
: review+
cpearce
: feedback-
Details | Diff | Splinter Review 
User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:36.0) Gecko/20100101 Firefox/36.0
Build ID: 20141114030206

Steps to reproduce:

For example: http://www.youtube.com/watch?v=Tv3RgbA2UTo

Video will play for a few seconds and then stop or continues and then audio disappearance later.

Tested each in a separate clean profile:
Working in FF24 ESR and FF31 ESR, which both play the video using flash.
Broken in Nightly which is playing it using HTML5 video.
Might be broken in Aurora? I can't reproduce in a clean profile with flash disabled but in my regular profile the video doesn't show although the audio plays. 


Nightly specific:

Setting layers.offmainthreadcomposition.enabled to false doesn't seem to make a difference. I did that to disable OMTC, but if that doesn't do it let me know what I need to do to disable OMTC. I can reproduce with OMTC on or off, and with e10s on or off.



Actual results:

I can't view videos on youtube without interruption in latest Nightly.


Expected results:

I should be able to view videos on youtube without interruption in latest Nightly.
Setting media.mediasource.enabled to false seems to work.
Component: Untriaged → Video/Audio
Product: Firefox → Core
Congrats on bug number one million, one hundred thousand!
Confirmed in 36.0a1 (2014-11-17) Win 7 x64 both with e10s ON/OFF
Status: UNCONFIRMED → NEW
Ever confirmed: true
Do you get the same issue with WebM?

media.mediasource.webm.enabled=true
media.mediasource.mp4.enabled=false
(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #4)
> Do you get the same issue with WebM?

It works for the most part, however I tested about a dozen times and twice nightly froze but I don't think that's related to this since it's been randomly freezing a lot on me lately.

I did find another issue reproducible which is an audio pop 50 seconds when WebM. Every time it pops at that exact same point. I can see in the video player the gray thing showing how much has downloaded is stopped at about 50 sec, then once the active widget catches up to it there's the pop and then the gray extends all the way (as if all the video has been downloaded) and no other problems.

gecko.buildID = 20141118030201
gecko.mstone = 36.0a1
media.mediasource.enabled = true
media.mediasource.mp4.enabled = false
media.mediasource.webm.enabled = true
I'm making slow progress on this.

We hang when we run out of data, and the mp4 decoder is waiting for data to be appended.

It appears that because MP4Reader is asychronous, calls to MediaDecoderReader::RequestAudioData() result in two audio frames being decoded. We request the first via DecodeAudioData(), nothing happens immediately, so we post a task to call RequestAudioData again. That task almost always gets processed before the asynchronous decode finishes, so AudioQueue().GetSize() == 0 and we call DecodeAudioData() a second time.

We hang we the first of those two decodes the very last frame within our buffered range, and the second one gets stuck in the demuxer trying to demux the 'next' one. This means we never actually deliver the last frame we have, so our current position never reaches the end of the buffered range.

This is hard to reproduce, the majority of the time youtube appends audio much faster than we're playing and this isn't an issue. On rare occasion it doesn't, and we neither reach the end of the buffered range, nor update the waiting state.
(In reply to Matt Woodrow (:mattwoodrow) from comment #6)
> It appears that because MP4Reader is asychronous, calls to
> MediaDecoderReader::RequestAudioData() result in two audio frames being
> decoded. We request the first via DecodeAudioData(), nothing happens
> immediately, so we post a task to call RequestAudioData again. That task
> almost always gets processed before the asynchronous decode finishes, so
> AudioQueue().GetSize() == 0 and we call DecodeAudioData() a second time.

Not an answer to this problem, unfortunately, but bug 1100701 is another issue caused by the RequestAudioData task.
It seems like the simple fix for this is to make sure that we only call DecodeAudioData on the first call to RequestAudioData, and skip it on following calls while we're waiting for data to be added to the queue. 

Is there ever a case where we need to poke the decoder multiple times in order to get a single frame?

There seems to be a related issue (unsure if we need it to fix this bug) where we won't return NEED_MORE_DATA when we try to decode at the end of the buffer. The demuxer only fails with END_OF_STREAM if the SourceBufferResource has been Ended, which won't be true for the most recently decoder. The demuxer instead just blocks indefinitely waiting for data, and we won't ever update the state on the media element.

Maybe we should instead compare the buffered ranges to the end time of the most recently decoded frame, and bail out before entering the demuxer if we know there won't be any more frames available.

We can do this check in MediaSourceReader::RequestAudioData *if* my first suggestion is done, because we know that mLastAudioTime will be up to do. In the current state we can get a second RequestAudioData call before the first one comes back to OnAudioDecoded/OnNotDecoded, so we'd have to do the comparison within MP4Reader (which demuxes synchronously and will know what the previous frame time was).
I *think* this fixes the issue, but it's hard to be sure since it's very intermittent for me.
Assignee: nobody → matt.woodrow
Attachment #8525725 - Flags: review?(bobbyholley)
Comment on attachment 8525725 [details] [diff] [review]
Avoid calling DecodeAudioData multiple times

Review of attachment 8525725 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good for now I guess, but let's run it by cpearce.
Attachment #8525725 - Flags: review?(cpearce)
Attachment #8525725 - Flags: review?(bobbyholley)
Attachment #8525725 - Flags: review+
Comment on attachment 8525725 [details] [diff] [review]
Avoid calling DecodeAudioData multiple times

Review of attachment 8525725 [details] [diff] [review]:
-----------------------------------------------------------------

Let's see if bug 1032633 fixes this, and if not we can re-evaluate this.
Attachment #8525725 - Flags: feedback-
See Also: → 1103357
Any updates here? I've still been seeing terrible playback in most YouTube videos. Any useful data I or others can help gather?
Are you using nightly?

What platform are you on?

We've fixed a lot of bugs that caused freezing, and I haven't had any issues recently.
I've been using webm and I still get an audio pop about 50 seconds in (refer to comment #5). I switched back to mp4 just now and it seems to be working well.

(In reply to Justin Dolske [:Dolske] from comment #12)
> Any updates here? I've still been seeing terrible playback in most YouTube
> videos. Any useful data I or others can help gather?

Are you having problems with the html5 player (youtube.com/html5) or the flash player? You can right-click on a playing video to see which one it's using. I've had a lot of problems with the flash player / plugin-container since I started using e10s.
(In reply to Ray Satiro from comment #14)
> I've been using webm and I still get an audio pop about 50 seconds in (refer
> to comment #5). I switched back to mp4 just now and it seems to be working
> well.

We are focusing on MP4 at the moment which is supported on Windows Vista+ and Mac. Some demuxer changes landed yesterday in 1098126 which should fix a number of issues. There is the potential for regression there as well.

The 50 second blip is a timescale discontinuity. Our current behaviour is to play silence when this happens which doesn't work well in MSE/VP9.
Assignee: matt.woodrow → ajones
Status: NEW → ASSIGNED
Comment on attachment 8537631 [details] [diff] [review]
Remove silence during audio discontinuity

Review of attachment 8537631 [details] [diff] [review]:
-----------------------------------------------------------------

This is roc's code to feed data into the MSG, you'll need to talk to him about whether he wants to keep this behaviour.

The code for media playback that feeds silence is the PlaySilence() calls in AudioSink.cpp.
Attachment #8537631 - Flags: review?(cpearce)
Bit of a noobie question, but wouldn't it make more sense to play the last observed value until you're caught up to avoid causing a sudden jump in pressure? Or if you already know the next value, linearly interpolate between them.
Attachment #8537631 - Attachment is obsolete: true
The audio pop appears to be a WebM issue.
Priority: -- → P5
(In reply to Matt Woodrow (:mattwoodrow) from comment #13)
> Are you using nightly?
> 
> What platform are you on?

Current Nightlies on OS X 10.10.

(In reply to Ray Satiro from comment #14)

> Are you having problems with the html5 player (youtube.com/html5) or the
> flash player? You can right-click on a playing video to see which one it's
> using. I've had a lot of problems with the flash player / plugin-container
> since I started using e10s.

I'm using the HTML5, as I don't even have Flash installed on this system. Oh, and I have E10S disabled as well.
In my experience webm videos are problematic when attempting to view youtube videos in full screen. They stutter in and out are impossible to watchat least with with HA off. Setting layers.offmainthreadcomposition.enabled to false helps considerably but I wonder how long that ability will remain.
FWIW, this has been working much much better for me (on OS X) over the last week or so. (Perhaps due to the fixes mentioned in comment 15?)
Are you still experiencing the issue with MSE turned on?

the various stalls should be all fixed now
Flags: needinfo?(raysatiro)
This seems fixed for me.
(In reply to Jean-Yves Avenard [:jya] from comment #23)
> Are you still experiencing the issue with MSE turned on?
> 
> the various stalls should be all fixed now

I've been using mostly webm since. Currently:
Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:38.0) Gecko/20100101 Firefox/38.0
gecko.buildID = 20150212030231
gecko.mstone = 38.0a1

I tried a few times and I can no longer reproduce when:
media.mediasource.enabled = true
media.mediasource.mp4.enabled = true
media.mediasource.webm.enabled = false
media.mediasource.youtubeonly = false

I can still reproduce the webm audio pop I reported in comment 5 when:
media.mediasource.enabled = true
media.mediasource.mp4.enabled = false
media.mediasource.webm.enabled = true
media.mediasource.youtubeonly = false
Flags: needinfo?(raysatiro)
Actually I just had the audio issue again on a WebM video, the video kept playing but no audio.

Mozilla/5.0 (X11; Linux x86_64; rv:37.0) Gecko/20100101 Firefox/37.0
hi,
so what's new about this bug?
We are on april, 30th now :-)
(In reply to Julien from comment #27)
> hi,
> so what's new about this bug?
> We are on april, 30th now :-)

I still get an audio pop in WebM about 50 seconds in. Not 100% reproducible, but happened 4 out of 5 times, each time using a just created new profile set to webm mediasource.

gecko.buildID = 20150430030201
gecko.mstone = 40.0a1
media.mediasource.enabled = true
media.mediasource.mp4.enabled = false
media.mediasource.webm.enabled = true
Ray, any particular reason you've disabled mp4 and only enabled webm?

At this stage, only MSE/mp4 is enabled by default and for good reasons.
(In reply to Jean-Yves Avenard [:jya] from comment #29)
> Ray, any particular reason you've disabled mp4 and only enabled webm?

I left webm on after I had those problems with mp4. I'll change it back and see how that goes for a while.
Summary: youtube video freeze and/or audio disappearance html5 → [WebM] youtube video freeze and/or audio disappearance html5
Component: Audio/Video → Audio/Video: Playback
This is most likely fixed already. Re-open if necessary.
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.