627153 - Chrome and Opera get to a ready state of HAVE_ENOUGH_DATA sooner than Firefox

Status: VERIFIED FIXED

(Core :: Audio/Video, enhancement)

Description

[Rick Waldron [:rwaldron]]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.634.0 Safari/534.16
Build Identifier: Gecko/20100101 Firefox/4.0b9

http://scotland.proximity.on.ca/dxr/tmp/readystate/

When loading a video WebKit and Opera reach readyState sooner then Firefox. According to tests conducted by David Humphrey, we surmise that Chrome is buffering differently and therefore about to rach "HAVE_ENOUGH_DATA" sooner.

Reproducible: Always

Comment 1

[Matthew Gregan [:kinetik]]

I doubt this has anything to do with buffering.  We should be buffering as fast as possible, so unless something really weird is going on there should be no difference in the speed of that compared to WebKit.

I think the problem is that we're too conservative about reaching HAVE_ENOUGH_DATA.  I was looking at this last night after noticing how much longer it takes to start playback of WebM videos on YouTube in Firefox compared to Chrome.  I've got a bit of analysis written up which I'll post shortly.

Comment 2

[Rick Waldron [:rwaldron]]

I mentioned this on my other ticket, but will mention again, thanks for your time and help looking at this ticket, it's greatly appreciated.

Comment 3

[Matthew Gregan [:kinetik]]

Pasting from the bug I was going to open about this:

Bug URL: http://www.youtube.com/watch?v=mYODGHCOXAY&html5=1 (but any video should be fine to test with)

Opening the above URL in Chrome and Firefox at the same time, with empty caches, Chrome has been playing the video for ~13 seconds before Firefox begins playback.  I've noticed this behaviour with the YouTube HTML5 beta a number of times--Flash videos and HTML5 videos in Chrome both appear to start up much faster than in Firefox.

Aiming for a more easily reproducible testcase: running an HTTP server locally serving a 207MB WebM video that is 31 minutes and 32 seconds long, giving it an average bitrate of ~113kB/s.  Using lighttpd with connection.kbytes-per-second set to 120 and testing in Chrome and Firefox, Chrome begins playback within 100ms of the document loading.  Firefox doesn't begin playback until over 16 seconds have elapsed.  When playback begins, Firefox has ~45 seconds of video buffered (according to @buffered).

Examining a log trace:
48:59.939249 - load start
49:00.021446 - 311017 bytes received (one chunk)
49:00.040699 - have_current_data
49:00.098155 - have_future_data
49:01.037586 - 671465 bytes received (in two more chunks)
49:04.264018 - 2093056 bytes received (in two more chunks)
49:16.037540 - 2125824 bytes received (in one more chunk)
49:16.042214 - 2682880 bytes received (in one more chunk)
49:16.045513 - have_enough_data

We're waiting on the download rate computation to become reliable (essentially a 3 second timeout), and then for the download to exceed the readahead margin (CAN_PLAY_THROUGH_MARGIN, which is playback rate * 20 seconds).  In this case, the readahead margin is 2305098, so we're very close to it by 4.5s of loading the video (short just 212042 bytes, which at 120kB/s should arrive within 2 seconds).

The large chunks in this download are causing worst case behaviour here, and I think that's being triggered by lighttpd's bandwidth throttling.

Other possible issues:
- it takes a long time for the download rate to stabilize because the data comes in large chunks with large delays in between
- the playback rate is computed once we have a duration (as size / duration), and is then fixed for the rest of time.  This gives us the average rate for the entire media, but the instantaneous rate is unlikely to be this consistent.
- it looks like the large CAN_PLAY_THROUGH_MARGIN is intended to protect the buffering stability from lumpy bitrates during media startup

So, in this case, the majority of the problem is caused by the readahead margin.  I'll try tracing a YouTube video and see what that looks like now.

Comment 4

[Chris Pearce [:cpearce (Not reading bugmail)]]

When we play preload=metadata videos, we move into buffering state, and won't start playback until we exit that state. We may be able to safely be less conservative and start playback earlier in that case too.

Reducing CAN_PLAY_THROUGH_MARGIN would achieve this, we need to remember to verify it.

Comment 5

[Chris Pearce [:cpearce (Not reading bugmail)]]

Attachment: Patch: reduce CAN_PLAY_THROUGH_MARGIN

* Reduce CAN_PLAY_THROUGH_MARGIN by 50% to 10s, and remove the "download size safety margin" from the canplaythrough calculation. This still gives us some margin for error in the calculation, but we'll be less conservative. I've found smaller values had enough error to misfire.

Comment 6

[Matthew Gregan [:kinetik]]

So, stepping back a bit...

Chrome always reaches HAVE_ENOUGH_DATA immediately after the load begins.  This is set in WebMediaPlayerImpl::OnPipelineInitialize, which is part of the media load process.  So Chrome's canplaythrough event will often be... rather optimistic.  There's a TODO in the code which reads "change this to report the correct status".  We're not going to change our behaviour to what Chrome does, for what I hope are obvious reasons.

Safari reaches HAVE_ENOUGH_DATA based on the underlying QTMovie reaching QTMovieLoadStatePlaythroughOK.  I couldn't find any documentation that described in detail what that meant, but the basic description makes it sound equivalent to HTML5's HAVE_ENOUGH_DATA.

From a quick experiment using lighttpd's not-very-accurate rate limiting, it seems like Firefox's current behaviour is more aggressive at reaching HAVE_ENOUGH_DATA (and HAVE_FUTURE_DATA) than Safari is.

Comment 7

[Matthew Gregan [:kinetik]]

Just did the same test with Opera as well.  It seems to fire canplaythrough at the same time it fires canplay, and it's... extremely optimistic about canplaythrough.  The behaviour is very similar to Chrome.

Comment 8

[David Humphrey (:humph)]

Talking with Brett today re: Web Made Movies projects, and this is hurting us pretty bad.  I'm assuming there's no way to get this into 4, but wanted to confirm.

Comment 9

[Matthew Gregan [:kinetik]]

Can you explain how it's hurting exactly?  Chrome and Opera are doing the wrong thing, we're not going to copy their behaviour.  We can tweak the current behaviour to reach HAVE_ENOUGH_DATA somewhat more aggressively, but if you're expecting Chrome/Opera behaviour it's unlikely to solve your problem.

Comment 10

[David Humphrey (:humph)]

Yeah, I hear what you're saying from a spec/technical point of view.  The trouble is, users don't really care about either, and the experience of using these other browsers is that they start videos faster.  If we can't copy them, but can make ours more aggressive, maybe this is all I can ask.  It's just too bad that doing it wrong means you're fast, and doing it right, slow.

Comment 11

[David Humphrey (:humph)]

Matthew and I spoke on irc and debugged the butter code.  It looks like Chrome doesn't implement the preload attribute, or does so badly.  We follow the spec, and since the video element hasn't got a preload attribute set, we default to metadata.  This gets us to a ready state of 2, and no further.  The butter code is stuck in a loop via interval, waiting on the ready state to hit 3 or 4, which it never does by design.  Chrome, on the other hand, goes from a ready state of 0 to 4 and stops, bypassing 1 through 3 altogether.

The fix here is to set preload="auto" on the video element, forcing it to actually download beyond metadata.  Doing so causes us to go from 0 to 2 to 4 and halt.

Rick, can you fix your butter code to do this?  Chrome will ignore it until they implement, not causing you any grief, and continue to what it does now.  Mozilla will then get to 4 in the same amount of time as Chrome.

Comment 12

[David Humphrey (:humph)]

By the way, comment 11 refers to comment 8, which isn't really related to this bug.

Comment 13

[Chris Pearce [:cpearce (Not reading bugmail)]]

(In reply to comment #11)
> Matthew and I spoke on irc and debugged the butter code.  It looks like Chrome
> doesn't implement the preload attribute, or does so badly.

Chrome does not support preload at all. Try pointing Chrome at http://pearce.org.nz/video/preload.html and you'll see.

> The fix here is to set preload="auto" on the video element, forcing it to
> actually download beyond metadata.  Doing so causes us to go from 0 to 2 to 4
> and halt.
> 
> Rick, can you fix your butter code to do this?

You'll want to leave the autobuffer attribute on your media elements, so that Firefox 3.x (and Chrome & Opera & other clients which don't support preload) will load the entire media.

I still think we can safely reduce the CAN_PLAY_THROUGH_MARGIN, were way more conservative than we need to be.

Comment 14

[Matthew Gregan [:kinetik]]

Comment on attachment 509331 [details] [diff] [review]
Patch: reduce CAN_PLAY_THROUGH_MARGIN

Let's just do this for now.  There's more work to be done here, but it'll be more efficient to base it on top of the patches in bug 628665.

Comment 15

[Chris Pearce [:cpearce (Not reading bugmail)]]

Comment on attachment 509331 [details] [diff] [review]
Patch: reduce CAN_PLAY_THROUGH_MARGIN

Can I have approval to land this? This just lowers a threshold for firing the "canplaythrough" event in <video>, meaning videos will start playing sooner. Minimal regression risk, it just lowers a threshold.

Comment 16

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Matthew: can you please file a bug against Chrome? I can try to make sure it gets attention. In the bug, you'll want to point out that leaving this unfixed means we get into a race to the bottom which ultimately will make canplaythrough useless.

Comment 17

[Chris Pearce [:cpearce (Not reading bugmail)]]

http://hg.mozilla.org/mozilla-central/rev/be19d8833032

Comment 18

[Matthew Gregan [:kinetik]]

(In reply to comment #16)
> Matthew: can you please file a bug against Chrome?

http://code.google.com/p/chromium/issues/detail?id=73609

Comment 19

[Matthew Gregan [:kinetik]]

And Opera bug: DSK-329100

Comment 20

[Neale Upstone]

Big thanks everyone.  I ended up reading this after a scan through what landed in beta 12.  It's the sort of rigour shown around this bug which gives Firefox a bright future.