626027 - HTTP byte range requests are not cached

Status: RESOLVED DUPLICATE of bug 286082

(Core :: Networking: Cache, defect)

Description

[Stefan]

User-Agent:       
Build Identifier: Mozilla/5.0 (X11; Linux x86_64; rv:2.0b10pre) Gecko/20110115 Firefox/4.0b10pre

Sometimes media files need three HTTP requests to load. In these cases firefox requests byte ranges instead of the whole file. The server replies are not properly cached but are fetched again upon reload.

Reproducible: Always

Steps to Reproduce:
1. use fresh profile; close all but one tab;
2. open tab 1, enable web console (shift-ctrl-k)
3. load http://www.vogtner.de/content-disposition/vuvuzela.ogg
4. open tab 2, about:cache

Actual Results:  
3. ogg file is transfered in three requests (206 partial content)
4. about:cache states that vuvuzela.ogg is associated with "4320 bytes". The
   links gives:
   "Cache entry information
    The cache entry you selected is not available."

Expected Results:  
4, "107583 bytes" and proper display of the cache item as one gets only after a firefox restart and reload of vuvuzela.ogg.

1-reload in first tab does not finish
2-after closing the first tab, about:cache lists "99360 bytes"

Comment 1

[Matthias Versen [:Matti]]

the range request is bug 502894

moving to core:Video/audio but it could be also Networking:cache

Comment 2

[cajbir (:cajbir)]

You should find the media is cached if you send a Content-Duration header with the video giving the duration. This stops the byte range requests to find the duration from happening since it trusts the Content-Duration header for that information instead. Current network cache code does not cache byte range requests IIRC.

You'll also need to ensure you send an Accept-Ranges for seeking support. If we don't get this we send a byte range request starting from 0- and use the result of that to determine seeking support. Again, that byte range request will prevent caching from occurring.

Comment 3

[Stefan]

Thanks. Setting Content-Duration (even to 0) works as expected. The multiple requests remind me of the corresponding problem with MPEG-4/H.264 files/flash ("MOOV atom").

Remains the missing cacheing of byte range requests.

Comment 4

[u279076]

This sounds more like a Networking:Cache issue.  Moving it there.

Comment 5

[Boris Zbarsky [:bzbarsky]]

Well...  It depends on what the consumer is doing.  What exact requests does the media code make here, how does it handle the responses, an what did you expect to be cached and when?  We certainly cache byte-range requests in general.

Comment 6

[cajbir (:cajbir)]

It looks like you are right that the data is cached. Although there is some oddness I don't understand. Take this video for example:

http://media.tinyvid.tv/1kn0rtfbcf5m.ogg

This is served with a HTTP header that tells our media engine the duration of the video (x-amz-meta-content-duration: 37.637). Because of this we send a single HTTP request without using byte ranges for the full resource. The server responds with HTTP 200 and the video is stored in the cache. about:cache reports a 'data size' of 2017889 which matches the media's content-length. Subsequent requests for the video appear to come from the cache (as evidenced by the web console showing not HTTP request being made).

The following is a URL to the same video but on a server that doesn't send the content duration header. This means we need to do byte range requests to get the duration:

http://www.bluishcoder.co.nz/bug626027.ogg

Three HTTP requests are made. The first is the initial non-byte range request where we read the video metadata and discover we need to make a request for the duration. This second request is a byte range request with the range "bytes=1998848-" where we read the last frame of data to get the duration. The third is the byte range range from the position of the first frame of video onwards with range "bytes=8688-"

about:cache shows the file cached with a size of 8688, but clicking on the cache entry shows 'the cache entry you selected is not available'.

Comment 7

[Boris Zbarsky [:bzbarsky]]

> The first is the initial non-byte range request where we read the video
> metadata and discover we need to make a request for the duration.

But you leave the original request running while making the request for the duration?  Or do you cancel it?

For GET requests I'm pretty sure the cache key is just the url.  So of those three requests you made at most one will be cached when all is said and done...  Which one might depend on the exact handling of the responses.

Comment 8

[cajbir (:cajbir)]

We cancel the original request with:

  mChannel->Cancel(NS_ERROR_PARSED_DATA_CACHED);

in msMediaStream.cpp, line 566. We create a new channel soon after with the load flag nsICachingChannel::LOAD_BYPASS_LOCAL_CACHE_IF_BUSY in nsMediaStream.cpp line 684. Could the cache still be busy at this point and it's bypassing caching the subsequent channels doing the seeking?

Comment 9

[Boris Zbarsky [:bzbarsky]]

> We cancel the original request

OK, so we'll doom that cache entry, I think....  And yeah, the other channels might race against the cache entry becoming available too, depending on what releases what when.  :(

We really need a cache that doesn't need that "bypass if busy" crap.  :(

Comment 10

[Honza Bambas (:mayhemer)]

The third request processes 206 and stores data to the cache as coming from the net, behaves as expected.  

But, it gets suspended before the channel load finishes, so the cache entry is left open and about:cache is not updated.  The channel hangs indefinitely, so the video cannot be open any more...

Time to allow reading of a cache entry while we still write into it.  This really seems to be a priority after Fx4, IMO.


This bug should be renamed and moved to video/audio component.

Comment 11

[Stefan]

> But, it gets suspended before the channel load finishes, so the cache entry is
> left open and about:cache is not updated.  The channel hangs indefinitely, so
> the video cannot be open any more...

Sure: Bug 594327

Comment 12

[Brendan Dahl [:bdahl]]

Starting in FF24+ range requests are used to partially load the PDF so it can be displayed faster. This bug is quite noticeable when reloading a pdf as it re-fetches all the data.

Comment 13

[Stefan]

(In reply to Stefan from comment #0)
> 3. load http://www.vogtner.de/content-disposition/vuvuzela.ogg

moved to
http://www.vogtner.de/mozilla/626027/vuvuzela.ogg

Comment 14

[Matthias Versen [:Matti]]

marking new (confirmed by Brendan)

Comment 15

[zvezda]

Seeing this behavior when trying to cache byte range requests with an HTML5 video tag loading webm. Unless you load the entire video, requesting partial content again results in downloading the it again.

Would be great to have this working so we can try more multimedia-rich experiences in the browser with some nice programmatic video preloading.