542405 - prefetched content with cache-control/gzip causes browser encoding error

Status: UNCONFIRMED

(Core :: Networking: Cache, defect, P3)

Description

[Graeme Merrall]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 (.NET CLR 3.5.30729)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 (.NET CLR 3.5.30729)

Filing in Networking:Cache since it appears to be a cache management issue

There's a few of defects already open on this issue, however I'm opening a separate one as I've isolated a cause and server based fix, however I believe this should be fixed in Mozilla

See #453988 #460818 #469352 #488438 #506591 etc

The issue manifests most easily when visiting Google

When a browser visits Google, the search engine includes the prefetch link tag for the first search result only. e.g. <link rel=prefetch href="http://www.example.com/"> The  idea being that your browser will download the page in the background and then when you click the link it’s already in your browser cache.

Here's the headers from a prefetched request. Note the Content-Encoding and Cache-Control headers in the server response

GET / HTTP/1.1
Host: www.example.com.au
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 (.NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://www.google.com.au/search?q=deleted&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-GB:official&client=firefox-a
X-Moz: prefetch

HTTP/1.1 200 OK
Server: Sun-ONE-Web-Server/6.1
Content-Type: text/html
Akamai: server1-prod
Last-Modified: Fri, 04 Dec 2009 03:36:00 GMT
ETag: "90e-4b1883a0"
Accept-Ranges: bytes
Content-Encoding: gzip
Content-Length: 1127
Expires: Tue, 26 Jan 2010 23:13:48 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Tue, 26 Jan 2010 23:13:48 GMT
Connection: keep-alive
Vary: Accept-Encoding


Now I click on the link in the search results and Firefox issues a Range request for the page

Host: www.example.com.au
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 (.NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Referer: http://www.google.com.au/search?q=deleted&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-GB:official&client=firefox-a
Cookie: server-id=2
Range: bytes=0-
If-Range: "90e-4b1883a0"

With the response from Akamai of
HTTP/1.1 206 Partial Content
Server: Sun-ONE-Web-Server/6.1
Content-Length: 2318
Content-Type: text/html
Akamai: server2-prod
Last-Modified: Fri, 04 Dec 2009 03:36:00 GMT
ETag: "90e-4b1883a0"
Content-Range: bytes 0-2317/2318
Expires: Mon, 25 Jan 2010 01:37:24 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Mon, 25 Jan 2010 01:37:24 GMT
Connection: keep-alive
Vary: Accept-Encoding


This the results in Firefox displaying an error about incompatible content encoding and that the user should contact the website owner.  Clearing the cache or doing a forced reset solves the problem.   Note here that the content is *not* compressed which I think is the key

I've been having a machine temp issues and the above requests are not sequential if you check the Date but the issue is the same regardless

To prevent this problem occuring I did a lot of testing and discovered that you can prevent this problem occuring by either removing the cache-control header for the initial request *or* turning off gzip for the initial request


Here's what I think is happening
- Firefox makes an initial request for the prefetched content and instructed by the cache control header to delete the content which is does
- Firefox then has a second request for the content and the error occurs

Based on reading https://developer.mozilla.org/En/Link_prefetching_FAQ my guess is that Firefox sees that it had the request but doesn't any more (because of the cache control header) and interprets that as an interrupted prefetch request which it must complete, hence the header "Range: bytes=0-".  However the second request is not compressed but Firefox is attempting to complete the previous request which was compressed. That therefore creates a content encoding mismatch which results in the error

Sending any cache control header which results in removal will result in the issue occuring e.g. Cache-Control: max-age=1

When fixing this issue at the server, it can be done in one of two ways
- Do not compress the content.  The prefetched content will be downloaded and deleted as per usual and the second range request will occur. However because both requests are now uncompressed, the range request completes as expected
- Do not issue a cache-control header for the prefetched request. Firefox will not delete the content from it's cache so a second request is not required so the issue doesn't occur. 

Ideally I think the fix here is for Firefox not to delete the file from cache when it's a pre-fetched request with a no-store Cache-Control header

Reproducible: Always

Steps to Reproduce:
1. Create a piece of content that is delivered via gzip and with a cache-control header of no-store, no-cache etc. Check that a range request results in non-compressed content
2. Create a link to the content in a second page containing <link rel=prefetch>
3. Clear cache
4. Load the page containing the link and verify the prefetch fires
5. Click the link to the page 
Actual Results:  
Content Encoding error

Expected Results:  
Page should load

Comment 1

[WADA:World Anti-bad-Duping Agency]

Because If-Range request, server should return one of next:
  - Remaining data requested by Range: (remainig gzipped data in this case)
  - Whole data with 200
    (plain data, or gzipped data with appropriate Content-Encoding)
As for this, it's server side problem.

However, Fx requested with "Range: bytes=0-". I think Fx's usage of "Range: + If-Range:" for "bytes=0-" is bad, even if it's never RFC violation.
I think, when content-length is known, "Range: + If-Range:" is better to be used only when whole data is large and partially downloaded data is relatively not-so-small, especially when Content-Encoding:gzip was returned for previous HTTP GET. I believe client should't request "Range: + If-Range:", at least when partially downloaded data is ZERO byte.

Comment 2

[Firefox Bug Husbandry Bot]

Bulk change to priority: https://bugzilla.mozilla.org/show_bug.cgi?id=1399258

Comment 3

[Firefox Bug Husbandry Bot]

Bulk change to priority: https://bugzilla.mozilla.org/show_bug.cgi?id=1399258