154115 - (squidpc) does not issue byte-range request to HTTP/1.0 servers [was: Partial caching is broken through Squid proxy.]

Status: RESOLVED WONTFIX

(Core :: Networking, enhancement)

Description

[Adam D. Moss]

Mozilla now uses byte-range requests to attempt to resume
cached partial documents.  This is good.

This does not work through Squid.  Squid uses a nonstandard
way to do this.

Squid is a very common proxy.  Thus lots of users aren't
experiencing the partial-caching goodness.

It would be nice to have resumption of partial documents work
through Squid.

There's a public Squid proxy on webcache.dial.pipex.com:3128
running Squid 2.4.STABLE4 which exhibits the problem.

Comment 1

[Adam D. Moss]

Forgot to mention: Netscape 4.x supports range requests
through this same proxy where Mozilla does not.  ->  4xp

Comment 2

[tenthumbs]

I have to ask, what's non-standard about squid's responses?  My 
2.4STABLE6 returns just what my Apache server sends out and I can't 
see any error from rfc2616.

Comment 3

[Adam D. Moss]

I was going on what Darin suspected from bug 116365.
Changing summary to be vaguer.

Comment 4

[Adam D. Moss]

n.b.: Mistake, the version of Squid on webcache.dial.pipex.com
(the version which works for NS4 but not Moz1/trunk) is
2.4STABLE2, not 2.4STABLE4.

Comment 5

[tenthumbs]

If I talk directly to Apache I get

--->telnet localhost 80
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET /bad1.html HTTP/1.1
Host: localhost
Range: bytes=200-
Connection:close

HTTP/1.1 206 Partial Content
Date: Tue, 25 Jun 2002 15:10:58 GMT
Server: Apache/1.3.26 (Unix)
Last-Modified: Sun, 13 Aug 2000 14:23:32 GMT
ETag: "3b11a-f2-3996af64"
Accept-Ranges: bytes
Content-Length: 42
Content-Range: bytes 200-241/242
Connection: close
Content-Type: text/html

 src="bad1.gif">
    </body>
</html>
Connection closed by foreign host.

and talking to Squid I get

--->telnet localhost 3128
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET http://localhost/bad1.html HTTP/1.1
Host: localhost
Range: bytes=200-
Connection:close

HTTP/1.0 206 Partial Content
Date: Tue, 25 Jun 2002 15:09:47 GMT
Server: Apache/1.3.26 (Unix)
Last-Modified: Sun, 13 Aug 2000 14:23:32 GMT
ETag: "3b11a-f2-3996af64"
Accept-Ranges: bytes
Content-Length: 42
Content-Range: bytes 200-241/242
Content-Type: text/html
X-Cache: MISS from localhost
Proxy-Connection: close

 src="bad1.gif">
    </body>
</html>
Connection closed by foreign host.

Is something wrong here? Maybe 2.4STABLE2 is broken.

Comment 6

[Adam D. Moss]

Right, I see the same 'correct-looking' behaviour at
the telnet prompt from the specific 2.4.STABLE2
proxy I use (webcache.dial.pipex.com 3128).  Either
the problem is subtle or it's the Mozilla side that's
actively broken in some way.

tenthumbs: Out of interest does Mozilla successfully manage
partial caching through your local 2.4.STABLE6?  I imagine it
could be very hard to tell running locally (fast) though.

Comment 7

[Adam D. Moss]

> Maybe 2.4STABLE2 is broken.

It's certainly a possibility, though I'd like to reiterate
that NS4 resumes partial documents through the same proxy.

Comment 8

[tenthumbs]

I've actually never seen this problem but who knows. Do you have a 
repeatable way to make mozilla cache partially?

Comment 9

[Adam D. Moss]

> Do you have a repeatable way to
> make mozilla cache partially?

Sure.  It's a matter of hitting 'stop' while the
document is downloading (d'oh!).  It's easy-peasy
to see it happen on a 56.6K line and a fairly
big image.  Then hit 'reload' and it should continue
where it left off.

Comment 10

[Darin Fisher]

the only "problem" with squid is that it is a HTTP/1.0 proxy server.  we
explicitly do not issue byte range requests to HTTP/1.0 servers because they
probably don't know anything about the HTTP/1.1 If-Range header.  when i wrote
the initial version of byte-range support for mozilla, i didn't have time to
field bugs pertaining to bad servers.  i therefore limited the scope to HTTP/1.1
since it has very well defined byte-range semantics.

Comment 11

[Darin Fisher]

updating summary.

Comment 12

[Bradley Baetz (:bbaetz)]

Sure, but if it doesn't understand If-Range, then you'll get a 200 back, not a
206, won't you?

Comment 13

[tenthumbs]

Squid 2.4STABLE6 understands If-Range headers. If I feed it incorrect 
Range or If-Range headers then it returns a 200 response. The latest 
squid 2.5 snapshot identifies itself as HTTP/1.0 so this "problem" will 
be around for a while. Maybe mozilla needs a downgrade-to-1.0 pref for 
each proxy server used. Any given user is not likely to have a lot of 
proxies.

Comment 14

[Darin Fisher]

bbaetz: if it doesn't understand If-Range then there's no entity identifier
(like an ETag or Last-Modified date) for the server to use to determine if and
when a 206 might be appropriate.  IOW: byte range requests will only work on
very-static content.

Comment 15

[Bradley Baetz (:bbaetz)]

Darin: do we only use byte-range requests if thgere was an etag? My point is
that we could send the request anyway, and the server's reponse will tell us if
it understood the range header we send. What am I missing?

Comment 16

[Darin Fisher]

bbaetz: yes, we require an ETag or Last-modified value before attempting a
partial GET.  the reason is simple: we don't want to receive part of the wrong
document, which could easily happen without the use of some sort of "previous"
entity identifier.

Comment 17

[Adam D. Moss]

<whine> Netscape4 managed this okay though! </whine>
Thanks for looking into the matter anyway.  :)

Comment 18

[Darin Fisher]

the only good solution for this bug is probably to sniff for Squid.

Comment 19

[Darin Fisher]

not sure when i'll have time to work on this, futuring.

Comment 20

[Andrew Hagen]

Here's the tracking bug in Squid's Bugzilla for HTTP/1.1 compliance.

http://www.squid-cache.org/bugs/show_bug.cgi?id=411

Comment 21

[Marc Boon]

In response to Comment #10:
I recently encountered a problem with the Flash plugin. Images downloaded from
within a Flash movie didn't load properly, and after checking the logs I found
out that the broken downloads were all served with a HTTP 206 (Partial Content)
response code. This should be fine, but apparently Flash/Mozilla can't handle this.
In an attempt to solve this, I tried switching Mozilla to HTTP 1.0, and saw to
my surprise that even in HTTP 1.0 mode, Mozilla was issueing Range and If-Range
headers, which is out of spec. Range headers are not defined for HTTP 1.0.

GET /works/icons/219.jpg HTTP/1.0
Host: 127.0.0.3
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113
Accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,image/jpeg,image/gif;q=0.2,*/*;q=0.1
Accept-Language: en,en-us;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Range: bytes=0-
If-Range: "25dab040-f76-eda42a00"

Comment 22

[Darin Fisher]

-> default owner