67675 - FTP: Inconsistent download behavior with and without proxy

Status: RESOLVED WONTFIX

(Core :: Networking: HTTP, defect)

Description

[tenthumbs]

If I specify a direct connection for FTP, navigate to something like
<ftp://localhost/pub/>, and select <ftp://localhost/pub/foo.tar.gz> then
mozilla will pop up a dialog asking what to do. I select save to disk and,
eventually, mozilla puts the file in the correct location with the correct
file size. There is no copy of this file in the cache.

If I use my squid proxy then mozilla repeats the same actions but the file
is a different size and is much larger, and a copy of the larger file is in
the cache.

Via the proxy, squid synthesizes a set of HTTP headers so mozilla sees this:

--->telnet localhost 3128
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
HEAD ftp://localhost/pub/foo.tar.gz HTTP/1.1
Host: localhost
Connection: close

HTTP/1.0 200 OK
Server: Squid/2.3.STABLE4
Mime-Version: 1.0
Date: Sun, 04 Feb 2001 14:15:18 GMT
Content-Type: application/x-tar
Content-Length: 484520
Last-Modified: Sun, 04 Feb 2001 14:05:26 GMT
Content-Encoding: gzip
Age: 2054
X-Cache: HIT from localhost
Proxy-Connection: close

Connection closed by foreign host.
---

Obviously, mozilla is uncompressing the object. Obviously as well, this
behavior should be the same with or without a proxy. It's also bad because
mozilla is removing a content-encoding from the proxy case and that's not
allowed. It is very bad for the cache because the a potentially large
object is stored there and may affect cache performance.

It is extremely bad for the disk as well. Gzip will typically do 4 or 5 to 1
compression on text so disk requirements become 4*2 or 5*2 for an object. I
tried this with a 20MB compressed tarball that has a 5:1 compression ratio
so mozilla wound up using 200MB of disk space. Not good at all.

Comment 1

[Doug Turner (:dougt)]

ftp proxy is broken.  62353 is a meta tracking bug.

Comment 2

[Doug Turner (:dougt)]

Futuring all directory viewer bugs.

Comment 3

[tenthumbs]

Why is this a "directory viewer bug?"

Comment 4

[Doug Turner (:dougt)]

reassigning to [e-mail].

Comment 5

[Bradley Baetz (:bbaetz)]

"ftp proxy" is really http, so moving.

Is this still the case in current builds?

Comment 6

[Hans Schleichert]

Maybe my problem has the same origin. I am running a local proxy on my PC for
ftp/http/POP3 access. On some occasions, an attempt to d/l the most current
mozilla build resulted in the file obviously being downloaded from cache (took
10 secs instead of the 20mins it usually takes), and what I though of as being
the latest nightly turned out to be the version I had installed days ago.
Naturally, I first though something had gone wrong with the latest build
directory on mozilla's side, but I found out that when I download the same file
with a different local filename, the download contains what it ought to.

Comment 7

[benc]

qa to me.

ftp proxy is done via http, so I think caching is somewhat unavoidable.

as for handling the compression, I think there are other bugs that describe the
ideal behavior here. Not completely current on how that works...

hans: on face, your problem sounds like a possible cache problem. Probably a
separate bug would be good, so we can have some people get information on your
cache entry values, and decide if the proxy or the cache is doing this wrong.

Comment 8

[Hans Schleichert]

I'd like to leave it up to the insiders to decide whether or not to open a new
bug. Any information I could supply? I will try to reproduce the bug next time I
d/l a nightly, and if it persists, will try to bypass the proxy.

Comment 9

[tenthumbs]

Hans: proxies have to invent expiration dates for ftp URL's because ftp
doesn't give any hints and this occasionally causes trouble. Perhaps
your proxy is slightly mis-configured.

Comment 10

[Doug Turner (:dougt)]

moving neeti's futured bugs for triaging.