502253 - FF 3.5 : failure to download 4.3GB file using http. Only 0.3GB is taken.

Status: RESOLVED INVALID

(Firefox :: General, defect)

Description

[ISHIKAWA, Chiaki]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; ja; rv:1.9.1) Gecko/20090624 Firefox/3.5 (.NET CLR 3.5.30729)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; ja; rv:1.9.1) Gecko/20090624 Firefox/3.5 (.NET CLR 3.5.30729)

FF 3.5 : failure to download 4.3GB file. Only 0.3GB is taken.

FF 3.5 fails to downlod 4.3GB file.
Only the first 0.3GB (327MB) is downloaded and
the downloading is finished there prematurely.

I was afraid that the server side was behaving oddly for
a file that is larger than 4GB.
But when I checked the downloading using "wget", it went over
the 327MB limit without the problem and go on copying,
I assume there is no problem on the server side, and the
problem is that of  FF3.5.


Reproducible: Always

Steps to Reproduce:
1.Try downloading a 4.3 GB file
knoppix_v5.3.1DVD_20080326-20080523-AC.iso
from a Japanese web site.

http://ring.nict.go.jp/archives/linux/knoppix/iso
/knoppix_v5.3.1DVD_20080326-20080523-AC.iso

2. You see that the download finishes prematurely at 327MB.
Actual Results:  
The download leaves a smallish file (0.3GB, 327MB) instead of 4.3GB.

Expected Results:  
FF 3.5 should be capable of downloading the full 4.3GB file as "wget" can from
the same server.


cf. wget run with the initial server response.

 Interrupted wget session: it was copying well over 0.3GB) when
    it was stopped by control-C.

    I added "-S" to show the server response.

$ env LANG=C LC_ALL=C wget -S http://ring.nict.go.jp/archives/linux/knoppix/iso
/knoppix_v5.3.1DVD_20080326-20080523-AC.iso
--2009-07-04 02:48:04--  http://ring.nict.go.jp/archives/linux/knoppix/iso/knopp
ix_v5.3.1DVD_20080326-20080523-AC.iso
Resolving ring.nict.go.jp... 133.243.3.209
Connecting to ring.nict.go.jp|133.243.3.209|:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Date: Fri, 03 Jul 2009 17:47:44 GMT
  Server: Apache/2.2.11 (Unix)
  Last-Modified: Sat, 24 May 2008 23:37:47 GMT
  ETag: "79c9e1-114765000-44e026f0f08c0"
  Accept-Ranges: bytes
  Content-Length: 4638265344
  Keep-Alive: timeout=15, max=100
  Connection: Keep-Alive
  Content-Type: application/octet-stream
Length: 4638265344 (4.3G) [application/octet-stream]
Saving to: `knoppix_v5.3.1DVD_20080326-20080523-AC.iso.1'

 8% [==>                                    ] 385,256,176 1.68M/s  eta 30m 58s

ci@LENOVO-23C6C69F /cygdrive/c/download
$


My guess: a 32 bit unsigned entity is used to record the download file size
where 64 bit unsigned entity is appropriate.

Comment 1

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1) Gecko/20090624 Firefox/3.5 (.NET CLR 3.5.30729)

I got to 342 MB before I decided not to continue.  Can you use an addon like Live HTTP Headers to see what headers Firefox is sending to the server when you attempt to download?

Comment 2

[Matthias Versen [:Matti]]

And you didn't use a fat32 drive to save the file ?

Comment 3

[ISHIKAWA, Chiaki]

To comment #2, I used 100GB NTFS partition.
The same partition was used for the sample "wget" run in my post.

To comment #1, I will install an addon to see the header.
BTW, so your FF3.5 downloaded more than 342MB and still kept on
continueing? Hmm, could it be related to I18N/L10N issues in a mysterious way?

Anyway, I will my header findings later.

Comment 4

[ISHIKAWA, Chiaki]

I installed "Live Header" addon.
It is quite nifty.

And I found out the culprit very quickly.
Below is the header when I visited the 4.3GB and found that
it got truncated.

Note the " Via: 1.1 winxp.example.org:3128 (squid/2.7.STABLE6)"
header line in the final OK response.
It seems to be that the proxy chain (I use privoxy and squid due to
historical reasons) and somewhere the connection is cut
at 0.3GB. 

When I turned off proxy and accessed the mentioned URL directly from FF3.5,
it went past the previous limit and kept on copying.

So,  this bug can be closed. I have to check the proxies and
then report the problems up stream.

Sorry for my premature investigation, and thank you for the suggestion to use Live Header addon.

(These proxies worked like a charm. The last time they caused problems was
several years ago when junkbuster didn't handle HTTP/1.1 connection. You can
tell it was a long time ago in the Internet time scale.)

http://ring.nict.go.jp/archives/linux/knoppix/iso/knoppix_v5.3.1DVD_20080326-20080523-AC.iso

GET /archives/linux/knoppix/iso/knoppix_v5.3.1DVD_20080326-20080523-AC.iso HTTP/1.1
Host: ring.nict.go.jp
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; ja; rv:1.9.1) Gecko/20090624 Firefox/3.5 (.NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-2022-JP,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Proxy-Connection: keep-alive
Referer: http://ring.nict.go.jp/archives/linux/knoppix/iso/

HTTP/1.x 200 OK
Date: Sat, 04 Jul 2009 00:13:25 GMT
Server: Apache/2.2.11 (Unix)
Last-Modified: Sat, 24 May 2008 23:37:47 GMT
Etag: "79c9e1-114765000-44e026f0f08c0"
Accept-Ranges: bytes
Content-Length: 4638265344
Content-Type: application/octet-stream
X-Cache: MISS from winxp.example.org
X-Cache-Lookup: MISS from winxp.example.org:3128
Via: 1.1 winxp.example.org:3128 (squid/2.7.STABLE6)   <=== AHA!
Connection: close
Proxy-Connection: keep-alive

Comment 5

[Masatoshi Kimura [:emk]]

-> INVALID per comment #4