288585 - Camino does not handle downloading of files > 2 GB

Status: RESOLVED FIXED

(Camino Graveyard :: Downloading, defect, P2)

Description

[riot.nrrrd.bugzilla]

User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8b2) Gecko/20050315 Camino/0.8+
Build Identifier: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8b2) Gecko/20050315 Camino/0.8+

I'm an Apple ADC Developer at work, and Apple has been releasing Tiger seeds as
DVD disc images.  These .dmg images (as of the last few months) have started to
exceed 2 GB in size.  When I try to download them in Camino (0.8.2, or a
semi-recent nightly build), they get downloaded until they reach 2 GB exactly,
then Camino stops the download and considers it finished.  Needless to say, it's
not really finished and of course the disc image doesn't mount.

Reproducible: Always

Steps to Reproduce:
1. Log into http://connect.Apple.COM/ if you're an ADC member
2. Go to Downloads
3. Go to any recent Tiger build and click on the download link for the DVD image

Actual Results:  
A 2 GB (exactly) downloaded file.

Expected Results:  
Downloaded the complete file, past the 2 GB length.

Comment 1

[Samuel Sidler (old account; do not CC)]

This is a known issue. When the patch from bug 228968 gets checked in, we'll
need to verify it fixes Camino.

Comment 2

[Christian :Biesinger (don't email me, ping me on IRC)]

(In reply to comment #1)
> This is a known issue. When the patch from bug 228968 gets checked in, we'll
> need to verify it fixes Camino.

it won't. camino will need additional changes.

Comment 3

[Simon Fraser [no longer active]]

I presume we need to switch to 64-bit file sizes and byte display in the
download code, and use 64-bit web progress APIs.

Comment 4

[Christian :Biesinger (don't email me, ping me on IRC)]

yep. In camino/src/download/nsDownloadListener.mm, in OnProgressChange64, you'll
have to do "something sensible". I didn't want to touch that Objective C code, I
don't know the language and wouldn't be able to test changes.

Comment 5

[Jasper]

Bug 228968 was fixed today, could we perhaps look at this soon?

Comment 6

[Hiro]

Attachment: screen shot of Download Manager

The download of the DVD ISO image of FC2 was tested. 
The size etc. of the total are displayed by mistake like the screen 
shot. 

http://ring.asahi-net.or.jp/archives/linux/fedora/linux/core/2/i386/iso/

Comment 7

[Hiro]

sorry spam...

In download manager, even if the indicator becomes full, download seems to
continue. 
But It doesn't increase from the following sizes of the file. 
Actually, download seems to have stopped. 

-rw-r--r--  1 sek  sek  75673600 10  4 08:56 /Users/sek/Desktop/FC2-i386-DVD.iso

camino 2005040808 (v0.8+)

Comment 8

[Mike Pinkerton (not reading bugmail)]

this is sorta embarrasing. we should at least be able to d/l to 4GB (size of an
unsigned long). 

Comment 9

[Mark Mentovai]

I have a patch.  I'll submit it later, once I'm done testing.

Comment 10

[Mark Mentovai]

Attachment: Use 64 bits for progress reporting

According to Moore's Law, this should buy Camino another 48 years.

This patch installs the last mile of plumbing that was left out of the fix to
bug 228968.  (Nice work, Christian.)  It brings 64-bitness all the way through
to the status bars (which themselves accept doubles).  I updated both
nsDownloadListener and CHBrowserListener to properly support
nsIWebProgressListener2.

This tests well in Camino.  I was successfully able to download a 5GB file over
FTP and have the progress bars and sizes do the right thing.  HTTP is a
different story.  I have been unable to download anything larger than 1GB over
HTTP on the trunk.  The fox and the lizard both agree, and not just on the Mac,
so this isn't a problem with the 'mino.  With this patch, the status bar DOES
accurately reflect that the HTTP download stalls at 1GB/5GB (for example).

I don't see any bug reports on the 1GB problem, so I'll file a report on that.

Comment 11

[Mike Pinkerton (not reading bugmail)]

awesome mark. i landed what you put up here. please file that other bug on http,
though it doesn't sound camino (or mac) specific.

thanks!!

Comment 12

[Mike Pinkerton (not reading bugmail)]

Comment on attachment 181364 [details] [diff] [review]
Use 64 bits for progress reporting

r=pink

Comment 13

[Mike Pinkerton (not reading bugmail)]

if the dependent bug ever lands on the 17branch, we should try to land this
there as well for 0.8.x

Comment 14

[Mark Mentovai]

Filed bug 291230 for the truncated download problem.  It's actually truncating
at (filesize % 4GB).  I just tested Camino with large files under 4GB and it
downloads them properly, and with my patch applied, the progress bars and text
display flawlessly.

Bug 291230 explains the behavior seen in comment 7, provided that the file was
supposed to be 4370640896 bytes.  A Google search for that number and the
filename confirms this.  This is good, because I wasn't really sure what to make
of that comment before, but it seemed to be a different bug.

Oh, and - my pleasure!

Comment 15

[Christian :Biesinger (don't email me, ping me on IRC)]

(In reply to comment #13)
> if the dependent bug ever lands on the 17branch, we should try to land this
> there as well for 0.8.x

I don't really want to land all those API changes on a stable branch...