309879 - Downloads broken with disk cache disabled (fix sniffing in download code)

Status: RESOLVED INCOMPLETE

(Camino Graveyard :: Downloading, defect, P2)

Description

[Joel Craig]

User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.9a1) Gecko/20050921 Camino/1.0+
Build Identifier: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.9a1) Gecko/20050921 Camino/1.0+

This has been an ongoing issue for a long time. Attempting to download a file
using either the contextual menu command "Download Link Target" or via
Option-clicking a link will result in an "Interrupted" download IF you have
disabled Disk Cache in your "user.js" file, or disabling Write access to the
Camino cache folder in ~/Library/Caches. A new profile MUST be created after
removing the offending line from user.js or restoring Write access to the Cache
folder.

Firefox does not produce this exibit this behavior.

Reproducible: Always

Steps to Reproduce:
1. Disable Dsik Cache in user.js (user_pref("browser.cache.disk.enable", false);) OR
2. Allow Read Only access to the Camino Cache folder
3. Attempt to d/l using Download Link Target or Option-Click
 

Actual Results:  
Result is an "Interrupted" download

Expected Results:  
File should download successfully

Comment 1

[Paul Borokhov (lensovet)]

Can confirm this with both 2005092304 (v1.0a1+) and 2005091604 (v1.0a1+).
This is extremely annoying, and I have absolutely no use for a disk cache. I'm
on a broadband connection, and oftentimes, browsing is actually slower if you
have it enabled (after all...memory is quicker to retrieve than disk, and w/a
fast connection, there's no need for it all anyway)

Comment 2

[Chris Lawson (gone)]

Confirming.

For the record, I haven't seen the problem since I re-enabled my cache *without*
creating a new profile, so it's possible that the new profile step is a red herring.

I've also seen this in just a straight cmd-S download of images (usually on the
Astronomy Picture of the Day), so it looks like an issue with our download
handling in general, rather than something specific to link targets or the
opt-click.

cl

Comment 3

[Mike Pinkerton (not reading bugmail)]

do we differ from firefox in this behavior?

Comment 4

[Joel Craig]

Yes we do. Firefox is able to successfully download in the manner described with
Disk Cache disabled.

Comment 5

[Paul Borokhov (lensovet)]

bump!
it appears that the new iconcache fixed this, because i can now download using the contextual menu item. my "web page" disk cache is still disabled, so my guess is that the addition of the icon cache made this bug go away.
can someone else confirm this? i'm currently using 2005122604 (v1.0b1+).

Comment 6

[Simon Fraser [no longer active]]

Sounds fixed then.

Comment 7

[Joel Craig]

(In reply to comment #6)
> Sounds fixed then.
> 

Confirming seems to be WFM as well. D/l 2 .dmg files from MacUpdate successfully, one via option+click, the other right-click+Download Link Target. No more "interrupted" downloads. Nice work fellas (and gals)!

Comment 8

[Joel Craig]

(In reply to comment #7)
> (In reply to comment #6)
> > Sounds fixed then.
> > 
> 
> Confirming seems to be WFM as well. D/l 2 .dmg files from MacUpdate
> successfully, one via option+click, the other right-click+Download Link Target.
> No more "interrupted" downloads. Nice work fellas (and gals)!
> 
Just to add, I'm using krmathis G4 Trunk nightlies, currently the Dec. 31 build. Seems to work both on branch and trunk.

Comment 9

[Chris Lawson (gone)]

This *may* not be totally fixed.

http://forums.mozillazine.org/viewtopic.php?t=370253

Specifically, "After playing with variations of the cache settings, it turns out that the only combination that works is to turn on disc cache and turn off memory cache."

I'm using them both, and I've had no problems. It's entirely possible something is screwy with that guy's system, since a fresh profile didn't fix it, but I'd like to see a few other people try to reproduce his problem.

cl

Comment 10

[Paul Borokhov (lensovet)]

fwiw, here are my "cache" settings pulled through about:config:
http://img61.imageshack.us/my.php?image=aboutconfig1gw.png
downloading seems to work w/out any problems...

Comment 11

[Joel Craig]

(In reply to comment #9)
> This *may* not be totally fixed.
> 

Well, now I've got a problem again. I disabled Disk Cache in CamiTools and tried to d/l today's OpenOffice update (.dmg file from a couple different mirrors). Both Option-Click/Save and Download Link Target resulted in an error and an interrupted download. Simply re-enabling Disk Cache and trying again didn't help. I had to quit and restart Camino, which then allowed a successful d/l in the mannet described.

Using Jan. 29 krmathis trunk build.

Comment 12

[Chris Lawson (gone)]

I can repro this 100% with disk cache DISABLED and memory cache ENABLED on Camino 1.0RC1. This configuration consistently gives an error sheet saying "your network connection was interrupted" and gives an interrupted download.

If disk cache is re-enabled, the problem goes away.

If memory cache is disabled, the problem goes away.

If disk cache is enabled *and* memory cache is disabled, the problem goes away.

Reopening so we can take a more thorough look at it.

cl

Comment 13

[Chris Lawson (gone)]

Tweaking summary. As noted above, this happens whether or not "Download Link Target" was used.

cl

Comment 14

[Simon Fraser [no longer active]]

Here's what seems to be happening:

When we start a download, we fire off a request via nsIWebBrowserPersist. When we get the OnStateChange for nsIWebProgressListener::STATE_START, we save the "content-type" and "content-disposition" headers, then we cancel that download. We then show the file save dialog to the user, and, if they confirm, start ANOTHER nsIWebProgressListener download.

On this second download, nsHttpChannel::ProcessResponse() sees a 206 status, and we have cached partial content (from the previous cancelled download), so we go into ProcessPartialContent(). This hits the Cancel(NS_ERROR_INVALID_CONTENT_ENCODING) code path, and then seems to try to read from the cache.

HTTP log coming.

Comment 15

[Simon Fraser [no longer active]]

Attachment: http log

Comment 16

[Darin Fisher]

It seems to me as though you should avoid attempting to re-download the file.  There are many cases where you cannot re-download successfully.  You have to keep the original stream going.

Comment 17

[Paul Borokhov (lensovet)]

i guess this makes sense with simon's and darin's comments, it works on some sites but not on others.

Comment 18

[Mark Newland]

Just trying to save a web page with Version 2006040804 (1.0.0+)
http://linux.slashdot.org/article.pl?sid=06/04/09/1910215

Camino gave the Interrupted error and ? icon in the downloads window.
From the console log

2006-04-10 09:04:00.316 Camino[242] html
2006-04-10 09:04:09.493 Camino[242] html
2006-04-10 09:04:19.305 Camino[242] html
2006-04-10 09:07:33.297 Camino[242] html
2006-04-10 09:08:09.701 Camino[242] Download failure code: 804B001E

Comment 19

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

The sniffing causes other problems throughout our downloading experience (not just with cache disabled and so forth), so we really should fix this sooner rather than later.

Comment 20

[Paul Borokhov (lensovet)]

any progress on this? i swear, downloading DOES work on some sites, though i really couldn't tell you why.

Comment 21

[Chris Lawson (gone)]

*** Bug 353523 has been marked as a duplicate of this bug. ***

Comment 22

[Stuart Morgan]

(In reply to comment #14)
> Here's what seems to be happening:
> 
> When we start a download, we fire off a request via nsIWebBrowserPersist. When
> we get the OnStateChange for nsIWebProgressListener::STATE_START, we save the
> "content-type" and "content-disposition" headers, then we cancel that download.
> We then show the file save dialog to the user, and, if they confirm, start
> ANOTHER nsIWebProgressListener download.

Simon, you appear to have written this code originally--do you remember why it was done that way?

Comment 23

[Simon Fraser [no longer active]]

(In reply to comment #22)
> (In reply to comment #14)
> Simon, you appear to have written this code originally--do you remember why it
> was done that way?

It was copied from the XUL/JS downloading code of SeaMonkey/Firefox. I believe that you had to start the download before showing the Save dialog to get the correct MIME type and content-disposition, but I don't know why we had to abort the old download, and start another one.

Comment 24

[Chris Lawson (gone)]

*** Bug 325946 has been marked as a duplicate of this bug. ***

Comment 25

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Minusing this for 1.1 since no one's looked at it; we really need to fix the sniffing soon-ish, though.

Comment 26

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Setting priority per 1.6 roadmap.

Comment 27

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Setting priority per 1.6 roadmap.

Comment 28

[Stuart Morgan]

kreeger, have you looked at this? If not, we should probably accept that rewriting download handling is (unfortunately) probably not going to happen at this point in the 1.6 cycle, and push to 2.0.

Comment 29

[Nick Kreeger]

(In reply to comment #28)
> kreeger, have you looked at this? If not, we should probably accept that
> rewriting download handling is (unfortunately) probably not going to happen at
> this point in the 1.6 cycle, and push to 2.0.
> 

Sorry... time hasn't allowed.

Comment 31

[Chris Lawson (gone)]

See also the discussion in http://forums.mozillazine.org/viewtopic.php?t=606864 . We're starting to get a lot of rather non-obvious symptoms associated with this bug, and I'm wondering if we should push this up priority-wise.

cl

Comment 33

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Bug 299372 has some interesting discussion of issues surrounding this (our "start and cancel" behavior probably originated as an attempt to solve the "provide a filename, but don't wait forever to show the user the save dialogue" problem)--if you can wade through it (look for biesi, bz, and dmose and you get most of the important bits).

Comment 34

[Stuart Morgan]

As much as I'd like to see this fixed, it's looking unlikely to happen for 2.0 as well. If a patch shows up, we'll obviously take it.