Closed Bug 55690 Opened 24 years ago Closed 21 years ago

Spool file should be moved once the user picks a filename (saving a file creates a file before location is chosen)

Categories

(Core Graveyard :: File Handling, defect, P2)

Product:
Core Graveyard
Component:
File Handling
Type:
defect

Tracking

(Not tracked)

Status:
VERIFIED FIXED
Milestone:
mozilla1.7alpha

People

(Reporter: devsin, Assigned: Biesinger)

References

(Blocks 3 open bugs)

Details

(Keywords: arch, Whiteboard: se-radar[adt2] (py8ieh:248) read comment 238, 280 before commenting)

Attachments

(1 file, 3 obsolete files)

patch v3
2.23 KB, patch
bzbarsky
: review+
darin.moz
: superreview+
Details | Diff | Splinter Review
Mozilla M18 2000100608, Mac OS 9.0.4 - 1) Try to open a link to an unsupported file type (.tgz, .gz, .zip, etc.) 2) You get a dialog asking what you want to do (save or open) 3) Choose to save the file to disk 4) Navigate to your default downloads directory A file has already been created (file.ext). If you choose to save the file in that folder, you get a dialog saying a file already exists in that location and asking if you want to replace it. If you replace the file, the download proceeds as expected, but this is unnecessary confusion for the user. Mozilla should not create any type of file until the user selects a location for said file.
This has been 75% resolved in recent times - temporary files that are created are given wonderfully non-descriptive titles ("x4as73ggp.ext") so you no longer get the "Do you want to replace this file?" dialog. My only concern is that these temp files are created in the user's downloads directory instead of some temporary folder in the Mozilla data folder or Preferences folder. But the way downloads work (files saved to disk keep the obscure names until downloading is complete) this may not be an issue.
OS: All
See bug 60203 for the "wonderfully non-descriptive titles" given to these "temporary" files.
Moving to networking per bug 60203
Assignee: don → gagan
Component: XP Apps → Networking
QA Contact: sairuh → tever
Also seeing this on WinNT and Linux [netscape 6.0 official release]. Marking platform = ALL.
Hardware: Macintosh → All
->dougt
Assignee: gagan → dougt
Blocks: 62352
voting for nsbeta1. this is a bug (not a feature, imho) that causes us to need to do other crazy things, such as salt filenames. the solution is to fix this bug, not bend over backwards and confuse the user.
Blocks: 60203
Severity: normal → critical
Keywords: nsbeta1
adjusting milestone
Target Milestone: --- → mozilla0.9
bill, I think that this is your bug.
Assignee: dougt → law
Not going to happen for mozilla0.9, at any rate, since this was reassigned, the target milestone should be reset.
Target Milestone: mozilla0.9 → ---
The original bug description says "Mozilla should not create any type of file until the user selects a location for said file." Unfortunately, that is preceded by description of a scenario that no longer occurs. I'm left wondering what exactly the bug is. If it is (was) that saving sometimes results in a "file already exists" scenario, then that is fixed (by way of the "salting" of file names). If the problem is that we download before the user tells us where, then that's still a bug. So which is it? Or is it something else? BTW, there's not much chance we're going to change the current behavior.
The bug is that we still start the download *before* the user specifies a location to save the file. We do the background spooling thing to a temporary file, which then forces us to salt names and stuff because we're downloading before the user has accepted the download. We should just not start downloading until the user has specified a save location, then not salt the filename.
A couple of reasons why you shouldn't start downloading a file before the user selects the filename and hits OK: * It is totally unexpected behavior. Sure it sounds like a slick PR feature, but it could potentially be very annoying on a slow (PPP) link. It also makes the "Cancel" button in the filename dialog essentially worthless (the file may have already really been downloaded, like it or not), and the "elapsed time" and progress indicator in the download dialog box are (sometimes) worthless as well. * It pretty much requires mangling the filename, which is also fairly unexpected. This clobbers the ability (on Linux, at least) to either remotely monitor the progress of large downloads (since you don't know the filename) or 'mv' the file to a new location while it is downloading (which I used to do rather frequently with Netscape).
The current behavior is due to a couple of reasons: 1. By the time we've determined that the content can't be displayed in the browser/mail/etc., we've already downloaded some/all of it (i.e., it's coming across the wire). If we don't start saving it to disk (and thereby accept the data coming in), then we must re-establish the connnection, thereby wasting the initial connection and/or transfer. 2. In some cases (imap attachments?), it is not possible to re-establish the connection and get back to the data so we have to do something with it immediately. Changing the current behavior is unlikely to fit within the resource limits we have for pre-mozilla1.0. I'm going to triage this and "minus" it, as we say.
Keywords: nsbeta1 → nsbeta1-
Why can't we hold the connection open until the user dismisses the dialog? I'm sure most servers wouldn't timeout for that period. If they do, then we need a way to re-establish the connection, which surely can't be that hard. Just fire off the URL again.
re: why not hold connection open At one point, it was impossible to get Necko to do this (either that, or impossible to figure out how to get it to do this). I should be more precise, though: the problem was in handing off the request from the machinery under the browser window to another set of observers/listeners/etc. That's been solved to a considerable extent (as a byproduct of code related to streaming into the temporary file) so it may no longer be as difficult a hurdle as it was previously. re: re-establishing the connection I'm not sure how hard that would be. We're not real good at dealing with that now (in other contexts), from what I can observe. That still leaves the issue of the content that we can't re-request. Sorry, but I'm not familiar with the details of that.
Marking nsbeta1- bugs as future to get off the radar
Target Milestone: --- → Future
This is so catfood, but it's latered. Anyone?
if you want to delay downloading until a user has selected a download location then you should really delay starting the download. i don't think we want to try to solve the problem of re-establishing broken connections. it seems that this behavior was put in as an optimization, to reduce the time the user is waiting for content to download. but, some users don't like the fact that we do this. sounds like justification enough for a preference.
or not. Why even bother downloading until a file location is know? How much download time is lost on the average while the user searches for a place to save it? 2-5 seconds? Lets keep it simple. Ask the users where they want it and download it there. Don't pretend to do something smart.
exactly!! *pinkerton hugs dougt*
Unfuturing because concensus is that this sucks too badly to ignore.
Target Milestone: Future → ---
sounds good to me... any idea how difficult this fix will be? we probably do a lot of extra work just to be able to support "Saving a file creates a file before location is chosen."
Transferring to mscott (I fended these people off as long as I could, Scott :-)
Assignee: law → mscott
marking milestone as 0.9 so that it will appear on mscott's not so short shortlist.
Target Milestone: --- → mozilla0.9
moving to 0.9.1
Target Milestone: mozilla0.9 → mozilla0.9.1
*** Bug 74008 has been marked as a duplicate of this bug. ***
Additional Comments From timeless@mac.com 2001-04-01 11:58 in Bug 74008 Um silly question... is it possible for a download session to change where it writes while it's downloading? eg. we d/l 12% to a temp file. the user tells us where to write (and it's on another drive), we copy that 12% to the new file. [2% builds up], we copy that 2% and then update the file writer to start writing to the correct file (actually, assuming we have binary seek we could probably start writing the incoming data immediately where it belongs and then copy over the start buffer when possible). when we're done, we kill the start buffer. otoh, we could just ask the os to buffer the entire file in memory until the user picks a location (wouldn't this make more sense on most operating systems w/ decent vm?)
Keywords: arch
No longer blocks: 74008
*** Bug 76718 has been marked as a duplicate of this bug. ***
moving to 0.9.2
Target Milestone: mozilla0.9.1 → mozilla0.9.2
Timeless, I like your ideas, I was thinking more or less the same thing as I read through this bug. [I think the suggestion about downloading the file, then copying over the start buffer is not so good though. It would prevent things like playing audio/video files whilst they are being downloaded.]
mass move, v2. qa to me.
QA Contact: tever → benc
*** Bug 74739 has been marked as a duplicate of this bug. ***
moving to 0.9.3
Target Milestone: mozilla0.9.2 → mozilla0.9.3
+mostfreq...
Keywords: mostfreq
No longer blocks: 74739
Little note: IE also stores the download in a temporary location on my C: drive. I often download very large files (Slackware ISOs) to my F: drive, which has, say, 2 GB free. If my C: has less than 600 MB available, I cannot download the file. In this situation, I use NS4 to download things, as it downloads directly where I tell it. I'd very much like for Moz to follow NS4's lead on this issue. :) Adding 4xp.
Keywords: 4xp
This could also be related to stopping and resuming of downloads, because if you know a file's location, it makes it much easier to resume an aborted download.
As has been said before we really must stream the data someplace while the user is selecting a final location for it, otherwise we'd risk losing our network connections, etc. That being said, I think we should move the file to the correct location ASAP. It seems like it should be possible to simply stop writing to the file, close it, and then move it (when it is still small) and then resume writing to it in the new location. mscott: does this sound doable?
>As has been said before we really must stream the data someplace while the >user is selecting a final location for it, otherwise we'd risk losing our >network connections, etc So how does 4.x deal with the same problem?
i don't see why we're so worried about losing connections and the like. 4.x does the right thing and no one has ever complained. why is this suddenly so difficult to implement?
i'm not sure why it wasn't an issue for 4.x, but i can tell you that when downloading something large, i have in the past really REALLY appreciated the fact that we start downloading before a location has been selected. it means that i usually get the file that much quicker, and over a fast connections it can sometimes mean no waiting at all. so, let's not even consider throwing out this feature. i think it should be simple enough to move the file part way through the download, and hence make it appear as though we are saving it to the desired location. how does that sound?
Pink it is hard to implement something like this because our download logic is keyed off of content type and this usually/always happens after we start downloading bytes. We could stop worring about this, if mozilla just had a default download folder and all of the saved bits would go there. If people wanted to explictly save something to a destination, we could do this via a right click. With this kind of explict "SAVE-AS" functionality, we could invoke the download logic immediately. my 3 cents.
what if the location you're moving it to is not local (say your iTools disk over a slow network connection)? Copying the file is not always trivial or instantaneous.
my point exactly: the copy is much faster when the file is still small.
Doug, how would having a default download directory help things ? The problem is with the file name, not with its location. Besides, there is already a default download location, e.g /tmp on Linux. What I would like to see is something like this: 1) User clicks to download file. 2) File begins downloading to /tmp as usual. 3) User now inputs filename for saving to. 4) *Mozilla checks to see whether the file name given already exists.* - If it does exist, and the existing file is *larger* than the downloaded chunk, the user is asked if they want to continue downloading, or to overwrite the file. If they select to continue loading, we delete the temporary chunk, and continue with the existing file. - If the file does not exist, or is *smaller* than our new chunk, or the user has chosen to overwrite, copy the downloaded chunk to the selected location, and continue downloading from there. And one last thing, I agree with Darin, I *love* the fact that mozilla starts downloading as soon as it can, if I were _forced_ to choose I'd rather keep this functionality than have mozilla wait for me to give it a file location before it started downloading. Navigating around the filepicker can sometimes take a while.
About the one-download-folder-for-all thing, I don't want that to happen. If we did that this would become Netscape SmartDownload, and I don't want that. *Represses memories of SmartDownload and shudders...* People want to save files where they want to save files, and nowhere else. Current behavior makes more sense - save it temporarily in one place and move it to destination later.
> About the one-download-folder-for-all thing, I don't want that to happen That may be the case on your platform, but on Mac, users expect files to go to their specified download folder (which is a system-wide preference). Right now, they have to work their way through two (2) dialogs to get there. Downloading with 4.x, or a dedicated FTP client, requires zero dialog messing.
here here. Download the file to the same place every time. If you want to be explict about where it goes, the UI should have some way to you can context click for a "save-as" dialog.
yeah, good idea! but, just to be clear, i don't think any of these dialogs should hold up the download. we could simply download to the default download directory (with a randomized file name), and then, once the user has made the final selection of where the file should go, we should move the file to the correct location (maybe just renaming it in the default download directory) and continue the load. the critical part to this IMO is to perform the file move immediately after the user has selected where to put the file. then there won't be any confusion about where the file is while it is downloading. this bug may also want to concern itself with the fact that file downloads get streamed to the disk cache as well. we probably don't want to do this.
Ick. Why have a randomized file name? Why copy the file anywhere? I see two scenerios: 1. User click on something that winds up being download. We toss a progress dialog and download the thing to the disk. The dialog remains asking the user if they would like us to open up where it was saved. It was saved to the default download directory. This probably is not /tmp, but it could be. more like ~/moz_downloads or something... On other platforms, this is what most clients do. 2. User wants to save the remote files somewhere explictly. User context clicks and selects where to save the file. We get that info, *then* begin downloading directly to where the user specified. No Copy.
I think the idea that downloading the file in the background while the user chooses a save location is a good idea, because it makes things appear quicker, is misguided. I think you loose in user confusion what you've gained in time saved. And I think that the user won't mind at all if the download is prolonged by the amount of time that they spent in navigating the dialogs. In short, I think not doing background downloading is much better from a user experience perspective; it's less confusing, and doesn't really save that much time.
I agree with Simon. This behavior seems totally unnecessary, and all of the proposed solutions seem like too much work for something as simple as downloading a file. The time it takes to choose a location to save or a helper application to process the file is negligible. If you run off to tiptoe through the tulips before performing this task, you deserve to wait. Like the less-than-30-seconds it takes to deal with the dialog is going to make some big difference. With all the suggested solutions, there's going to be a lot of steps for something so simple. Choose to save it to the default downloads directory (where the file is already being downloaded), and you just have to rename (really a solution to bug 60203 and not this one). If you choose to save it to a different location, the file has to be copied and renamed. What if resumable downloads ever becomes a feature that works? What happens if the download stalls or the program hangs before you can perform the copy/rename? You'll end up with a file possibly in the wrong location and definitely with the wrong name (and, if any amount of time passes, you might not remember the file you were downloading, and the filename definitely won't be any help). Darin says: "the copy is much faster when the file is still small." Uh, isn't the whole point of the argument so that you can lounge around with the download dialog up but still have the download begin? What assurance do you have the file is going to be small? If I have a super-fast connection, routinely download large files and realize that Mozilla downloads whether you tell it to or not, why shouldn't I go get another cup of coffee before the dialog even comes up? What if I end up with a 50-100MB file (let's say I'm a slow walker)? What if I decide to use the restroom, too, and have to go downstairs and end up with a 150MB file? Then, you have to copy this ridiculously large file, you have to rename it, and you have to begin writing to the new location and delete the old file. What if I only have 72MB of HD space left and you try to copy a 150MB file that's still downloading? With 4.x, it would work fine (I'm downloading a 200MB file and have 222MB left), but with Mozilla, I'm not sure what you'd do. If none of these seem to be real (or probable) issues, or it is too difficult to change the behavior to that of 4.x or IE, then go ahead and do what you want to do. If you absolutely must continue this behavior, then I'd suggest at least throwing up the progress dialog before the download dialog comes up. Even if the progress meter isn't doing anything, the user will be informed that download has begun and is progressing. Then you can perform the copy/rename voodoo, and hope for the best. My major problem with this still remains, though: it makes the behavior in bug 60203 necessary, and it's performing an action that the user has not specifically requested (I don't want some file on my HD until I have confirmed that I do indeed want to download that file).
if we hold up the downloading for an indeterminant amount of time, then we risk loosing the connection to the server. remember we have to handle the case in which the user clicks on a link and we discover an unknown content-type in the http response headers. what do we do with the data while the user is deciding where to put it? streaming it to a temporary location seems like the correct thing to do since you wouldn't want to buffer the data in memory. btw: this temporary location could also be the disk cache, though i would prefer if it could be in the user's "preferred" or "default" download directory, so that the cost of renaming the file would in most cases be minimal.
Is it so very hard, then, to re-establish the connection? Yes, this might result in a delay since you have to reconnect to the server, but why is that such a terrible thing? And why can't you just handle unknown content-type issues differently than what you'd normally do (have a differnet method for dealing with them)? The behavior in Netscape 4.x is perfectly acceptable, and displays none of the issues discussed here. Downloads and unknown content-type scenarios are handled properly and filenames are kept intact, and the file is always where you expect it to be. I haven't noticed any incredible gains in speed with the way Mozilla does this. Like I said, in most cases, you're only shaving off 30 seconds or less of download time. I understand that it's not as simple as I'm making it sound, but I am not at all impressed with the implied advantages of the current behavior, and none of the proposed solutions seem like they will ever make the process better, nor as simple and intuitive, as the behavior in Communicator.
if the move happens when the user selects a location, then from that point forward we will appear to behave identically to 4x with the exception that we'll have downloaded the file sooner. the perceived benefit of downloading in the background really depends on your network setup. over a 56k modem, it really helps to hit the wire as soon as possible... especially for smaller files. also, restarting a download is a non-trival thing. servers don't always support byte-ranges and the content being downloaded may have changed, etc. FWIW, IE also downloads in the background to a temporary location until the real location is selected.
Blocks: 85115
You made your point well. I still agree with Simon Fraser and Mike Pinkerton that this is unnecessary and not as good as people want it to appear, but I'm resigned to living with the behavior. The only thing I really care about anymore is bug 60203. That is the biggest pile of stink that's around. Implement the feature so that the file is renamed (I don't care about where the file is, because I always download to the same directory, and you don't have to worry about stupid things like \WINDOWS\TEMP on the Mac) when the user chooses a location/name/helper app and it will be acceptable. So, fix that (which will eliminate this and bug 60203), and I'll shut up. I promise :) (Note that you have to change the name to what it's supposed to be even if the user chooses to open the file with a helper app, otherwise you still get left with a junked filename and defeat the whole purpose.)
> Is it so very hard, then, to re-establish the connection? Sure, it could cost me $1000 if the server decides that I'm trying to cheat it out of a second copy of an Ebook, Erecord, Eproduct, or Eserver. > Yes, this might result in a delay since you have to reconnect to the server, > but why is that such a terrible thing? Welcome to Ecommerce, where it could cost me more pretty pennies than I can afford. wrt resumable downloads that really has very little to do w/ this bug. one possible implementation is to stick 'realfilename.realextension.mozresumableextension' into 'realdownloadfolder' (not to be confused w/ RealDownload) and have it contain source url and the intermediate directory path.
timeless, if reconneting in such a problem with eCommerce, why doesn't anyone bitch about 4.x? why is this all of a sudden a problem?
*** Bug 85844 has been marked as a duplicate of this bug. ***
I'm using Linux and I *DONT* like it downloading to /tmp and filling up my 125 MB /tmp Directory when I am trying to download the 650 MB ISO image to another place. It bugs me immensely to see mozilla downloading this to both the NewCache dir AND the /tmp directory. It also bugs me that the tmp file created in the NewCache dir got to be double the size of the file that used up all my /tmp space. And it bugs me even more that the smaller file in /tmp got copied to the location I specified the file should have been downloaded to. *IF* mozilla had either downloaded it *ALL* to the NewCache dir *OR IF* it had not filled up my /tmp dir *AND* downloaded the file to the *RIGHT* place I would have been one hell of a lot happier. Having a default download location is alright *IF* the users can override it. This has been a major pain to me. I do *NOT* want to have a 1 Gig /tmp directory just because Mozilla wants to download anything to /tmp while I figure out where it goes. (means I will have to stuff around in a big way on my work and home linux boxes) Plus as I pay for my ADSL connection to a monthly MB limit I am concerned about the way mozilla just starts downloading in the background... Every MB over the limit costs extra.
my proposed solution would solve your pains completely.
It would? What if the user dawdles at the save file dialogs, or goes away and make a cup of coffee before the save file dialogs come up?
if we just saved the file to a default (user-configurable) location, but allowed the user to specify an optional location while the file is being downloaded, wouldn't that work?
So the user now has to specify 2 places to save files: 1. The default location (set in prefs) 2. The location for this particular download Why do you need the second? If the user specifies a location for saved files in the prefs, just save to that, without any prompting. Or are we going to get all up in arms about saving files "quietly" because of security issues?
1. The default location (set in prefs) 2. The location for this particular download Simon Fraser wrote: > If the user specifies a location for saved files in the prefs, just save > to that, without any prompting. The "default location" for download is just a suggestion offered by Mozilla to the user. Still, the prompt for "location for this particular download" is necessary, coz the user need to have final decision on the filename and the location of the file to be saved. > Or are we going to get all up in arms about saving files "quietly" because of > security issues? As mentioned, I think it's just a normal attitude to notify the user that Moz is going to write something new on the user's hard disk. I don't think this is a security freak-out... That's normal way of interacting with the user.
I think adding a pref for a default download folder would be a bad idea, unless you could specify one folder for each of the file types (i.e. save MP3, MIDI, and waves to the /media folder, save html, text, and PDF to /docs folder, save ZIP's, MSI's, and executables in the /download folder, specifying each in the helper apps window). Unknown/unspecified file types should be downloaded to a user folder such as users/name/downloads/ and moved to the location the user initially specifies when the download is finished. Also, files that are set to run in an application should be run from the user folder, unless the user specifies a folder (in which case the process would be to download the file to the specified folder and run it from there). There would be no initial security risk because there would be no default folders (users who specify these folders later on acknowledge any risks one might expect from a "quiet download"). If no location is chosen by the time the download finishes, the finished file would be left in the user download folder and some notification would be present (e.g. opening a sidebar with "Complete Downloads" - below this could be "Interrupted Downloads" ready to be resumed).
Why not have an option in 'helper apps', so that when you define a mime type, you could also specify a default download location ? But that really belongs to bug 7840.
Yuck. Double and triple yuck. That has got to be the most confusing way to get around actually fixing this bug. Mozilla would have to keep track of every single path to a directory for a certain MIME type. What about users who don't know how to change settings to their helper apps? What about users who download everything to a single directory? Are you going to have Mozilla change the default directory for every single MIME type when you change the global downloads directory? I would suggest choosing a default location in the preferences and having a separate option to "Always download to the default downloads directory/folder/whatever." Is this a direct copy from the way IE does it? Yes. Do I care? No. This way, users can simply uncheck that option and choose different folders to download their MP3s to folder A and JPEGs to folder B and archives to folder C and I can leave the option checked and everything will download to folder A (the way I like it). Remember, if it works out best for Adam, it's usually the best solution. ;)
So, to summarize ... Reasons to create a file before the user asks you to: (1) It saves you up to about 20 or 30 seconds of downloading time (Darin's argument). (2) Waiting might lose the connection and require asking the server for the file a second time, which could worry pay-per-download businesses (Timeless's argument). But if that's the case, then you're also hosed if you get randomly disconnected by your ISP, or if you have a power blackout, or if the remote server goes down, or whatever. So any business which relies on pay-per-download is asking for trouble already. Reasons to not create a file before the user asks you to (problems which would *not* be solved by moving the partial download once the user specifies a location): (1) For small files, it appears that something has gone wrong with Mozilla, since the progress indicator is at 100 % when the progress window first appears. (2) The estimated time remaining and transfer rate shown in the progress window become very inaccurate. (3) If the default download folder is the desktop folder (as it is on Mac OS by default), unsightly files are put on your desktop until you specify the download location (bug 60203). (4) The desired destination folder may not be mounted (e.g. iDisk), and mounting it and moving the partial download to it may take long enough that the connection gets dropped anyway (Timeless's argument in reverse). (5) The volume where the default download folder lives may not be large enough to store the temporary file, whereas the desired destination folder is. (6) Mozilla, like the Finder, should refuse to even begin copying a file where it knows that the file will not fit. My decision to download something may depend on whether it fits on a Zip disk or not; and if it does not, but I am still charged by the megabyte for the partial download, I will not be amused.
AK: Any user who couldn't set a default path for helper apps doesn't need to be using Mozilla in the first place. (You don't need me to tell you that this program is NOT for newbies.) The functionality should be there, but in case someone doesn't want/know how to set the default folders for each file type, it would be unspecified and the default download folder would be used (as I suggested, something in a user folder) - and then the user should be prompted for save path every time they try to save. I suppose the default save folder could be changeable too, but I've never known anyone to want to save EVERY file type in one place. As I said, people will want their music here, their pictures there, their HTML over here, and their compressed archives over there somewhere. It might be a little more work getting the support in, but the end result is much better than just having one download folder.
mpt-6. has a bug. not all file systems can correctly indicate space available. :( my comment about that appears to be in another bug. from memory, Netscape4 will occasionally warn me that it doesn't think there's enough space in f:\programf\dev to download a file, it lets me tell it to continue anyways. this is often how my system looks: 25 Dir(s) 294,720 bytes free F:\> 31 Dir(s) 275,169,280 bytes free F:\programf\Dev> 2 Dir(s) 2,286,362,624 bytes free F:\build\beonex> If I try to download an iso disk image to f:\build\beonex and my browser refuses I'd be very upset. Various things can come into play, Disk Quotas, AFS partitions, Compression, Encryption, Disk Spanning, ... On occasion, i might even try to download that iso to F:\programf\Dev\ and merely race the download against my efforts to make space. This is kind of what mozilla is doing now. mpt-5 was addressed by some of my earlier recommendations. which means they would be solved 'by moving the partial download once the user specifies a location'. mpt-4 doesn't make any sense. I don't see how it even relates to 'problems which would *not* be solved by moving the partial download once the user specifies a location'. i suspect in the case of a remote mounted device the os/driver would be caching the data and sending it out on a schedule satisfactory to it. mpt-3 already includes a bug reference. mpt-2 is a red herring. there's another bug which he is aware of to revise the transfer rate and estimated time remaining. mpt-1 ignores the fact that we have a bug for rendering pages before downloading. People seem to like the idea.
-> XP Apps. I've been watching this debate for some time. This feature is a mess, and this discussion is the worst of it. Can we make this pref controlled, and send the debate to the netlib newsgroup?
Component: Networking → XP Apps
Okay, first a disclaimer: I've read through part of this mess, but not all of it as most of it seems to be ancient history. Next: If you feel I should file a new bug with the information I'm about to provide, then let me know. I'm adding this here because it seems incredibly similar to me. I don't care how a download starts, where it starts, or what name the file gets when it starts, as long as when I select a location and file name, that's what it gets and where it goes from that point on. Using a mandatory default download location is bad for several reasons, the most prominent in my case is that there just might not be enough room on that disk for the download to finish! On to the current problem: Trying to download a file with a MIME type (in this case, application/x-compressed) and change the choice from 'open with <application>' to 'Save to disk'. The process of saving a file to disk has been whacked for the last two nightlies I've used. (FYI I'm using W95B, and use the w32 talkback zips.) In the previous version (dated 16-07-01 11:29, but I'm afraid that's as close as I can get to a build ID), it was simply a matter of the program ignoring any instructions for downloading. That is, the file continued to download to my TEMP directory with an obscure filename, rather than download to the drive/path/filename I chose. Now, in build 2001072003, an even weirder problem has cropped up... Not only does the file continue to try to download to my temp directory, but I keep getting multiple "What do you want to do with it" dialogues which each produce a 'saving file' progress report all downloading different random filenames to the same place. :-( I don't know how long this might continue, because after five times of giving the same instructions and having them ignored, I canceled them all and came here. :-/
I don't know what the plan is for this bug, but a few things to consider regarding using /tmp on Linux and once UNIXes. On Linux, /tmp is often on a seperate disk partition then /home. When downloading a large file (say, 500M), this means: a) there quick likely might not be enough room on whatever partition /tmp is on. The download will be killed because of this. b) we will have to COPY the file across disk when the download is done, since you can't do a simple move when /tmp is on a differant partition then /home. This causes a lot of wasted IO. I should note IE5 also suffers from the two above problems. It starts writing to the temp directory on c:\, but if I specified to save to d:, the problems above apply. I can't tell you how many large downloads I've lost because C: filled up. On Solaris (and on some Linux machine), /tmp is a memory file system. This means: c) downloading a 500M file USES FIVE-HUNDRED MEGS OF MEMORY! It will use swap, if it's available. I'm not sure if there is a fix for this problem, other then letting the user decide between the options we have. (start download immediatly, saving to $LOCATION in the mean-time, or stall download until I pick a place). Both have their problems. Whatever the fix is, /tmp needs to NOT be the default download location, though. The users home directory is probably a better choice.
Perhaps it would be clearer if the open/save dialog appeared as a child of the download progress dialog so that the user could see that the file was already downloading while deciding how to open/save it?
Wouldn't the simplest solution be to let the user _set_ a default download location (if one is really necessary)?
moving to 0.9.4
Target Milestone: mozilla0.9.3 → mozilla0.9.4
*** Bug 93838 has been marked as a duplicate of this bug. ***
*** Bug 93847 has been marked as a duplicate of this bug. ***
-> xp apps qa.
QA Contact: benc → sairuh
*** Bug 62666 has been marked as a duplicate of this bug. ***
*** Bug 95210 has been marked as a duplicate of this bug. ***
Here is some of the info from the recently duped bug. Some of it has been mentioned here before, but some of it is new. 1. The temp folder is usually on C:\ and for WinNT this is often limited to 2 GB, where much of the space is already used up by other programs that insist on using C: - I tried downloading a 150 MB file to my F: drive and after 4 hrs!!! the download failed because C: had become too full - I lost that data = dataloss!? :( 2. Partiallydownloaded files become lost in temp folder nirvana because most users never look there. They do however usually know and use the folder THEY have chosen as the destination folder. We should follow GetRight's lead in this, as they are very experienced designing good download managers. 3. Again, following GetRight's lead, if the partially downloaded file has the extension filename.ext.MOZ, then users (A) would know that the file was a download file that was not finished yet. (B) could doubleclick the file to resume the download; (C) the MOZ extension would have an accompanying icon that further identifies it as belonging tio Mozilla. All thiss is most useful if the partially downloaded file is where the user expects it to be, namely in the destination directory he chose. The main reason right now for me is "1." because I am really upset that the partially downloaded file location is one I have no control over (without changing my entire system configuration - which is often not possible in an office environment). Additionally, implementing this has the added bonus of making points 2 and 3 easier to implement in the (hopefully near) future. BTW. Shouldn't this be KW "dataloss", since it has caused dataloss and has wasted up to four hours of PC-time for me.
we'e not going to hold .9.4 for this.
Target Milestone: mozilla0.9.4 → mozilla0.9.6
Looking through this bug, it seems that somebody needs to specify clearly and definitively what should happen when a file is downloaded. There are many suggestions and good ideas contained in here, but apparently no conscensus as yet.
*** Bug 97725 has been marked as a duplicate of this bug. ***
In my opinion the default behavior for downloading files should work similiar to this: 1. After clicking a filelink, Mozilla starts downloading to a (configurable) temporary download directory (e.g.: /tmp). 2. The user eventually picks the final location and filename. 3. Mozilla should check, if it thinks there is enough room on the harddisk. If not, the following dialog should open: Warning on <filepath> may not be enough room for <filename>. This assumption may not be accurate, depending on your system configuration. Should I continue downloading, anyway? Continue Cancel 4. Depending whether the destination file already exists, the following scenarios are possible.: a.: The file does not exist: The temporary file should be moved to its final location. Downloaded data is from now on written to the final location. b.: The file already exists: A window showing the follwing text should open.: The file <filename> already exists. What should I do with existing file? Replace Resume Cancel Where 'Replace' should move the temporary file to the final location, overwriting the old file. 'Resume' should replace the old file with temporary file, if the temporary file is bigger than the old file. If the old file is bigger, mozilla should try to resume the download (see separate bug). 'Cancel' should cancel the download. In any case the temporary file should be deleted. The following (advanced) download preferences could exist: 1. Path to the temporary download directory. Default should be the platform specific temporary directory like /tmp or C:\temp or whatever. This could be useful for users with limited harddisk space in these directories. 2. Option to turn off data prefetch (background download) before the final location has been choosen. This will help people with fast internet connections and/or limited space in the temporary download directory. With this option turned on, temporary files will not be created for downloads. My opinion to some other points mentioned in this bug: 1. Mimetype dependent download directories. Not important for me, as I usually download all files to my home directory. Might confuse users (where has my download gone?) 2. Partially downloaded files Partially downloaded files should be saved at the user picked location and with the selected filename. The user will most likely search the file at the location he picked. Moreover the user will know, if the download failed, because he got an errormessage, right ? Maybe these errormessages could contain some way (button, checkbox) to delete the partially downloaded file. Keeping the correct filename makes life easier for resumeable downloads. 3. If showing a progress dialog with 100% completed, for small downloads is really a problem for some users, then do not open it at all, if the file is already completely downloaded. This might however confuse other users... One last word... This is how in my opinion a solution for this bug should look like. It is mostly in consensus with other suggestions on this bug. But the most important point is: Whichever the final solution for this bug is, fix it soon. Downloading is one of the key-functionalities of a browser and imho more important than another new skin. I can't understand, why an almost one year old CRITICAL bug with the keywords 4xp, nsbeta1, nsCatFood (I would also add dataloss (I already lost downloads due to this bug)) and 7 votes, has still no developer assigned...
mscott: are you working on this? or, do you mind if i take this one? i'd like to try to implement some portion of the implementation outlined by snfuchs@gmx.de snfuchs: i like most of your plan, but i think that implementing the Resume feature will be difficult, and should probably not be attempted on the first pass. likewise, the preference to disable the prefetching feature will be troublesome. IMO giving users a way to configure the temporary download directory is half the battle. then, moving the download file from the temporary location to the choosen location should provide a robust and IMO satisfactory solution. the point after all is to prevent dataloss and basically not annoy the user, and i think this solution would defintely help minimize user frustration.
Hmmm...this seems to have been fixed already. When I download anything it goes straight to the location I type in, even while it is downloading. (Linux 2001090408).
*** Bug 98964 has been marked as a duplicate of this bug. ***
IMO, this should just be scrapped. No data should be downloaded except to the location specified by the user. The arguments for this behavior: * It prevents disk-space and other drive-specific issues. * It avoids a cumbersome pause-resume process (which some servers don't even support). * It clears the way for an efficient halted-download storage system (similar to that of GetRight) - which saves to userpath/filename.ext.moz. * It cuts down on file system move operations which could cause messy disk fragmentation and slowdowns on older systems. The arguments against this behavior: * It delays downloading by the user's reaction time. The user can avoid this by choosing to automatically open files of that type (which puts them in the temp directory anyway). * It blocks the possibility for a complex auto-save-to system as I described before. It looks like nobody is willing to put forth the effort to create such a system anytime in this century (in which there are about 1,190 months left). * The partially downloaded file could be mistaken for a finished download, and the user might try to open it, messing up the download. This would be much less likely to happen with the .moz extension.
Blocks: 18004
why not add a pref for temp files. you could choose between temps going in a folder in your profile (what i would want if my profile was local), going in / tmp (what i would want in any *nix system i set up) or in window's temp directory, going in the download directory (which is what it is now), or waiting for the user to respond until starting. this may be a little more work (might not be), but it is the right thing to do(tm)--at least in my oppinion.
nnooiissee: The hard part is finding someone who's willing to do that. And again, that makes things much more complicated when we start a download resuming system.
skewer: why should that make it any harder? if the preferences define a location for temp files, then when prematurely ending a download mozilla askes if the temp file should remain to be resumed later. if the user has chosen not to save until a location is choosen, then just abort and leave the file. you are right, finding someone to do it is hard, but with this much debate i would not touch this bug with a ten foot pole. rather than someone sitting down and writing the code i think someone should sit down and write a spec something like what is found at http://www.mozilla.org/projects/ui/communicator/index.html and then we can review, debate, and finalize that. then finding someone to do it would probably be alot easier.
For the record, I'm with timeless on this. I *want* Mozilla to be downloading files while I waste time trying to decide where to stick the file. I pay for my connection on a per-minute basis, so the less time wasted downloading the better. Saving the file to either the temporary directory, the cache, or the default download directory, and once the file name is known renaming/moving the bit downloaded so far, seems the best idea to me. I think this bug should be renamed to "Spool file should be moved once the user picks a filename".
If the 10 seconds it takes to decide on a directory and filename are sooo important, then I agree with Ian that the spool file should *definetely be moved* to the user-selected location once the user has selected such a location.
How would we go about moving the file, though? We can't stop transferring from the server (particularly when the server doesn't support resume), and it would be very difficult to move a file while it's still downloading.
Everybody is talking about saving the file to a temp directory or something similar, but why not keep it in memory until a place is chosen? On windows this would go to the swap file if we run out of memory, and on unix to the swap disk. Then after a location is chosen we can dump whatever we have in memory to file and start appending to that.
Memory won't work, what about downloading a 500M file? You could write it to disk after X seconds, but then you're back to the drawing board where to put THAT file, and what to do with it. There's only two solutions here that I can see. 1) pref for spool directory. 2) pref to disable saving until location is chosen (this is how 4.7 works, but does it do a HEAD or GET first to check the MIME type?) Let's pick one and get it in. I don't even care which any more.
i agree with hixie. pausing downloading to move the spool file to the user specified location is not a problem. necko automatically handles intermittent delays on the consumer/UI side, and there would be little concern that the server would drop connections due to such a brief pause. modified summary to reflect this solution.
Summary: Saving a file creates a file before location is chosen → Saving a file creates a file before location is chosen; spool file should be moved once the user picks a filename
How long a delay is acceptable, though? I've seen computers that take half a minute just to move a 1MB file. After that much time a server might timeout. Pause/resume seems like the better choice in such a case, but there's still the problem that neither Mozilla nor a percentage of web servers supports it.
if a computer is that slow, then it is certainly better that we move the file as soon as possible (ie. while it is relatively small) than waiting until the file is completely downloaded. tcp/ip applications are expected to allow for latency/delays/etc... the internet is not always fast, right? so, i don't think 30 second delays (however likely) are going to be a problem. now, if it were several minutes worth of delay, then that might be a problem. overall, i think moving the file early will give a much better user experience, especially on computers with slow harddrives.
If we're going to move mid-transfer, then I propose the spool directory default to $PROFILE/spool or $PROFILE/tmp. On UNIX, the profile is much much much more likely to be on the same disk as the chosen download location (profile is in $HOME, users very rarely save outside of $HOME). On Windows, it's not any less likely to be on the same disk, and probably slightly more ("Documents and Settings" generally placed on a disk with plentiful free space). Mac filesystems are alien to me, but it probably can do under $PROFILE there too.
Jeremy, saving the temp file inside the home directory will do terrible horrible things to people who have disk quotas (fairly common on Unix systems). It may make it very difficult to download a file that's bigger than remaining quota. /tmp or better yet $TMP/$TEMP (as discussed eslewhere) should be the preferred default, imo.
And saving to /tmp does "terrible horrible things" on many systems where /tmp is a memory resident file system. If it takes you a few minutes to pick where to save it (you walk away from the keyboard for a minute or two), and you're on a fast connection, you could easily tie up 50-100M of memory. If a user has a disk quota, they won't be downloading a file larger then it anyway, since they couldn't do much with it. It should probably be a hidden pref. The default is debateable.
> If a user has a disk quota, they won't be downloading a file larger then it > anyway, since they couldn't do much with it. You've obviously never tried downloading and using the latest Moz nightly for a single login session while having a 15MB quota.... Or downloading and printing a homework assignment that's image-heavy (on the same quota). :) > where /tmp is a memory resident file system. Aren't you asking for trouble from all the apps that actually use /tmp for temporary files (eg mozilla for files it feeds to helpers). The point is that I feel that we would be best served by making the location a non-hidden pref. At the very least, the pref should be well-documented.
Obviously you don't have 19 partitions, each with only a limited amount of space, either. If you ask me, there should be a very obvious and unhidden option (perhaps even on or along with the first download dialog, with a 'don't ask me this again' box) to not bother downloading at all until I tell you where to put the thing. Also a 'more info' button that will explain what is happening and the pros and cons of each choice (a link to the relevant help section would solve this). Just my $0.02
My parents use Mozilla and they're confused enough without being prompted about where they want their spool directory. Lets let the general public figure out what "You are now entering a secure site" means before we prompt for more.
You make this so complicated. Here's the scoop: you pop up a dialog box. The user chooses a directory and filename. Then you start the download to that directory and filename. Done. If you can check before starting that the file will fit where the user selected, and it won't, warn the user. Anything else is too much. I say this as a user, not a programmer. Do you ever want to finish this thing or not?
bob: This isn't about running out of space where we PICK to save (I assume we *do* check for that), it's about if, how, and where the file is stored during the before, during, and after the dialog. Since we've been all over the map here, let's try a proposal: --- Defaults stay as is. Add a check to the code to verify there is enough space on both the spool and chosen location, if it's not there already. Add a download spool directory pref under Advanced->Cache: [V] Begin downloads before I choose a location to save to Spool in: [ C:\WINDOWS\TEMP ] Whether or not we move the file from the spool to the chosen location as soon as it's chosen doesn't matter very much, as long as those prefs are available. If someone REALLY feels like coding it though, it couldn't hurt. We'd have better performance then IE (for a change) with that, which would be nice. --- OK, any complaints? No? Good. Code away.
jeremy: you missed my pint. My point is: you don't put the file ANYWHERE until the user tells you where. After some years as a software user, I've come up with 2 rules that I expect out of a good user interface: 1) Do what I tell you to do. 2) Don't do stuff I didn't tell you to do. I'm sure there are more, these are just the 2 which, when violated, aggravate me the most. When I first used the download as (previously) implemented, it gave me 2 problems: b) I thought it wasn't working at all because I couldn't see the file where I had asked it to be. Rule 1 violoated. a) It put a file that would grow to 660M (yes, it was a Mandrake iso) into a filesystem with only 120M. Rule 2 violated. If, instead of trying to save a trivial amount of download time, you had just waited until the user selected a directory/filename and put the file where the user told you to put it, you could have saved yourselves this whole bug. The time squashing this silly bug could better have been used. Of course, 0.9.4 seems to do the right thing, so I'm not sure why we're still disucssing this, unless your'e still creating this spool file, then moving it when the user selects a name. If you're doing that, you're still violating the 'rules' I gave above, and sooner or later another grouchy user like me will be opening a bug on it after taking too long to select a name and filling a filesystem. Anyhow, thanks. mozilla looks much better at each milestone.
bob: there is no way to implement what you want... consider dynamically generated content from a CGI script for which there is no built in viewer. we would have to save the document, and we could not refetch it (because perhaps refetching would mean repeating some credit card transaction, etc). the solution to move the file to the user specified location ASAP is the best IMO, because it almost completely addresses both of the rules you outlined. 1) you will notice the file "growing" at the location you specified. 2) you will not have to worry about it "growing" at an inappropriate location, since we will have moved it to the specified location before the file gets to large. <-- this is not exact, and that's why i say that this solution "almost" completely addresses both of the rules you outlined.
darin: OK, I see what you mean. I'm not used to seeing a CGI script that generates content needing a viewer, but I'll take your word that it happens. I agree that for any reasonably quick selection of a filename, your approach nearly satisfies my "rules". (Though seeing the modem lights continuously flicker prior to selecting a filename will still puzzle some useres...) But I still, just out of curiosity, wonder: what happens in this situation for NS4.whatever? Does it just hold the session up until either the user chooses a filename or TCP times it out, or what? (posted w/ 0.9.4 on Linux, which gets used for most of my browsing now.)
I don't see how the save-before method is better. Either 1. After receiving the unknown content-type header, you hold off the saving of data (which involves a slight timeout risk, but perhaps less of one at this point) until the save-to location is picked by the user. or 2. Automatically start saving to a location which might not have enough room for the whole file (which involves a risk, and might merit hours of extra coding to add a custom spool directory and disk space checks), then HANG THE DOWNLOAD (with a shorter, but still present, risk of timeout during filesystem operations) while the file is moved. This special networking behavior be more complicated to code. With 1 the risk is an initial timeout. With 2 the risk is a mid-download timeout and conflicts with drive space, as well as the need for much more coding. As for paid transactions, if the download times out before any data is transferred there might be fewer problems than if it times out after some data is transferred. I just don't see any reason to do anything before the user chooses a location to save to.
we have a problem with HTTPS in which the user is required to verify a dialog before continuing to download a secure page. if the user does not respond to this dialog within 1 minute, the connection will be dropped. for file->saveAs, i really don't think we want to mess with more of these types of problems. so, while normal web servers may be more forgiving in the amount of time they'll pause, HTTPS servers are usually much more strict (probably because of the cost of a HTTPS connection).
skewer: the required code will not be as much as you think.
Blocks: 103766
spam: over to File Handling. i have not changed the assigned developer [or the other fields for that matter], so if anyone realizes that a bug should have a more appropriate owner, go ahead and change it. :)
Component: XP Apps → File Handling
*** Bug 103856 has been marked as a duplicate of this bug. ***
-> me to try to develop a solution.
Assignee: mscott → darin
Blocks: 104166
Target Milestone: mozilla0.9.6 → mozilla0.9.7
Blocks: 69938
*** Bug 106619 has been marked as a duplicate of this bug. ***
*** Bug 107088 has been marked as a duplicate of this bug. ***
*** Bug 107412 has been marked as a duplicate of this bug. ***
Blocks: 88566
the solution to this bug likely depends on the implementation of the download manager. that is, i would imagine the download manager will make some of the existing "file save as" code obsolete. anyone know who's working on the download manager?
Severity: critical → major
Target Milestone: mozilla0.9.7 → mozilla0.9.8
ben is working on the download manager.
*** Bug 109105 has been marked as a duplicate of this bug. ***
This is a very interesting issue. On the one hand, there is potential for seemingly blazing download speeds. There is also the potential for incomprehensible behavior, unnecessary complexity, additional defects, extra work for developers, and even performance degradation in some scenarios. Current users have learned to expect to have to specify a location in a file system hierarchy to save files, but really, most people could not care less where they are saved as long as they could be retrieved quickly and easily. Few AOL users have any idea where their downloaded files are saved, and why should they? One way around this problem would be to use a default download directory rather than having to specify a location every time, which I'm sure for many people is the same location nearly all the time. Some operating systems (MacOS) have this already, so our asking where to save the file to is a needless interrogation of the user. A better way would be to hide the file system from the user entirely, and provide much more sophisticated means for retrieving their documents. That probably should be a separate bug report tho. ;-)
126 comments and we've gotten nowhere. Solution Proposal: 2 modes, temporarily named auto-download and interactive-download. i-download is the default. If mac IE actually saves files automatically as trudelle alluded too, mac defaults to a-download. New pref pane ("Downloading", under Navigator?), looks something like this: (o) Interactive download: You will be prompted where to save every time you download a file. ( ) Automatic download: Downloads start automatically and are saved to the location chosen below. This can save time by getting downloads underway as soon as possible, but may end up saving files to drives that are full. Automatic download location [ /home/jmd ] I take it there are cases when downloading where we don't have a content length? Maybe for FTP or gopher... When we do have it, and a-download is selected, we should check for space, and if there isn't enough room, a dialog should pop up explaining this, and present a file chooser to manually pick a location. I suggest i-download as the default because most users expect this (4xp and IEp). If moz just starting tossing files to $HOME, users might wonder where it went. I plan to use a-download myself though, so am not against it much. Of course, with i-download, we DO NOT save to /tmp or whereever before a location is chosen. That's the whole point of the pref. Right now we do something inbetween a/i-download, which ends up being what no one really wants. I think (er, hope) this above proposal would satisfy everyones needs.
I agree with Dolan, provided interactive download does *NOT* begin transmission of the file until I have chosen the location; not to memory, not to the disk, not anywhere. Of course it should still wait for a content-type and (if available) content-length header from the server. It would be great for both of these options to integrate with download manager/resume when those come around. As requested before (here and in other bugs), the file should be saved as-is to the disk as it is downloaded. That's another bug, but whoever writes a patch should keep this in mind. If done correctly such a preference won't threaten stability or performance.
"A better way would be to hide the file system from the user entirely, and provide much more sophisticated means for retrieving their documents." You say your name is Bill Gates, eh? <ducks> Please don't.
"A better way would be to hide the file system from the user entirely, and provide much more sophisticated means for retrieving their documents." Since you don't know what application would be used for retrieving the documents, how do you propose to do this? A plea for comprehensibility and simplicity: the user chooses a path and filename. The file is then saved to to that path and filename. If the user can't deal with this, they probably shouldn't be using a computer. It's been pointed out to me that there may be a small number of situations where dynamic script out put is involved and pokey users may get timed out, and resubmitting the request is not valid - I suspect that this will be rare.
It will have to be supported in the OS. Anyone who thinks most users really want to deal with a hierarchical file system, and understand how documents can exist there, and in memory, and in various memory/disk caches, is simply deluding themselves. Until OS document interfaces evolve in this way, I agree with Bob that this should be kept as simple as possible, and the edge cases are inconsequential. Speeding downloads by a second or two is not worth the added complexity to the UI, nor the usual regressions. Adding modes in which downloads could lead to unexpected errors would be wrong. Unless someone does the extensive work required to make this function seamlessly from the users point of view (which I'm not sure is even possible), or it shouldn't be done at all.
These bugs used to go to networking, so this is still a sensitive point of pain for me. I think I could put down a coherent design in one or two pages, but the question is, if I do, will someone consider changes for 1.0? If so, I'll take the time for this. Peter?
I'm not sure what you're asking, but anything proposed for 1.0 should be considered, and performance, correctness and usability are part of the 1.0 roadmap.
Perhaps the solution is to start downloading it to the cache, then when the user selects a location, a new thread is spawned. The existing thread calls open(newfile, O_WRONLY|O_CREAT|O_TRUNC) The new thread calls open(cachefile, O_RDONLY). The new thread calls open(newfile, O_WRONLY) The existing thread calls lseek(newfile, tell(cachefile)) The existing thread continues to write the download stream to newfile, starting at the offset to which it seeked. The new thread copies the data from cachefile to newfile The new thread calls unlink(cachefile)
I don't recall seeing this among these comments, but I don't have time to re-read them all atm.. Am I the only one seeing this behaviour? Not only are the files being saved to cryptically-named temp files, but self-installing .exe files are being saved as .tgz files, confusing the h*ll out of WinZip (which shouldn't be called anyway).
trudelle> Speeding downloads by a second or two is not worth the added trudelle> complexity to the UI, nor the usual regressions. With regards to my ('the'?) proposal, I mentioned I would be enabling a-download myself. Not so much so get downloads finished 2 seconds quicker, but simply because I always always always save every file to $HOME. I've never once not. If I need it elsewhere (rare), I use mv(1). It's an annoyance issue of clicking 'save' after I already shift-click a link to save it, more then a perf one. Others, however, (BT internet users, etc) do want it to begin automatically to save download time and money. That said, if THAT isn't deemed 1.0 worthy, the half-assed attempt that is the current default desperatly needs to be removed. This also fixes most of the blockers as well.
You consider clicking 'Save' to be an annoyance, but it is currently the point at which users commit to saving the file. I still think the right solution is to add a way to save a file to a pre-specified location, without any dialog.
No, there should be a *definable* default download directory which is presented when selecting "save link". Then one can either simply hit ENTER to confirm that directory (and filename), or select another directory (it does happen). This is the ideal combination of speed and flexibility. And don't tell me hitting RETURN to confirm the default directory is a hastle ;) PS. Anyone know how I can change the current (nonsensical) "default" directory?
Presenting any alert or dialog is a hassle, because it breaks the users flow, can cause confusion, and does require action. Requiring any such action of users unnecessarily is very bad, we have no reason to continually interrogate them. In this case it could easily be prevented. If the download location is defined, then why should they have to confirm it every time they save a file? The whole idea of predefining it is so that stuff just gets saved there. Note that it could still be possible to get the dialog, if you really wanted it every time.
Oh please, EVERY application on this planet (e.g. WordPerfect) *asks* where the user wants to store a *new* file - and a downloaded file is a new file! Most applications have a default directory - but they still ask! Why break this useful convention, and why confuse novice users ("I clicked the download link, but where is the file?"). Users might "accidentally" click a download link (e.g., thinking its a hyperlink), so they will need a confirmation dialogue anyhow, asking them *if* they want to download the file. This dialogue should at *least* offer to store to another directory. I can't believe there is a need to discuss something so obvious.
"Presenting any alert or dialog is a hassle, because it breaks the users flow, can cause confusion, and does require action. Requiring any such action of users unnecessarily is very bad, we have no reason to continually interrogate them." Sheesh! Is M$ taking over development of Mozilla??? Don't you understand that if I wanted that kind of behaviour I'd just install IE? I don't -like- programs that just 'do' without any intervention or direction from -me-. That's why I use non-M$ products. Allowing the user to -define- a default directory if that's what they -choose- is fine. But I save to -many- different directories. I have a whole drive set aside for downloads, and that drive contains 102 directories, plus I often save to 19 other -drive-partitions- (on -so far, soon to change- 3 physical drives) depending on the situation. Obviously I don't want to be confined to one location. I don't pay any extra for bandwidth or telephone calls so I don't even want the thing to start until I tell it to what and to where. I have always been able to do so. In any other programs I run I have this choice. Name a program that doesn't provide a 'save as' option? Why is this asking for a filename and location suddenly -not- the intuitive thing to expect? Another thing: Saving to some obscure -filename- doesn't make any sense at all to me.. especially when even the -extension- is different. Saving a self-installing executable file to a .tgz file??? Why?
*** Bug 112175 has been marked as a duplicate of this bug. ***
Peter L., Lythande: Please calm down. This bug is for an OPTIONAL feature that many people want, to download quicker and not be annoyed by a dialog they NEVER use or want. It WILL NOT be on by default, that would of course be very unexpected to a new user. It is an automatic download/background download mode selectable in prefs and eventually in download manager. Lythande: Your tgz/exe issue is a seperate one, please stop discussing it here. Quick-saved files will be named as well as possible, not like the current temp files. Darin: any feelings on my proposal? (comment 127)
IMO, confirmation dialogs are, in general, an insulting waste of time; it is much better to allow some form of undo. In this case, users will get a progress dialog, which will offer them the chance to cancel the transaction (on the rare occasion they make a mistake), and offer to 'reveal' the downloaded file in the OS file system. I really don't much care what WordPerfect does with new documents, it has little bearing on people downloading files using a browswer. I'm not advocating getting rid of save-as, just allowing for a one-click download for what seems to be the typical case for the vast majority of people who don't even know what a partition is, let alone have 19 of them for downloading. I'm also not very interested in what default behavior anyone in this bug personally prefers (including myself), since there is little or no support for building a browser geared primarily for a tiny number of technophiles. If what we are building appeals mainly to us, rather than being adopted widely, then we will fail.
[OT] Peter trudelle, please stop spamming this bug. Comment #143 already explained that what you want (non-standard, nonintuitive filesaving) will be an *optional* feature. So go rejoice with the other 2 people that want this "feature". ;) [/OT]
I'm with trudelle. After all, he is manager of the browser team, with years of software development experience.
I am for the proposal in #143. Being a technophile myself, my preferences does not count :-), but I've seen how "normal" people use this feature (mainly family and friends). I've seen people download the same thing several times because they didn't get a save dialog. I've seen people searching all their hard disks in search for that file they know they downloaded (they saw the progress window), but where placed in some strange "default" place. I've yet to see anybody using Windows put their stuff in My Documents, they put things in folders, like "Work", "School", "The car", e.t.c. Granted, the population studied is not statistically significant, but then again, who has studies this issue at such length :-)? Another thing in putting up a save dialog as the default is because this is how Netscape 4 behaves.
*** Bug 112562 has been marked as a duplicate of this bug. ***
There's a possible security hole related to this bug. If a web page contains e.g. a 1x1 iframe whose source is a virus, Mozilla downloads the virus to a temporary file while prompting you were you want the virus saved...
You know, you're funny people. I'm on this list because I reported a bug that mozilla saves temporary files in a world accessible directory with world readable permissions, which blasts the whole idea of privacy. I was told that my bug belongs here. And what I see is endless discussions about usability vs. awareness and target milestone is 0.9.8. And during all that time mozilla will continue to save temporary files with world readable permissions. Very funny. I realize that there is a problem of cryptic filenames and user interaction and all that, BUT WILL SOMEBODY WHO KNOWS THE CODE ADD A SINGLE LINE TO CHANGE PERMISSIONS PLEASEEEEEE!!!! And then keep on discussing secondary matters. I have a machine here which is used by 20+ users as an application server and EVERY file to be opened by Acrobat, GV, mpg123 is saved in /tmp with world readable permissions and is kept there until mozilla quits. And if it crashes, they stay there for ages. But the biggest and at the same time easiest problem to solve is WORLD-READABLE permissions. Why does it have to wait for 0.9.8? I understand that it can be viewed as a small part of a bigger problem, but why wait for the bigger problem when it can be solved now? permissions have nothing to do with user interaction. Completely independent. Thank you Alex
+cc, is this a security issue?
Blocks: 109594
we currently only save the toplevel document... not the subframes, so i don't think there is a security hole as neil describes. alex: the bug you speak of is definitely more severe then this one... it should not be a dupe of this bug.
*** Bug 113368 has been marked as a duplicate of this bug. ***
Did this mess all start in bug 22796 where I said, "...To answer the question of where to download to - a few hundred K (to a Meg) of memory could be allocated for the download, and if it got too large, then it would be thrown in a temp file. After you clicked ok - it would transfer the data to the other directory and continue the download."? Not copying till the end is not at all what I intended. Many people have suggested copying the file right after they choose the location. That IS what I intended. >Um silly question... is it possible for a download session to change where it >writes while it's downloading? >eg. we d/l 12% to a temp file. the user tells us where to write (and it's on >another drive), we copy that 12% to the new file. [2% builds up], we copy that >2% and then update the file writer to start writing to the correct file >(actually, assuming we have binary seek we could probably start writing the >incoming data immediately where it belongs and then copy over the start buffer >when possible). when we're done, we kill the start buffer. >otoh, we could just ask the os to buffer the entire file in memory until the >user picks a location (wouldn't this make more sense on most operating systems >w/ decent vm?) Good idea. Couldn't we take advantage of a move for the first 12% if the copy-to and copy-from are on the same disk? The bytes downloaded during the move could be to a different tmp file. For instance file.moz.tmp.1 and file.moz.tmp.2, etc. As for the problem of Linux users having a small tmp directory, Mac's problem of showing files that hurt the aesthetic appearance of the desktop, and Solaris's problem of using RAM for the tmp, being able to set the location is a good idea. The default directory could be a subdirectory of Mozilla, but the user could change it to any directory they choose. I also agree with making a pref available. I don't know what the solution for timeless's "File system returns wrong size error", but I think that if the temp directory doesn't have enough room, it should wait until the user selects their directory. We should accept whatever the operating system says is available for the partition the directory is on. If this is wrong, and the disk fills before the user selects a location, then a message should be returned: "The operating system is returning the wrong available space for disk ____, or the disk size changed while downloading the file.". An example for the second part: If the user does two downloads in a row and the second download makes the disk size change for the first download. Should we also mention that they could try turning off the pref until more space is available? The maximum disk size needed would never be more than the file size. Even if the person clicked OK on the save as dialog when the file was 90% complete and both the temp location and save as location were on the same disk, the first 90% of the file could be moved. This would alleviate needing to have double the file size available on the disk as IE requires. As for the save-to location being on a network disk in the remote (but highly unlikely) chance that downloading is faster than copying to a computer on the network, then the whole tmp file might not get copied over by the end of download. In that case, copying the rest of the file might be necessary. This is not really a big deal.
After experiencing this 'optimization' for a while, I'm convinced it is just plain wrong. It downloads files before I have authorized, to a hidden location I haven't authorized, and with a name that is cryptic, so I couldn't find them even if I was looking for them. It doesn't show me a download progress dialog until after I've accepted the file save dialog, so I have no way of knowing it is sucking up bandwidth, even if I'm paying for it by the byte. It requires that I have disk space of at least twice the file size, for the copy, and takes up extra disk and processor time to make the copy after download. It complicates the code too, leading to errors such as leaving large, hidden, cryptically named files when the destination I've picked doesn't have enough room. In this last case, it doesn't even tell me, but falsely shows the download as completing, but the file never appears in my chosen destination. This behavior is just a collection of defects and potential defects, and should not be allowed in the product.
Most of Trudell's objections would be addressed if there was a user selectable default download directory, and if Mozilla used the filename specified by the webpage until the user selected a filename. Then Mozilla would immediately replace the webpage filename with the user given filename. This way: 1. files only go where the user defines and can find them. 2. broken downloads are still identifiable by their filename (albeit the name given by the web page in the short timeperiod where the user has not yet assigned a name). Also, GetRight does this excellently by giving the file an extra extension until the download is complete (e.g., RTCWolfenstein.exe.Mozilla) A simple search for "*.Mozilla" will reveal all unfinished/broken downloads. :) Also, most people have much more HD space free than they would ever download from the web. Even with DSL 100 MB will take over an hour, and everybody has at least 200 MB free - at VERY least. This is a cool feature because most downloaded files are in the 50k to 500k range and these are nearly completely downloaed by the time the user has selected a directory and filename. Can't this be worked out with the above suggestions?
[SLIGHTLY OT] And how about a dropdown list in the "Save As..." dialogues with quick access to the 10 last used download directories? [/SLIGHTLY OT]
I just filed a bug on my last idea (dropdown for 10 last used directories) - bug 115574
Blocks: 75364
There still seems to be a persistent misconception about save file dialogs, so I'll repeat myself: They are not just for selecting where the download goes, they are the point at which the user commits to the download. Programs have no business downloading files before the user clicks 'Save' (unless there is a default download directory that obviates the dialog). This is fundamental to the user model; failing to honor it is a breach of trust.
Amen! I thoroughly agree with trudelle's comments.
Hasn't the user *committed to the download* the moment he clicks on the download link? Also, Mozilla (and any browser) "download" all kinds of files from webpages and put them on the user's hard drive into the cache directory without the user consenting to it explicitly. I see little harm done by potentially few broken downloads landing in a user known place (default DL director) - but a greater benefit on the other side of the coin.
trudelle: what about when the file is already being downloaded? eg. click on a link... and that link turns out to be of a content-type that the browser does not know how to display? we cannot suspend the download indefinitely. nor, can we buffer the download indefinitely in memory. we must put the downloaded data on disk *somewhere*. the question then is: where? it seems to me that you are suggesting that we should define a default download directory (make it really easy to change from the preference dialog) and just put the file there. ok, but what if the user doesn't want the file to go to the default download directory? what are their options? perhaps the first dialog that comes up (in which the user can choose a handler app or a save-to location) should automatically come up displaying the default download directory. perhaps then the user could click a "select download location" button to alter the save as location for this file. then if the user presses ok and has save-to-disk selected, the file will simply be downloaded to the location specified in the dialog. under the hood, we'd have to be able to move the partial downloaded file from the default download directory to the new download directory. there could also be an option to make this new download directory the default download directory.
>we cannot suspend the download indefinitely. nor, can we buffer >the download indefinitely in memory. why not? that's what every other browser does, right?
It should be able to suspend until TCP/IP disconnects due to timeout, no? That is what, several minutes? As a user, I would not be surprised if there was a "connection lost"-message if I waited too long to commit the download.
When a user clicks on a link, she expects the browser to load and display the content. If the browser is unable to do that, it needs her permission to download the file. For all we know, she has no other program to view that content with, so downloading it may be an utter waste of time, disk space and possibly money. We should not try to discuss the design of a default download directory in this bug report, or any other. How about the UI newsgroup, where everyone can see it and add their ideas?
looks like 4x reads from the netwerk 1 byte at a time at a rate that diminishes exponentially or something like that. eventually, it seems to fall into a pattern of reading 1 byte every 30 seconds or so. once the user selects where they want the file to go, the download continues at full speed. very interesting... i'll try to see what IE does next. personally, i much prefer the way mozilla downloads files over 4x (with the exception of the fact that i can't control the temporary download directory). i like the fact that mozilla minimizes the amount of time i have to wait for the file to be downloaded. i don't understand why it is taboo to download in the background.
>i don't understand why it is taboo to download in the >background. Many users think that the download starts after you seleced the destination. If the user clicks on a link (he thinks it's a link to another site or a small File) and it's a 650MB file (Iso-Image) and the user leaves the PC after he clicked, Mozilla downloads the 650MB file in the background. This is a problem If you have a transfer limit/month or you must pay each MB. (many DSL/Kabel user have a transfer limit) FYI: I like the background download feature...
ok, i just tested IE6 and it appears to behave identically to NS4x. looks like it has a much steeper exponential falloff though. would anyone object to making mozilla behave identically to NS4x/IE6 as a first cut improvement to the current situation? eventually, i would love to see an option to always download in the background, but i guess i can be convinced that it shouldn't be the default mode.
nope. no objection to behaving exactly like IE/4.x here. ;)
No objections here, either. Let's just get this moving so that the original point of this issue (not to create a file where a user might not want it) is resolved first. THEN we can worry about background downloads and such.
Are we talking about a "Preference" here? If yes, then go ahead to implement both ideas and the UI to set the preference. I really can't see why it takes so long to discuss this.
No, we are not talking about a preference, we are talking about correct behavior versus incorrect behavior, and we don't want it wrong even if a few contributors would personally prefer it that way. It took a long time to discuss because no UI design was done up front, and there should have been UE involvement early on.
The term "Correct" or "Incorrect" is subjective.. If I "prefer" Mozilla to download file to a predefined directory without asking me again and again, pop-up a directory selection dialog is "incorrect" in my opinion.
RE comment 169 and comment 170: Yeah, let's take the easy way out and "fix" the first part of this bug ("Saving a file creates a file before location is chosen") by breaking what is a really great feature; and not fix the second part of this bug ("spool file should be moved once the user picks a filename"). I'ts just much easier that way - NOT. RE comment 168: > would anyone object to making mozilla behave identically to NS4x/IE6 as a first cut improvement to the current situation? I object. I definetely do not think it would be an improvement to the current situation; rather, a very unfortunate degradation. The only real two disadvantages we have now are: (1) That in the rare case that a download fails, we don't know where the file is (most people have learned to delete their TEMP folder periodically) and (2) that some #%&§ might click a download link, see the "Save As" dialogue and walk away from his computer for a long time (how dumb is that?) (That *rare* person is only going to do that only *once* anyhow). I just don't see the big reason for taking away this cool feature. Most users are going to have the "WOW, Mozilla downloads faaaast" experience. Why take that away? Suggestion: Make this bug dependent on the "pref for default download directory" bug (bug#?). RE comment 173: How hard is it really to strech our your right thumb and tap the numeric ENTER key? I can see this as a personal preference (definetely *not* mine), but certeinly not "incorrect".
comment 173: adding a predefined download directory to the UI would be a solution. comment 174: You are ignoring or failing to understand why the current behavior is wrong. You might personally prefer the behavior of any number of defects, but that doesn't mean we should leave them in the product.
[OT] RE comment 175: Peter T., how can you say I fail to understand the problem, when in the next sentence you say: > PT: "adding a predefined download directory to the UI would be a solution." and in my comment I had already concluded: > PL: "Suggestion: Make this bug dependent on the "pref for default download directory" bug (bug#?)." Where is the big difference between our two conclusions? I just want to find a solution *other* than giving up on pre-downloading. PS. I get the impression that *maybe* you guys are arguing with "Linux logic" (are you?) and I am thinking with "Windows logic" (I am, although I want to learn about Linux). If that is true, then please consider that there are far more windows users, and they should be given consideration too. If this assumption is incorrect, then just forget this "PS" section. [/OT]
I personally like this feature. It means I can go to grab a bite to eat, come back and then choose where to save the file and it is already downloaded. If you want to do a quick fix (temporarily), then feel free - but bug 22796 will have to be reopened. I think the proper method for this is having a pref where the user chooses whether they want to invoke this feature (automatically off). Have the default download directory be mozilla/download or something. Make the file filename.ext.moz.tmp or something like that, and make sure it cleans up broken downloads. To remove a feature permenantly because it is against some people's perception of how to download a file is incorrect imho. Step 1 (Quick fix): 1) Use the IE/4.x curve. 2) Use filename.ext.moz.tmp 3) Reopen bug 22796 4) Make it so that when you choose the location, it copies the file over immediately (which should be rather small because we are using that IE/4.x curve Step 2 (Re-implement correctly bug 22796): 1) Make a default tmp directory mozilla/tmpdownload or something 2) Make it so that number 4 of previous turns into timeless's idea of copying at the same time the file downloads. 3) Add UI pref for feature (default off) 4) Add UI pref for default download tmp dir Note: Don't have a default download directory. It should be chosen every time imho. It should also be what the user last chose as is the current behavior. Don't ask what the default temporary download directory is each time. It should be in prefs.
I have reopened bug 22796 to have this entire 'feature' removed as soon as possible. We can then consider whether any further work is appropriate to address the issues this was intended to resolve. In the future, please keep in mind that any changes to the UI should involve designers and UI stakeholders up front, in order to avoid such a waste of time and energy.
Whoops, looks like I have acted too hastily. Apparently, this was not just a simple checkin by one of my engineers, as the fixed resolution on bug 22796 led me to believe. Obviously, I need to understand this better, including exactly which scenarios it was intended to improve, and how much the implementation issues can be resolved, before we can decide on how to resolve the design problems it caused. I retract my implication that this was all a waste of time, and apologize to those who worked on it to improve performance, this may yet be salvageable.
*** Bug 116389 has been marked as a duplicate of this bug. ***
Just filed 116753 (downloading a file to a directory w/o permission silently fails), which would also be fixed (along with probably 30+ other bugs) by ripping this code out.
Blocks: 22796
Blocks: 65770
Sigh, sorry to keep beating on this bug, but it really actually makes my production use difficult at work. Let me explain: I do some of my work on a Sparc/Solaris 7 development box with only(!) 1024M RAM, which is usually full of servlet and database engines. It's on a 100Mbit connection to the campus ethernet. Somewhat frequently (maybe twice a month) I need to download ISO's -- 650M files. However, unless I'm *very* quick in picking a save location, my /tmp (mapping to main memory now) gets all filled up and my machine grinds to a halt as processes swap around. In other words, clicking on a link can effectively halt my machine (and it'd be less fun if I had a piddling 512M ram). My solution is to do this work using IE4 for Solaris. I've gathered, following this discussion, that it's debatable whether Moz should have the ability to download while picking a download location. It's not debatable, however, that in some cases this behavior causes serious problems. Perhaps this should be a pref, if you assume that Solaris users who download big files will know what's up. I doubt that this is a safe assumption, though. ;-)
RE comment #182 (downloading ISO's on Spark's): This could be easily solved if the partial file went into the *default download directory* or the *last used directory* (configuable in prefs) and *not into RAM* , then the user would select a default download directory (or last used) that had enough space.
-> 0.9.9 unfortunately :-(
Target Milestone: mozilla0.9.8 → mozilla0.9.9
After checking on this behavior a bit more, it seems we did it for parity with IE5, but have since received mostly complaints about it from customers and users, with comments such as those supporting it in this bug being the exception. I also found that IE6 has dropped the behavior, presumably due to similar complaints. IE6 still downloads 2-4KB of the file immediately, which I think yields most of the benefit for dialup users in the typical case where they quickly dismiss the Save file dialog.
Maybe the user complaits result from this feature not having been advertized and because the suggested visual feedback has not yet been implemented. The bottom part of the ASCII art below shows how this visual feedback could look: Suggested UI: +- Enter name of file to save to... --------------------+ | ____________________ | | Recently used directories: |____________________|\/| | <-- bug 115574 :) | ----------------------------------------------------- | | __________________ | | Save in: |_MyDownloads______|\/| [^] [Favorites] | <-- bug 115981 :) | +---------------------------------------------------+ | | | files (or favorites) listed here | | | | | | | | | | | | | | | | | | | +---------------------------------------------------+ | | _______________________________ | | Predownloading File ||||||||____37_%________________| | <-- this bug :) | | | The predownloaded file will be moved to the path and | | filename you select once you click SAVE, or deleted | | if you click CANCEL. | | | | [ SAVE ] [CANCEL ] | +-------------------------------------------------------+ I think once users realize what is going on (and the above UI will do that), then this will be recognized as an awesome Mozilla feature.
plairo: 2 points 1. that dialog doesn't have a place for me to pick a filename !! 2. for windows we should be using the places bar which includes the recent folder. (hint: one of the reasons ms did this was that the dialog would otherwise get unnaturally tall by default, just as yours is even without critical element #1)
RE timeless' comment #187: 1. (no filename selector): oops, how about this: +- Enter name of file to save to... --------------------+ | ____________________ | | Recently used directories: |____________________|\/| | <-- bug 115574 :) | ----------------------------------------------------- | | __________________ | | Save in: |_MyDownloads______|\/| [^] [Favorites] | <-- bug 115981 :) | +---------------------------------------------------+ | | | files (or favorites) listed here | | | | | | | | | | | | | | | | | | | +---------------------------------------------------+ | | File name: [________________________] [ SAVE ] | <-- fileNAME selector | Save as type: [_____________________|\/| [CANCEL ] | <-- fileTYPE selector | | | _______________________________ | | Predownloading File ||||||||____37_%________________| | <-- bug 22796 :) | | | The predownloaded file will be moved to the path and | | filename you select once you click SAVE, or deleted | | if you click CANCEL. | +-------------------------------------------------------+ 2a. (places bar, tallness): The favorites button costs ZERO vertical space, so that cannot be the reason MS added the places bar. BTW, many people have/will not use WinXP (older versions don't have the places bar). If we limit ouselves to the places bar, the (very) many win9x users will be left out in the cold. Also, with current common resolutions of 800 and 1024, the current design is not too tall, IMO.
In addition to being prohibitively complex for typical users, this dialog has the same fundamental usability problem, in that it purports to give the user control over whether the file is downloaded, yet it downloads the file anyway, to some unknown filename and location without any input from the user at all, potentially costing her bandwidth, disk space, and time spent trying to figure out why she has no space on her temporary volume for files that she *does* want to save. In bug 22796 comment #30, I have outlined a possible solution involving a default download directory, and addition of functionality similar to this to the download manager UI. I suggest we stop trying to subvert the normal file save dialog with this "pre-download" concept, and explore such alternate means that can be done by typical users who retain control over the program's actions.
If we implement the "Favorites" button, then maybe we don't need the "Recently used Directories" portion. Then all we would be adding is basically a simple progressbar at the bottom of the dialogue - hardly "prohibitively complex" (progress meters are very common in a browser). Also, we download files (to an unknown location) when clicking on links to other web pages, so I see no problem with downloading a file when the user explicitly clicks on a download link (especially, since it can be easily & quickly canceled). Ideally, the predownlooad would go to either the "default download directory (bug 7840) or to the "last used directory". The predownloaded file could have the filename = "URLgivenName.ext.moz". This would hardly be an unknown location/name. Fow the few fuddy-duddy "common users" that don't understand the benefits of predownloading, there could be a pref to toggle it ON or OFF. Updated UI suggestion: +- Enter name of file to save to... --------------------+ | __________________ | | Save in: |_MyDownloads______|\/| [^] [Favorites] | | +---------------------------------------------------+ | | | files (or favorites) listed here | | | | | | | | | | | | | | | | | | | +---------------------------------------------------+ | | File name: [________________________] [ SAVE ] | | Save as type: [_____________________|\/| [CANCEL ] | | | | _______________________________ | | Predownloading File ||||||||____37_%________________| | | | | The predownloaded file will be moved to the path and | | filename you select once you click SAVE, or deleted | | if you click CANCEL. | +-------------------------------------------------------+ PS. I know you all must be very stressed out with all the mozila 1.0 stuff. Instead of discarding this concept, maybe it should be "futured" to a time when more resources (and thought) can be put into it. :)
All of these points have been answered before, either here, in the newsgroups, or in bug 22796. We're just going round in circles. Does anyone other than Mr. Lairo still think that making such choices for the user is valid?
> Does anyone other than Mr. Lairo still think I (still) think that for a certain small subset of Gecko users, under certain circumstances, automatically "making such choices" is wanted. But not the way we currently do it. I am very much in favor of ripping the whole thing out that's in there currently, and implementing KISS, >>>reliable<<<, non-automatic downloading for 1.0. We can worry about 'enhancing' it for 2.0, once it actually works.
Peter Trudelle, I mostly agree with Peter Lairo. I do think the priority should be to get downloading working better first before implementing the pre-download feature. But the pre-download feature would be quite useful to advance users, so long as the initial download directory could be specified (to avoid all the problems with TMP or the default temp drive not having enough room).
Re 192: >I am very much in favor of ripping the whole thing out that's >in there currently, and implementing KISS, >>>reliable<<<, non-automatic >downloading for 1.0. We can worry about 'enhancing' it for 2.0, once it >actually works. Hear, hear! Making Mozilla the cleverest software in the world doesn't seem to be part of the 1.0 effort (as far as I've read). Making Mozilla WORK, however, does. As my teacher used to say, "Perfect is pretty, done is beautiful."
Keywords: nsbeta1
QA Contact: sairuh → benc
Whiteboard: se-radar
i don't expect to have any time to work on this before mozilla 0.9.9 :-(
Keywords: mozilla1.0
Target Milestone: mozilla0.9.9 → mozilla1.0
Status: NEW → ASSIGNED
Priority: P3 → P2
Keywords: nsbeta1 → nsbeta1+
To clarify what is going on in #182 (because I don't think some people understood what was really going on...) /tmp is mapped to memory in Solaris. If you dump really large files into /tmp, you cause the system to thrash as your downloaded file starts to cause other processes (including mozilla) to page to disk, then page back (because mozilla code needs to be executed in memory to continue the download that is chocking off the system to begin with). This is why I have argued on occassion that what we need is a single conversation where a download design is built from the bottom up. And by bottom up, I mean that we make a couple statements about how we understand memory, disks, cache, processes and error states would relate, and from there decide what kinds of ways we would add sensibly add files while browsing. From there we would proceed to prioritize features and then figure out how to present them to the user. It is too late to do a highly-robust, super-cool, frilly download manager. We probably had time, but nobody really picked up the ball and had the right conversations at the right time. Mozilla 1.0 is closing, and trudelle@netscape.com's approach, which is to get something that gets files to disk in a conservative but working manner is the way to go now. I urge individuals reading this (and related bugs) to use some retraint about commenting in ways that eat up trudelle's bandwith, but do not help us get 1.0 done. If we could find a place (newsgroup) to start a post-Mozilla 1.0 download manager conversation, lets do it. If we are going to do this in bugs, lets mark them clearly as 1.0+, and also people that agree but have no real insights to add in comments need to learn to use the voting system. Mozilla contributors are working code that works on more than the specific system you have. (If you can't live without a feature that works for you but would torment other systems, nobody is going to stop you from downloading the code and building your own features for your own system). Many of these bugs are features that just don't make sense at the OS level (for example, if I remember correctly, most systems would not easily be able to continue to pre-download a file and move it if the destination was a different filesystem). I'm sorry that mozilla and bugzilla did not provide a better format for discussing this feature. Everyone should have a voice, but mozilla is not as much a place about free expression as it is about free (and implicilty) working code. At this point, we know what we can and can't try to do. Please, "do the right thing" and keep reality in mind as we work on this. (In that spirit, I just deleted about 3 ranting paragraphs from this comment. They were persuasive, somewhat humerous, but possibly upsetting to small percentage of readers.)
I attempted a 150+M download recently and had a major hassle.... it saved to a temp location on drive C which had insufficient space.. and after 6Hrs (150+M download) it ran out of space for the silly temp file and bombed.... If using this terrible temp file kludge is absolutely necessary... then pls consider... 1) I have my diskcache set to a specific location for temporary data for all apps, so that I can one shot delete orphaned garbage... mozilla ignores this setting.. why not use that.... 2) check file space available before starting..... 3) why not create the temp file in the same location as the download destination, then only a rename is needed, instead of taking minutes to do a full copy.... 4) why not ask the user at the very first time he downloads sonething, where their download location is... ie start up with a blank location, then prompt the user, and save this for ever more... But best of all... get rid of the need for the temp file all together.... User selectable...? Thanks for your work on Mozilla, it is shaping up really well..... I do appreciate your, and everyone elses efforts... regards,
Suggestion for changing the downloader completely, so this is no longer an issue. Why not make a "mini" downloader EXE that mozilla launches after determining file content... All it would do is take a URL , ask for a location (if not specified already) and fetch the URL. basically, using a "helper" to download means: a) it's easier to extend the downloader (some other program handling binary downloads by a pref) b) mozilla doesn't have to worry about reconnecting as the external prog does connect and download. c) the helper could be used by other programs d) If mozilla crashes on some other content, the download would continue (BIG BIG FEATURE!) e) the helper wouldn't start downloading ANYTHING until the location was picked (this issue) I think a small program using the Mozilla DLLs could easily prompt for a location, using the system save dialog, and display a download progress indicator.
> e) the helper wouldn't start downloading ANYTHING until the location was picked > (this issue) Clarification: No, this bug aknowledges predownloading as valuable (it's implied), since it requests that the spool file be moved once the user picks a filename (see the "summary").
peter, pass me that bong when you're done with it. we've agreed on no such thing. we're still arguing over the first part ("saving a file creates a file before location is chosen").
> All it would do is take a URL , ask for a location (if not specified already) > and fetch the URL. Totally unacceptable. At the least, the following information is needed to properly retreive a generic resource: 1) URL 2) postdata 3) relevant cookies 4) HTTP Auth information 5) SSL certificates Passing all this to an external process (which, note, must handle SSL and so forth) is hard and likely a security issue. Oh, and reposting is often _not_ an acceptable thing to do when using systems that generate documents to be saved in response to post data...
As a note, the spool file that gets moved solution has one issue with it -- bug 124307...
Since this bug is seemingly not moving, I'm at least going to toss out a concrete proposal to try to get in for everyone to pick on. Predownloading (bad term) is the downloading that is done between the time when a user clicks on a link and the time when a user tells Mozilla where to save the data. Not predownloading causes Mozilla to make a connection when the link is clicked and to possibly make a new connection when the save location is chosen. As anyone who has worked on View Source can tell you, that's a BAD THING. Predownloading unsupported data types is not in itself a bad behavior since if the data had been a supported type, it would have been entirely downloaded with no further intervention by the user. Currently, the solution is to spool data to a temp file and move on download completion. That causes performance problems on machines which handle large temp files poorly and data loss if the spool area is exhausted or the spool file prevents the actual file from being created. No spool file solution can fully solve these problems because they depend on conditions independent of Mozilla's operation. The speed and available space of your spool area can change quickly and without notice. Other problems (security, usability, etc.) with the current solution have also been reported. Proposal: Adopt a default predownloading behavior similar to IE4.x/IE6.x/NS4.x. When a link is clicked, determine whether the data is a supported type in Mozilla. If not, begin downloading data into a memory buffer. The memory buffer supports emptying (reading) as well as writing with concurrency control as necessary. The buffer has some way of indicating whether the transfer has been completed. Whenever the accumulated buffer is larger than MAXBYTES in size, download 1 byte per THROTTLE seconds. Whenever the accumulated buffer is no more than MAXBYTES in size, download at full speed. A value of MAXBYTES and THROTTLE indicating unlimited should be designated. If the save is aborted, the buffer is immediately freed. The default buffer reading strategy is to do nothing until a save location is selected. Once a save location is selected, read from the memory buffer when bytes are available and write to the save location until the transfer is complete. MAXBYTES of 128K would fully buffer most files and should not cause undue delays when downloading larger files. THROTTLE of 3 would cause memory usage to be less than 1M per month and is less than the timeout of any reasonable protocol. As an option, the buffer reading strategy, MAXBYTES, and THROTTLES could be changed. This may include strategies with potential performance or data loss problems. At least one safe strategy should exist and be chosen as the default such as above. Other strategies could be suggested through separate bugs. User customizable values are nice but optional. Even if the save location is slow (less than 24 bps for the default!) or the save location box is left open, continuing to accumulate the buffer is not a problem as the memory footprint of Mozilla would not be significant for years. Impact on throughput would be noticeable only when both the location being read from and written to have speeds comparable to a memory-to-memory copy.
Blocks: 129923
*** Bug 129834 has been marked as a duplicate of this bug. ***
blake: does the download manager code have any effect this bug?
At least from my end, doing my own builds, DL manager doesn't work and DL still doesn't work. I'm starting to use curl for all my dl needs; copy link and send to curl. How hard would it be for you guys to just redirect your dl process to curl, anyway?
Please read comment 201 carefully.
Whiteboard: se-radar → se-radar[adt2]
*** Bug 133404 has been marked as a duplicate of this bug. ***
Switching the two parts of the summary around so I don't have a knee-jerk reaction of coming to this bug and invalidating it each time I see it...
Summary: Saving a file creates a file before location is chosen; spool file should be moved once the user picks a filename → Spool file should be moved once the user picks a filename (saving a file creates a file before location is chosen)
That's really an attempt to change the sense of the original defect report, which is that a file is saved without the user's knowledge or consent. Dismissing the save dialog is not just picking a name and location, it *is* the Save command in this case. The difference between supported and unsupported types is an important distinction when you're filling up someone's temp directory with data they don't know about and for all we know have no way to use. A scheme such as described in comment #203 might be an acceptable compromise, assuming all the other defects associated with this behavior are fixed, but it still makes the flawed assumption that a Save command has been issued when it has not.
For crying out loud. I didn't try to change the meaning of anything. I merely switched the two independent parts of the summary around, from the format "cause; solution" to the format "solution (cause)". I have no interest in rehashing the issues that have already been beaten to death in this bug. If you want to discuss this, take it to n.p.m.ui or n.p.m.netlib, where it should have been brought up months ago.
You took what was originally the entire summary of this bug, and moved it to a parenthetical comment, prefixed by one proposed 'solution'. The summary doesn't even sound like a defect report anymore so much as an enhancement request.
> The summary doesn't even sound like a defect report anymore so much as an enhancement request. To me (at least), this bug *is* an enhancement request. Please, off to newsgroups.
"Off to newsgroups" is not a valid response. As one of the users who opened a problem that folded into this bug, I can say the following: 1) The proposed solution addresses most of the nastiness that caused me to open my version of this bug. In particular, downloading ISOs using the badly flawed implementation in place last time I tried it, when whatever filesystem that Mozilla chooses to load to can't hold the ISO, is doomed to failure, even if the destination I chose could hold the ISO. Also, trying to figure out what's happening when mozilla uses some random filename, instead of the one I chose, is more dificult. The proposed solution does address these issues. 2) I'm not convinced that the proposed solution is wholly correct, especially if this is to be done soon. I think starting to download, at least more than a few frames worth to maintain a connection, possibly even completing the entire download for small files, before the user has confirmed download, is probably a mistake, since you are doing something the user hasn't actually finished asking you to do. I have previously maintained that the benifit is small in any case, especioally considering the debating time that has been spent on it. Anyhow, thanx for the work on mozilla, it looks better and better (I usually use the Galeon variant, since it make easier in the user interface a few things I like to do (disabling popups, disabling/enabling images, scaling text regardless of brain-dead webmasters' [is there any other kind?] use of the font tag with absolute size specified...)
*** Bug 140817 has been marked as a duplicate of this bug. ***
no hope of getting this fixed in time for 1.0
Target Milestone: mozilla1.0 → mozilla1.1alpha
*** Bug 144253 has been marked as a duplicate of this bug. ***
Target Milestone: mozilla1.1alpha → ---
mass futuring of untargeted bugs
Target Milestone: --- → Future
Mass removing self from CC list.
Now I feel sumb because I have to add back. Sorry for the spam.
If this bug is really just about *where* the spool file is located (which sounds like, in terms of this bug, downloading before the user has chosen what to do with the file is a forgone conclusion), I will go ahead and reopen bug 133404 which is about the fact that downloading is happening before I have picked what I want to do with the file.
QA Contact: benc → sairuh
QA Contact: sairuh → benc
What is this bug for? It has two completely different summaries. One says the spool file should be moved. One describes auto downloading. Those are two very seperate things. Which is considered the bug? Have meetings occured regarding how to go about resolving the mass of (auto)download bugs?
as I understand it, the main issue is: after the user picks a filename (including location, of cause) the retrieved data should be written to this file, even if the download is not completed entirely. (so the user could watch the file grow and probably resume a canceled download - but this resuming would probably be another bug) it's just for that the user can observe what mozilla does, while it's still in progress. therefore things would be more convinient for users. then we're discussing if mozilla should start downloading the file to a spoolfile *before* the user picked a filename (at this point mozilla does not know where to save the file). resolving this issue would make things easier. if you don't have a spoolfile, you don't need to move it but can start downloading to this file directly. but hey guys. while i'm on it: i think, we should be implementing the move of the spoolfile anyway! this is what this bug (or what's left of the issue) is all about in the first place. in another bug could track the preference stuff, if the user decides downloads should start before a filename is picked or not. this second issue really should be carried to another bug, leaving the primary issue to be fixed more rapidly.
*** Bug 133404 has been marked as a duplicate of this bug. ***
*** Bug 156799 has been marked as a duplicate of this bug. ***
Can't someone just pick an option and implement it? Then if people don't like it, they can complain about something new. But at least in the meanwhile, this two year old bug can be closed. (IMHO, of course.) Dan.
Hi, this bug should be regarded as 'critical' due to blocking e.g. ID:69938 which is critical itself. Best regards, Peter
Compare bug 164433. There, I argue that it's a bad idea in the first place to make any network request before asking for a filename, at least on Unix.
I'm not sure that makes sense. We don't know if we're going to open or save the link until we make a network connection and find out what type the content is. How can we ask for a file name before knowing if it's going to be saved?
We need to determine if we can open it, but the dialog isn't just 'asking for a filename'; we don't know if it is going to be saved until the user okays it.
> we don't know if it is going to be saved until the user okays it Why else would a user click on a link to a file, if not to download or view it?
Because they thought it was a link to something else. Happens to me all the time. I see a link that says, "read more" and I click it. Then it turns out to be a Word document or something. Also, people click file links to see how big a file is. I do that occasionally.
Right, they click on links expecting the browser to open them. When it cannot, it needs the users permission before downloading something that the user may have no other way to open.
If you are talking about my comment, please first read the bug I cited. There, I talk about Save Target As Link and Save As (in the file menu, for displayed content). In both cases, it is already sure that we want to save to disk.
Ben, in neither of those cases is there a spool file involved. The spool file is only created when we go through the helper app "what do I do now?" dialog. In other words, that has nothing to do with this bug.
Attached patch step 1 (obsolete) — — Splinter Review
This implements step 1 of the "Make this code behave" N-step program. What this very rough patch does: 1) Makes the helper app dialog modal to the browser window involved. 2) Makes the filepicker modal to the helper app window (if it exists; otherwise it's modal to the browser window as currently). 3) Moves lots of code from nsHelperAppDlg.js to helperAppDlg.js and has helperAppDlg.xul include the latter and call that code (this step is semi-optional, but I wanted to separate the component that implements that interface from the dialog it pops up. They were far too intertwined in the old world). Effects of this patch: Pre-downloading is basically eliminated. I opened up a largish remote (http) PDF and waited 20 minutes. The temp file size was 0. I then clicked OK in the helper app dialog and the download completed and opened in my helper app. Looking in my server log, one single connection was made to fetch that file (and maintained open for the 20 minutes in question). Remaining issues: 1) Decide how much of the infrastructure to support async pre-donwloading we want to keep (for embeddors' sake, if nothing else). 2) Fix nsExternalAppHandler::OnStartRequest to not create the temp file until _after_ the dialog has returned (separate the "default filename" code from the SetUpTempFile() function). 3) Fix nsExternalAppHandler::OnStartRequest to always content-decode data that's going to a helper. 4) Clean up the nsHelperAppDlg.js/helperAppDlg.js code. I just moved it wholesale for now, commenting out random bits, putting random stuff on the window object, etc. 5) Clean up the way we interface with preferences; hopefully we can pass in just the nsHelperAppDlg object and not the window object we're passing right now.... 6) Test like crazy. I've tested the basic functionality of the helper app dialog with the changes I've made so far, but more testing would be incredibly welcome.
I forgot remaining issue #1.5. Now that we know where we're going to be saving to (if we're saving), we should start spooling to the correct directory. What filename should we spool to? The "final" filename? The "final" filename with ".moz-download" appended (a la ".getright")? A salted filename? Should this different on different platforms?
Oh. And before we restart the years-long debate, anyone suggesting pre-downloading had better have a good solution for handling pre-donwloading of content-encoded content (possibly with multiple encodings). Such content should _usually_ be decoded before passing to helper apps and _usually_ not decoded when being saved.
re comment #237: I dont see a reason why the filename used should not be the final one (i.e. the one the user specified). After all, that is what one would expect in the first place. It would also make many things easier: e.g. checking a large mp3 or movie download to see if it should be continued or aborted would be easier; the correct name would be remembered in DM, aborted downloads can be continued using other managers like wget more easily etc.
Johann, from what I've read somewhere, the idea of a different filename was to let people know when a download is truly finished so they don't try to use a file that is only partially downloaded.
reasons to claim ownership for files we haven't finished downloading: * parity: (beos)net+, (mac)icab, (mac)ie, (win)getright * icon handling to show progress * useful behaviors for properties and opening the document before the download completes * resume - if mozilla dies (crash, forcequit, os crash, powerfailure) and the user tries to double click the partially downloaded file they get mozilla which tries to find out where the source is and tries to resume it (bug 18004 is the only reference in the blocking list), instead of the final application yelling and screaming about a broken file. for windows, the extension .moz-download would allow us to hook up an icon handler for the file so that we could provide a progress indicator as part of the icon and for resume/progress/properties if someone tries to open it too early. for beos, i'd suggest we write the correct filename with the wrong mime type. I'm not sure if we can use the right mimetype +";creator=mozilla" or if we should just use a constant mime type (something like application/octet-stream;creator=mozilla) but we should look at how NetPositive does this. again when we finish, we fix up this magic. for macos, the creator should probably be MOZZ until we're done writing the file (check MacIE), I think that handles the same issue as .moz-download for windows. for linux, as usual, I don't really want to think about it, but I'd suggest we follow windows, the reasons still apply although convincing associations to work is someone else's problem. for os/2, i'd imagine some cross between the preceding (iow, ask mkaply) for photon, i'd imagine some cross between the preceding (iow, i'll ask irc.joher.com #qnx when the time comes). We can check what QNX Voyager does. note that QNX users might be more likely to use QNX Voyager w/ mozilla_server (replacing voyager_server) than to use the mozilla front end, so whatever we do has to be flexible enough so that an embedding system can choose (not) to use it. the fun platform is macosx (jaguar), because some of the documentation apple recently published was basically targetted at mozilla. i think that we'd do the following: * if the user hid the extension, we replace the extension with .moz-download until we finish, at which point we replace .moz-download with the real .ext * if the user did not hide the extension, then we use .ext.moz-download and hide the extension until we finish, at which point we remove .moz-download and unhide the extension i'd suggest that we address the above in a bug per toolkit and a meta.
I was able to apply the patch and spun a build. Mozilla appears to do no predownloading to disk. There are problems. 1) Mozilla does appear to still be predownloading something to somewhere. I see 10s of KB being transferred after the helper app dialog appears lasting for about 20 seconds. 2) Downloading of executables completely broken. The helper app dialog appears like this: ---------------------------------------------------------- Downloading #1 ---------------------------------------------------------- You have chosen to download a file of type: "#1" [#2] from What should #1 do with this file? ( ) Open using #1 [Choose...] ( ) Save this file to disk [ ] Always ask before opening the type of file ----------------------------------------------------------- [Advanced...] [ Ok ] [Cancel] With neither option selected. Selecting save and clicking ok results in no file transfer. Error on console: JavaScript error: chrome://global/content/helperAppDlg.js line 12: nsnull is not defined 3) After sitting for a few minutes at the helper dialog, the download fails to begin after clicking save. Download dialog appears but no data is ever transferred. 4) After dismissing a download dialog, helper app dialogs for random files I've tried to download from that server in this session appear. 5) JavaScript strict warning: chrome://global/content/helperAppDlg.js line 166: reference to undefined propert y window.choseApp 6) JavaScript strict warning: chrome://global/content/helperAppDlg.js line 276: reference to undefined propert y window.givenDefaultApp 7) Not sure if this is yours (when clicking advanced button in helper dialog): WARNING: waaah!, file x:/mozilla/content/xul/document/src/nsXULPrototypeDocument .cpp, line 782 JavaScript strict warning: chrome://communicator/content/pref/pref-applications-new.js line 120: function o nOK does not always return a value WARNING: waaah!, file x:/mozilla/content/xul/document/src/nsXULPrototypeDocument .cpp, line 782 JavaScript strict warning: chrome://communicator/content/pref/pref-applications-new.js line 184: function o nOK does not always return a value JavaScript strict warning: chrome://communicator/content/pref/pref-applications-new.js line 51: reference t o undefined property window.arguments[0].chosenApp 8) JavaScript strict warning: chrome://global/content/helperAppDlg.js line 276: reference to undefined propert y window.choseApp
As a user I _want_ the spool file to keep downloading. I have plenty of disk space (gigabytes) and I often take a long time to decide where to download a file (sometimes minutes) during which the file has plenty of time to finish downloading. We've already covered ways to address the problems this raises: * Spool. Once a download location has been picked, move the file there and continue downloading to that location. * If you get near to running out of room on the download disk, stop spooling and wait. (Another reason we have to keep downloading is that you don't want to let the connection time out.)
Some do, some don't. And doing background spooling has security and disk space issues. I don't see why we can't accumulate a fair size chunk in memory (several megabytes) to avoid the disk spooling issue.
how about this: let's move forward w/ bz's patch because it does indeed fix a lot of very serious bugs. then, let's discuss how best to support pre-downloading in another bug :-)
OK. Responding to comments one at a time: Comment 239: see comment 240. You don't want to open up a partially-downloaded Word file... (again, we may want this to be OS-dependent; on Unix the expectation is that the filename you chose is the filename used, period). Comment 241: agreed that separate bugs are a decent idea. Comment 242: 1) We may still be downloading once that dialog comes up, but that's because the data is already in the pipe, so to say. we will only do it for a little bit (20s sounds about right) and cache it in memory. Then we'll suspend the socket and let the TCP stack maintain the connection as best it can. 2) Thank you! The executable thing is really a Windows-only check, which is why I did not catch the typo. Change the "nsnull" in line 12 to a "null". 3) Ugh. That part is bad. What was the server involved? I was trying to cause that to happen but failed; it's likely very server-dependent... 4) I won't worry about the strict warnings for now... like I said, this was a very preliminary hack. Most of those variables will get moved around a bit. Comment 243: You didn't read comment 238. Read it. Then answer my question. So in summary: When does this fail to work completely (per comment 242)? What can we do about that?
About pre-downloading: why not assume that the file will be saved (more conservative) until the user decides what he wants, then when the file gets moved (iow, when a decision is made about saving vs. opening), do any necessary decoding?
Having spoken about this with bz on IRC: There are two totally separate issues here which an architectural problem is incorrectly forcing us to consider together: the UI, and the backend. The two should be separate. The UI issues involve the parenting of dialogs and windows, and their modality. The filepicker app should have the helper app window as parent, not the browser. The helper app window should not have a parent, and thus not be modal -- it should be a sibling to the browser windows, so that closing a browser window will not close the spawned helper app windows. The filepicker should be modal to the helper app window. All of this is UI and should have no bearing on what happens in the background. The backend issues involve predownloading and content decoding. The file should be downloaded to the temporary folder straight away, and continue doing so in the background while the dialogs are up. This should all happen on a separate thread from the UI. Once enough of bytes have been downloaded to decide what to do with the file, if it is found that we need user intervention, then you spring open a helper app dialog. Note that downloading should continue, at least while there is plenty of free disk space. Once the user has picked what to do, or once we know what to do from having looked at the headers/content downloaded, then we should move/rename the file to the appropriate directory, and if appropriate apply any content decoders to the file. This may mean moving the content decoders out of the HTTP code, but that will have to happen anyway since other protocols have content decoding requirements and we don't want code duplication. The security issues are moot (the same problem exists with all kinds of downloads, not just predownloads), the content decoding issues are moot (with this system we do it all afterwards anyway) and the running-out-of-diskspace issues are moot (you stop downloading before that becomes an issue). In the meantime, the people who are paying for their connection by the second have not wasted entire minutes while trying to decide where to save their ISO.
My proposal above would also fix the following bugs: bug 109594, `Reveal Location' button doesn't work while file is downloading ...because by the time that button appears, we'll have moved the file to the right place bug 103766, Browser hangs after downloading a large file ...because that work would happen on a separate thread (and could even be given UI like in IE) bug 132702, Mozilla doesn't display image in helper app ...because we wouldn't apply the content decoders until after we know what we plan to do with the file bug 124307, Awful permissions for file saving as of recently ...because you can just reset the permissions after moving the file bug 88566, [RFE] display partially downloaded file in target location ...because the file would be moved as soon as we know the target location ...and probably several others for the same reasons.
Comment 247: because I don't have the time, knowledge, or patience to rearrange necko to expose the content-encoding stuff. So I'm taking the "correctness first" approach here. We currently compeletely flub downloads or opening things in helper apps in many situations. We should not. Comment 248: Every Windows user I asked said that their expectation (on Windows) is that the helper app dialog is modal to the browser. My expectation (on Linux) is the same. Comment 249: Would not fix bug 124307, for the reasons described in that bug (it would be functionally equivalent to my existing patch in that bug, which suffers from a security-related race condition. I agree that we should have a cleaner separation between UI and back end. My proposed fix for this bug will take the first steps in that direction. Again, I would rather have us behaving correctly than "conveniently" in this instance. (I quote "conveniently" because imo being unable to save things to disk or open things in helper is _incredibly_ inconvenient). Back to the technical issue at hand. Under what circumstances does this lead to dropped connections? Until that issue is resolved, everything else is moot.
I saw silent server dropping with both ftp://ftp.mozilla.org and http://java.sun.com. It's not something I can consistently get to happen though. Changing nsnull to null does clear up the problem with downloading executables on Windows. Also, to chime in on the modality issue- I could see the helper app dialog being modal to the content window that spawned it, but the dialog is globally app modal. I can't, for instance, close the browser or switch tabs with a helper app dialog open. That makes no sense in terms of parenting.
It's modal to the window that spawned it... so you should be able to use other windows (but yes, you will not be able to use tabs in the same window or close the window). I'll do some testing with those sites; thank you.
bz wrote: > Fix nsExternalAppHandler::OnStartRequest to always content-decode data > that's going to a helper. and later: > content-encoded content (possibly with multiple encodings). Such content > should _usually_ be decoded before passing to helper apps and _usually_ not > decoded when being saved. (Although I am not sure how this is relevant to this bug:) That seems to contradict each other. "always" or "usually"? First, did I understand it correctly that content-encoding is e.g. gzip when a tar.gz files lies on the server's harddisk and is delivered with HTTP Content-Encoding: application/gzip or however the syntax is? What about transfer encoding (the file is being requested as .ps and the server compresses it with gzip on the fly)? (Please correct me, if I misunderstood somemthing.) It is a tough question, if that should be decoded (uncompressed) by Mozilla before handing it to a helper app. I think that *usually*, content encoding should be preserved, while transfer encoding should not. There are other cases, though. If I request a tar.gz, I most likely don't want it to be touched by Mozilla. If I want a requested ps.gz (.gz appears in the URL) to be uncompressed depends on my PS viewer. There might even be cases where I don't want the transfer encoding to be touched, but the helper app then of course needs some way to figure out that the file is encoded(compressed). bz wrote: > Every Windows user I asked said that their expectation (on Windows) > is that the helper app dialog is modal to the browser. My > expectation (on Linux) is the same. Yet, my expectation is also that a download spawn from a certain browser window will not terminate unless I close the windows directly related to the download, not the browser window. I often open a browser window, start a download and then close the browser window. I yelled at Opera Linux when it quit after I closed the last browser window with the download manager still open and downloading, i.e. breaking my downloads. > Under what circumstances does this lead to > dropped connections? Until that issue is resolved, everything else is moot. Not necessarily. If all fails, you can still do a HTTP HEAD, open file picker, then do the real download, not?
Ben, the theory is that .tar.gz would be delivered with content-encoding and a PDF compressed on the fly would be delivered with transfer encoding. The practice is that no browser or web server supports transfer encoding (vicious cycle), so both are sent with content-encoding in the real world. So while your comment is correct that content-encoding probably should not be decompressed in practice that will break lots of things. That's where the "usually" creeps into what I said.. We can't tell which the site meant without some guessing... In the helper case, we must always decode because we picked the helper based on the decoded type. Of course whether we wanted to pick that helper is different issue (and a separate bug should be filed at some point). HTTP HEAD is so unreliable as to be worthless. This is even true for GET requests, and don't even mention POST (which are becomming more and more common as a method of kicking off downloads). Also, keep in mind that we can't do an HTTP HEAD on _every_ link click, which is what you'd need to use HTTP HEAD here.
------- Additional Comments From bzbarsky@mit.edu 2002-09-10 22:20 ------- > The practice is that no browser or web server supports transfer encoding > (vicious cycle), so both are sent with content-encoding in the real world. Yes, I know that reality doesn't match the specs here. Curiosity: Is it possible for Mozilla to support transfer-encoding correctly, so that we can get out of the cycle as far as Mozilla is concerned? > That's where the "usually" creeps into what I said.. We can't tell which > the site meant without some guessing... In the helper case, we must always > decode because we picked the helper based on the decoded type. What, if I have ark (that's how it's called, IIRC) sat up to handle |tar.gz|s? Hopefully, you won't fill up my /tmp with an uncompressed tar. E.g. Mozilla source: compressed <30 MB, uncompressed >200 MB, not funny.
Based on a 30-second glimpse at the http channel implementation, it should not be too hard to support transfer-encoding... file a bug and cc me? ;) > What, if I have ark (that's how it's called, IIRC) sat up to handle |tar.gz|s? You can't. You can have it set up to handle application/x-tar or application/x-gzip. In the former case, we need to uncompress, no? In the latter case, we would either not uncompress (if that's the content-type the server sent) or not pick up "ark" as a helper.
> support transfer-encoding... file a bug and cc me? Bug 167905. > You can have [Ark] set up to handle application/x-tar or > application/x-gzip. In the former case, we need to uncompress, no? In the > latter case, we would either not uncompress (if that's the content-type the > server sent) or not pick up "ark" as a helper. Apache's default config seems to be: srm.conf: AddEncoding x-gzip gz mime.types: application/x-gtar gtar tgz taz application/x-tar tar # Note: Compression schemes like "gzip", "bzip", and "compress" are not # actually "mime-types". They are "encodings" and hence must _not_ have # entries in this file to map their extensions. The "mime-type" of an # encoded file refers to the type of data that has been encoded, not the # type of the encoding. which I interpret, that |.tar.gz|s will be sent as mimetype "application/x-tar", content-encoding "x-gzip". Which means that you would decompress on the fly, store the uncompressed file in /tmp and open it with ark, which is the default handler for "application/x-tar" per mailcap (I assume). Meaning that a) my space on /tmp is wasted b) I can't easily save the file as tar.gz once opened by the helper app unless I recompress it manually.
Apache's default config sends: Content-type: application/x-gzip Content-encoding: gzip For .tar.gz. We have code in http-land to deal with this crap (we basically ignore the content-encoding in such cases). There is no default tar handler in mailcap (at least on RedHat 7.2; nothing stopping people from shipping that I guess). And this is why we always prompt the first time a helper from mailcap is used for a given content-type -- the helper may be wildly wrong (need I mention that "lpr" was the default application/postscript helper for years?). In any case, could we take the discussion to the newsgroups if we're not talking about things directly relevant to this bug or this patch?
Blocks: 152275
Whiteboard: se-radar[adt2] → se-radar[adt2] (py8ieh:248)
Comment 248: Once we know that we cannot handle the file, we should stop downloading it until the user decides what to do. We have no business using their bandwidth and disk space downloading content that we cannot display and they have not told us to save. The click was permission to display, not to download.
comment 260 is incorrect. A click on a "DOWNLOAD" button or .EXE, .ZIP, .MP3,... link almost always _IS_ permission and intent to download.
Peter Lairo, it is odd to say that the opinion stated in comment #260 is "incorrect". Unless you can prove that it is wrong for all users, all you can do is disagree. I, for that matter, disagree with *your* view. There are many reasons why a user might back off from downloading: the file is already there, the file is not what was expected (oh its a dialer.exe and not a cutie.jpg). I fully support Peter Trudelle's opinion that we should not waste bandwith or space until the user knows and approves that the download starts. (speaking of which, the download manager should show additional info to make that decision easier, like mime type, and file size, if available). So,since there are different views possible we should try to compromise instead of qualifying another person's opinion as "incorrect" - and your own one as "correct", by the law of "tertium non datur".
OK, the statement is "mostly" incorrect because it claims "The click was permission to display, not to download", where in reality it *usually* is permission to DL. > There are many reasons why a user might back off from downloading: > the file is already there, That is a *very rare* case that a user clicks a link to a file he already has. > the file is not what was expected (oh its a dialer.exe and not a cutie.jpg). That is yet another *extremely rare* case (NEVER happened to me). Can't you provide any example(s) that would represent the *majority* of cases, that would justify removing this useful feature? No? (latinum est arrogantum)
oh, I agree that those cases I mentioned occur in the minority of the download requests. However even if they never occur it does not prove that simply clicking on the link already should be regarded as the permission to start the download. It also depends on personal taste, but usually it is a request to bring up the save dialog, nothing more. Clicking the OK button then is the permission to actually start the download. Note that this protocol is nearly used everywhere: an ordinary "save" dialog in practically all applications will do absolutely nothing until the user presses OK. Practically all other dialogs will not do anything until a user confirms with the OK button, so having it work differently here is to say the least, different and unexpected. I really dont see much usefulness of the feature when compared to the problems that arise - after all, all you gain in speed is those few split seconds of time between clicking the link and clicking OK. It doesnt seem right to me to accept all the trouble this is causing - from a function not working as expected by the user to the space and file name issues - just for this.
To comment 261: I woudln't have guessed that you could tell so much context when starting your download, such as clicked on a download button. Just because the href points to a exe or mp3 or zip doesn't mean that the hyperlink itself displayed that information. A lot of times, I click on a link thinking that it will take me to a page which will have the download listed with a file size when in fact it is the file to be downloaded. To comment 263: I guess you'll have to define *very rare* since this happens to me quite frequently when I'm trying to download the next version of something. In fact, I use the notion that the filename will already be present if I have already downloaded that version and I let the system match the name for me. Maybe it's because I have a bad memory, but I wouldn't classify it as rare at all. Is this the majority of cases? No, but I wouldn't call it rare either. I would also tend to debate the usefulness of the feature. What isn't rare is one group of people finding something to be useful while others call it a bug (Witness the ordering of the "open in New Tab" .vs "open in New Window"). Personally, I don't find the preload useful or buggy. I just didn't like Mozilla filling up my tmp space when I didn't want the file to go there.
Comment 261: Do you really think it is possible to divine the user's intent from the content that they clicked on? We know our own intent when we click, but I don't understand how the code could know the user's intent when it doesn't even know how to display the link target, which is the expected fundamental behavior of a browser when a link is clicked.
Peter Lairo, read comment 238 and shut up unless you have a technical solution to the technical problem I mention there (which absolutely has to be solved). Could we lay off the flaming and design something that works? The current approach to helper apps is why we are, after 3 or 4 years, without a reasonably functioning helper app system. Again, I will fix correctness issues first and do nice features later. If you want the nice features, code them.
qa to sarah. (you moved it to File Handling, but you've given me two times now...) If this is in File Handling, then I'm done. I am already tied down in helping out with some Doc Shell and URL bugs that directly affect Networking, so I'm not going to be doing other non-Networking work for now. bz: my suggestion to you is to write a brief 2-3 page design document on mozilla.org, that addresses these issues everyone keeps "realizing" over and over, and just post a URL to the document everytime someone repeats a previously discussed item. I had previously thought I had time to help with this, but I don't in the near future.
QA Contact: benc → sairuh
-> jimmylee for now. i don't have the cycles at present to work on all of the aspects of this (other than the frontend, perhaps).
QA Contact: sairuh → jimmylee
Blocks: 135736
Could this be causing bug 181374?
Brian: this bug has had no patch checked in. it cannot possibly have caused another bug.
I am not talking about the patch. I'm talking about the issue that this bug is meant to fix.
Jimmy, a reminder you should qa assign this...
Yes, I am not the proper QA Contact for File Handling. Reassigned to default.
QA Contact: jimmylee → petersen
Depends on: 176919
This bug can't be fixed before mozilla 1.0 anymore ;) ? I suggest reviewing keywords.
You're right. Removing Mozilla1.0 keyword.
Keywords: mozilla1.0
*** Bug 191464 has been marked as a duplicate of this bug. ***
Why is bug 55690 (this one) a different bug than bug 69938 ? I understand that originally they started for different reasons, but this bug (saving a file creates file before you choose a location) is just a symptom of the real problem (downloads are stroed in /tmp regardless of filename picked). Together they've got 120 votes now, and have each been open for over 2 full years now. From reading through both of them, it seems like the real problem is that people disagree about the right solution. One camp says we should wait to open a file to save into until they pick the file/path, risking timing out the connection if they're really really slow about it. The other camp says we've got to keep the data flowing, and need a file to put it into until they choose, even if it means that this might prevent them from downloading the whole file after they waste a lot of time filling up their /tmp partition. Both solutions are valid, both have risks, and different people would rather take different risks. So make it a pref and be done with it. Both of those solutions are easy to implement (first one just waits to do something it already does, second one keeps going like it does now). Making a pref and having both of those options available would seem to let pretty much everyone get their gripes resolved.
> Why is bug 55690 (this one) a different bug than bug 69938 ? It's not, really. That's why this bug blocks that one. > From reading through both of them, it seems like the real problem is > that people disagree about the right solution. No. The real problem right this second is that I'm too busy with school to have time to fix this and that everyone else is having a lot more fun complaining than trying to follow up on the patch I posted in this bug. Though at this point, that patch is pointless (it would not come anywhere near applying, and there may be better ways to do waht needs to be done using the new necko async I/O stuff). > Both solutions are valid No. See comment 238. You did read the bug before commenting, right? > Both of those solutions are easy to implement Why don't you do it then? > just waits to do something it already does There is no way to "wait" to return from a function call without creating a new event queue to keep the UI painting and all the surrounding shenanigans. This is what the patch I attached does, by the way. The other options are busy-waiting (freezing up the whole UI in the process, which does not help, since they you can't pick a place to save to) or rearchitecting the caller to not assume that once your function has returned everything is set to go. Don't you think if this were easy it would have been DONE by now by one of the very helpful people who have ignored the patch and the "helpwanted" keyword for 5 months now?
Whiteboard: se-radar[adt2] (py8ieh:248) → se-radar[adt2] (py8ieh:248) read comment 238, 280 before commenting
Comment on attachment 98567 [details] [diff] [review] step 1 So to use the async I/O stuff, here's all you have to do: 1) Change nsExternalHelperAppService::DoContent to take a channel, not just a URI 2) Make it suspend the channel and put up the helper app dialog 3) Make the helper app gandler unsuspend the channel once the user makes a decision 4) Sort out the issues involved in setting a proper stream listener in the URILoader once the user decides what to do (that could wait at first, maybe; we could start with just setting the helper app handler as the listener to fix this bug). 5) Sort our what happens when the user clicks a link in the page while the dialog is up. Is the previous load cancelled? (this may be less of an issue depending on what happens in item #3). 6) Change the actual helper app handler code to not use a temp file (since at this point it will know what's happening to the data _before_ the data starts flowing).
Attachment #98567 - Attachment is obsolete: true
In my entirely biased and subjective opinion, the only things that need doing immediately are: 1. Fix bug 7840 (which is now four and a half years old). 2. Go back to the old, effective pre-downloading behavior, except store the temp file in the user-chosen default download directory with some kind of .moz-temp extension. 3. Don't move the temp file until the download finishes. 4. Content-decode stuff, if necessary, after it finishes downloading. If anyone wants a pref to not pre-download, then let them go through the toil of re-implementing the download system.
Clement, what you're suggesting is a hell of a lot more work to implement than what I'm suggesting. Feel free to do it, of course. I'm willing to review the patch once you're done.
Blocks: majorbugs
*** Bug 199405 has been marked as a duplicate of this bug. ***
*** Bug 200050 has been marked as a duplicate of this bug. ***
*** Bug 200187 has been marked as a duplicate of this bug. ***
Blocks: 124307
I'm not in the process of developing Mozilla, so I'm not a help in coding. Some weeks ago, i started the downloading of three files all about the size of 200MB. To this time I didn't know the files will be first saved to temp which only had 300MB. So each download started and after a half day of downloading I didn't get any of the files because no space on temp was left and resuming was not possible. That's annoying! What about to start the download as usually but with a limited bandwith. So it's possible to download into memory until mozilla knows what to do with the file. If you think of a downloadspeed of 1 kb/s it's enough to download small files in no time as usually. Even after 10 Minutes of thinking where to save the file to, not more than 1 MB downloaded. After the location is set, the connection can speed up. With this logic, it's possible to keep the connection alive, small files are downloaded immediately. If the user doesn't react within a given time the partially downloaded file can be saved to a tempfile and continue with the old behavior.
*** Bug 211650 has been marked as a duplicate of this bug. ***
*** Bug 211886 has been marked as a duplicate of this bug. ***
*** Bug 217143 has been marked as a duplicate of this bug. ***
Blocks: 224692
ok this bug is way too long to read it all I have a temporary workaround (yes I do intend that this will be temporary), namely that the file will be moved once the user picks a filename (compare summary). bz's suggestion is the real solution but requires work in other mozilla areas that I don't want to do right now. Fixing this either that way or the right way is required for resuming downloads, which is why I want this temporary solution. The effect of this patch: Files are moved to the final directory, with the additional extension .mdownload. taking bug, will attach a patch in a second
Assignee: darin → cbiesinger
Status: ASSIGNED → NEW
Target Milestone: Future → mozilla1.7alpha
Attached patch patch (obsolete) — — Splinter Review
Attachment #136558 - Flags: review?(bz-vacation)
(I will fix this "the right way" in bug 69938)
Status: NEW → ASSIGNED
Keywords: helpwanted
Comment on attachment 136558 [details] [diff] [review] patch found a problem with this patch (it should also Cancel() the channel)
Attachment #136558 - Attachment is obsolete: true
Attachment #136558 - Flags: review?(bz-vacation) → review-
Attached patch patch v2 (obsolete) — — Splinter Review
Attachment #136559 - Flags: review?(bz-vacation)
Comment on attachment 136559 [details] [diff] [review] patch v2 >Index: nsExternalHelperAppService.cpp >=================================================================== >+ mOutStream->Close(); >+ mOutStream = nsnull; codesize nit: won't the stream's dtor Close() it for you anyway?
ok, I'll change it to just ->Close and not set to nsnull (to be sure that it will be closed)
Could we make it .part instead of .mdownload? .mdownload looks kinda silly, and as far as I can tell there's nothing Mozilla-specific about a partially downloaded file. Thanks...
why not just make it save to the actual filename immediately?
read up?
to indicate that the file is incomplete, and to allow showing the progress in the file icon at some point. (and actually this was slightly simpler to implement)
The filename extention .part looks much better to me than .mdownload for partially downloaded files.
Plus, .part extention is already a de-facto standard for partially downloaded files. See http://filext.com/detaillist.php?extdetail=.part&Submit3=Go%21 for a list. eDonkey and Go!Zilla use it.
hm... I was leaning towards .download because it is a download, while .part can be anything. I am not sure if it is good to share this extension with other applications. hence I'd rather use .download. (I will attach no new patch. this is a trivial change)
If the "format" of a .part file (i.e., it's identical to what the final file will be except truncated, there are no extra leading/trailing bits, encoding, etc.) is consistent with other applications, then I think it would be a very good idea to use it, so users can begin a download with one application and resume it with another (for protocols and applications that allow resuming). I don't like ".mdownload" due to branding issues (should Netscape use ".ndownload", Beonex use ".bdownload", etc.?). I don't like ".download" because I don't feel it really suggests the nature of the file (in progress, partially completed). It's a rather long extension; keep in mind that on some platforms (e.g. Mac OS X), the middle part of a filename may be abbreviated "..." to save space on the screen, so having a long extension may cause less of the useful part of the filename to be visible. ".part" suggests that the file is partially complete, which is exactly right. The only way I see the meaning could be confused is with archiving programs (ZIP, RAR, StuffIt, etc.) that split an archive file into multiple segments; a user seeing a .part extension could think it to be one of these. However, presumably the user would only see them in their designated download directory, would recognize the filename, would know they started downloading something that hasn't finished yet, and would be smart enough to figure out what's going on.
Unlike Christian, I (as with others) see no problem with sharing ".part" with other applications. Assuming that, technically, it has the same format - then, IMO, having Mozilla follow the unofficial "standard" can only be a good thing. I'll also agree with others that if we are *not* to use ".part", I don't want to use ".download". It's too long and just doesn't look right. Even something like ".dwn" would be better.
FYI: Getright (windows Downloadmanager) adds .getright to unfinished downloads. It also registers itself as application for this extension and it will continue the download if you open this file if it's in the database and it asks for the URL if the file isn't in the database.
Oh - and, having said that, I should also mention that the naming of the download is *such* a minor point in comparison to what we're trying to fix here that I'd be happy with it being called ".somedownloadstillinprogress" if only that resolved this bug. So, if it makes it easier to stick with ".download" for now let's do that.
I like jason's last comment :) matti's comment is something I want in the future so whatever. I can use "part". that is a pretty trivial change. I don't care much. if bz has any preferred wording I'll use that. The truncating of the extension was a good point; and I really hate .dwn, so if bz has no opinion I'll use .part. (I would appreciate it if that discussion wouldn't take place in this bug. yes, the file will be the final file but truncated.)
My two bits on the file name mangling: (sorry to add to this, but I really do think my new proposed name has value) When I first saw 'part' mentioned here, I didn't quite get it. Part of a set? It's an ambiguous word. Made me think of USENET postings that come in parts. So I have a part. Is it a complete part? Also, because it's a mere four letters, it looks like perhaps it should have an application associated with it. (And I don't mean a download manager). MPEG, JPEG, PART... Probably some new media file I need to go find a viewer for. And Andy voiced my feelings on using 'download'. "Yes, I did download this file! Now why won't it open?". 'downloading' or some more active form of the word would work better, but is still awkward. A program called "pyslsk", a linux clone of the SoulSeek file sharing program, names files in progress as "INCOMPLETE<random_id>filename.mpeg". I forget the whole rational for the random_id, but I do very much like the "INCOMPLETE" part. It's at the beginning of the filename. This means if you have 3 or 4 incomplete downloads in a directory, you'll see this immediately--they're sorted together. It's at the beginning of the filename. You can't miss it, even if your file manager truncates the long name. It still has it's proper extension. If resuming the file isn't possible, you can still (depending on the format, of course) easily access what you DO have of the file by opening it as normal. Mp3s, mpegs, jpegs, and most media files will display/play/whatever with only a partial file. 'INCOMPLETE' is completely unambiguous. It's not a 'part' of something, it's not something you've 'download'ed already, it's incomplete.
so use .incomplete? Using a prefix removes the ability to write a shell icon handler to show the progess in the file icon. I do want that feature.
> So I have a part. Is it a complete part? No, by definition. :) I'm happy with .part or .incomplete (though in the latter case there might be cause to anally localise) -- but I'd be happy with .pastasurprise and agree with comment 307.
> Using a prefix removes the ability to write a shell icon handler to show the > progess in the file icon. That is a marginally neat feature. But comes at the cost of what I feel are integral features: Sorting Noticability (Windows truncates filename display very often) Ability to open/play/view file as it downloads If a fancy schmancy "shell icon handler" is deemed to be more important for Windows users than the above, can UNIX builds use the INCOMPLETE<filename> naming scheme? There will be no "shell icon handler" equivilent being added, so it would be nice if UNIX at least got the benifit of the 3 things listed above, instead of being completely shafted for the sake of Microsoft users. (as usual)
>instead of being completely shafted for the sake of Microsoft users. (as usual) osx supports icon handlers as well, and I wouldn't be surprised if nautilus or konqueror supported it too, which could ideally be supported. you're too negative :) note that you can sort by extension.
(I read enough mails -- now I want to say something, too :-) ) There are pros for both having 1.) INCOMPLETE prefix 2.) .incomplete extension 1.) Playing,... incomplete/downloading files is neat (e.g. you don't have to wait till the file is complete if reader supports that) However I see more points for 2.) -- Sorting is possible, too -- Having a dedicated icon is quite noticeable -- Extensibility: if we have a special extension, we can make tools that deal with that kind of files e.g. the shell-icon-animator, even better: a download-resumer (we'd need to store the URL somewhere...), "data-feeder", "multi-part downloader"
I don't really like the idea of adding a suffix to partial files, but I think it's a lot better than a prefix. Prefixed files would be very hard to find because files are usually sorted alphabetically. I don't see how using a prefix would partial files any more playable, at least on non-DOS-based OS'.
re "I don't see how using a prefix would partial files any more playable, at least on non-DOS-based OS": in most GUIs (classic Mac OS being the obvious exception but we don't support it anymore), the file extension determines which application should be used to open the file by default, in addition to which icon to display. If the filename is "INCOMPLETE-Funny.mpg" it will have a movie icon and double-clicking will launch your preferred video player. If the filename is "Funny.mpg.pastasurprise" it will look like a plate of spaghetti and double-clicking could potentially launch a download manager to resume it. Those using a CLI would only be concerned with how the filename displays alphabetically, since they have neither icons nor default applications. These users are in the minority; Mozilla is a GUI browser after all. CLI != UNIX; see comment 313. .incomplete could be shortened to .inc, but IMHO that's worse than .part. As mentioned in comment 314, to resume files with a download manager we'd need to store the URL; this negates the advantage of sharing a common extension with other applications. Might cause a problem if other apps use the same extension as us and want to use their own download managers, since they'll store URLs their own way, and it won't be obvious whose partial files are whose...
It doesn't matter what the filename is. It can be changed later. In another bug. Let's not block this one because of such sillyness. See http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING
I'll pick .part, mainly because of its shortness.
Comment on attachment 136559 [details] [diff] [review] patch v2 This looks pretty reasonable, but why all that use of "native" nsIFile methods? I'd think you would in fact want the Unicode ones.... (since you are appending a string, not a byte sequence).
Comment on attachment 136559 [details] [diff] [review] patch v2 In particular, I like this patch not a bit if "native" encoding is UTF-16.....
Attachment #136559 - Flags: review?(bz-vacation) → review-
One other thing that worries me is that if we download a good chunk of file and then the user selects the location, we will freeze up for a while as we move, giving the connection a decent chance to time out. :(
if native encoding is not ascii compatible, a lot of mozilla code will break... compare http://lxr.mozilla.org/seamonkey/search?string=appendnative but ok, will change that
Oh, jeez. Could we please file some bugs on the most egregious things there? I don't care much about extensions, but expat, modules, etc should really not be screwing up like that.... :(
Attached patch patch v3 — — Splinter Review
now uses the non-native functions.
Attachment #136559 - Attachment is obsolete: true
Comment on attachment 138080 [details] [diff] [review] patch v3 (bugs on those other issues filed) as for that freeze mentioned in comment 321: it already happens now (at the end of the download), but granted, currently the connection has no chance of timing out. But, for the connection to really time out, you probably need a file that takes several minutes to copy, and I doubt mozilla is able to download such large files anyway ;) (god, comment 321? we should disallow people other than the assignee and requestee from making comments in a bug once it reaches say 100 comments, imo)
Attachment #138080 - Flags: review?(bz-vacation)
biesi or anyone else interested: 100 comments would be too arbitrary a number. Bugs that could reduce bugspam: bug 207090 - Quiz feature for certain bugs before filing a comment bug 148563 - NOSPAM setting and warning bug 164310 - "Reason for my vote" feature bug 148564 - Ability to ignore specific bugs in prefs bug 48570 - Allow votes against a particular enhancement request bug 65801 - Allow "pending" votes or weighted votes.
Comment on attachment 138080 [details] [diff] [review] patch v3 >Index: nsExternalHelperAppService.cpp >+ rv = NS_NewLocalFileOutputStream(getter_AddRefs(mOutStream), mTempFile, >+ PR_WRONLY | PR_APPEND, 0600); Fix the indent here. r=bzbarsky with that.
Attachment #138080 - Flags: review?(bz-vacation) → review+
Attachment #138080 - Flags: superreview?(darin)
Comment on attachment 138080 [details] [diff] [review] patch v3 >Index: nsExternalHelperAppService.cpp >+ // Get the old leaf name and append .part to it >+ nsAutoString name; >+ mFinalFileDestination->GetLeafName(name); >+ name.Append(NS_LITERAL_STRING(".part")); >+ movedFile->SetLeafName(name); >+ >+ nsCOMPtr<nsIFile> dir; >+ movedFile->GetParent(getter_AddRefs(dir)); >+ >+ mOutStream->Close(); >+ >+ rv = mTempFile->MoveTo(dir, name); how about using the "native" versions of the nsIFile methods to avoid extra native <-> unicode conversions. sr=darin with that change.
Attachment #138080 - Flags: superreview?(darin) → superreview+
>if native encoding is not ascii compatible, a lot of mozilla code will break... >compare http://lxr.mozilla.org/seamonkey/search?string=appendnative > >but ok, will change that nsIFile's concept of "native" encoding is assumed to be a superset of ASCII, so appending an ASCII string is ok. moreover, we control the definition of the "native" encoding, so if necessary we could always force the "native" encoding to be a superset of ASCII (e.g., UTF-8).
i should also add that the "native" encoding is compatible with PR_Open and what PR_GetEnv returns. it therefore cannot contain embedded nulls, so UTF-16 is not a possible "native" encoding. anyways, none of the STDC library routines would work with UTF-16, so that really is not something we should worry about.
> nsIFile's concept of "native" encoding is assumed to be a superset of ASCII Assumed where? nsIFile.idl certainly doesn't assume it... There are also other non-ascii-compatible encodings that do not involve embedded nulls (EBCDIC, to be silly). If this is in fact a safe assumption to make we need to clearly document that in nsIFile.idl so nsIFile implementors (anyone can drop in, eg, a ftp-based impl of non-local nsIFiles) won't break this assumption....
> Assumed where? nsIFile.idl certainly doesn't assume it... we assume that the "native" multi-byte encoding is a superset of ASCII. we assume this when working with PR_GetEnv/PR_SetEnv, PR_Open, etc. this same assumption applies to nsIFile. how many times have you seen someone call AppendNative, SetNativeLeafName, etc. passing an ASCII valued string? in fact, you see this quite a lot. while it may limit mozilla from working with some very obscure multi-byte encodings, that limitation is assumed to be acceptable for the entire product. i'm not arguing against documenting this fact in nsIFile. indeed, nsIFile could use a lot more documentation.
Sure, I would be fine with that (updating the documentation in nsIFile to clearly say this). Would you mind doing that? Or should I?
ok, I now checked the patch in using the native functions. I filed Bug 231243 to document that nsIFile's native functions are ascii-compatible
Status: ASSIGNED → RESOLVED
Closed: 21 years ago
Resolution: --- → FIXED
*** Bug 233435 has been marked as a duplicate of this bug. ***
*** Bug 242291 has been marked as a duplicate of this bug. ***
I've been using the Internet since ARPANET days, and this one isn't a real problem. Download to the default download directory under a generated name, and rename or delete or copy as needed. If the instantaneous start of download is taken away, there will be no reason to use Mozilla's download function at all. As it is, most small files are downloaded before a name is selected, making them essentially without cost. What could be better. Those who want downloads to wait to start until OK'd should be able to turn that off. Now, if the download manager actually displayed URL and allowed repeated downloads or controllable restart of download...that is to say, it actually worked, all of us would love it.
(In reply to comment #338) > I've been using the Internet since ARPANET days, and this one isn't a real > problem. Well, /tmp may not have enough space for the file. And, when you save to /tmp, you can't really implement resuming, because /tmp tends to get deleted. > Now, if the download > manager actually displayed URL column picker -> source > and allowed repeated downloads or controllable > restart of download...that is to say, it actually worked, all of us would love it. there are bugs filed on this already.
*** Bug 242679 has been marked as a duplicate of this bug. ***
Just a few observations about the differences in the downloads between Firefox and Mozilla in OS X. I have noticed that Mozilla appears to be downloading into the temp file while I am going through the process of selecting the target destination of the download. Occasionally I will have a completed download of X number of seconds (say 20 seconds) when I finally click on the destination when using Mozilla. On the other hand Firefox does not appear to download in this manner as it only begins after I click on the final destination of the download. The other difference that is quite noticeable when comparing downloads between the two is that Firefox starts out at quite a fast download rate and then typically tapers off to something lower as a "typical" download rate. (It is impressive how fast some of the Firefox downloads are.) Mozilla, by way of contrast, starts out rather slower and then builds to some faster "typical" download rate. Mozilla has an elapsed time for the download, but Firefox does not and so I am approximating the total download time for Firefox, but it seems to be a shorter length of time in most instances. (It would be nice to have an elapsed time feature in Firefox as well...actually, there is one browser which I can not recall that also calculates the average download rate for the completed download. It does not improve performance, but is interesting nonetheless.
No longer blocks: majorbugs
No longer blocks: 88566
Status: RESOLVED → VERIFIED
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: