Last Comment Bug 159107 - page saving/downloads takes too much time (is slow) ('marooned' entries in downloads.rdf)
: page saving/downloads takes too much time (is slow) ('marooned' entries in do...
Status: VERIFIED FIXED
[adt3]
: perf
Product: SeaMonkey
Classification: Client Software
Component: Download & File Handling (show other bugs)
: Trunk
: All All
: P3 normal with 11 votes (vote)
: ---
Assigned To: Jan Varga [:janv]
: Chris Petersen
Mentors:
: 145851 146037 157864 160624 185554 187314 204099 204648 208576 214785 217403 217839 224792 227346 229626 239086 239210 242604 250267 250462 254114 254595 254649 261424 268328 285363 295001 297693 307803 311178 316175 331768 347608 350162 351136 372627 374032 415227 (view as bug list)
Depends on: 233238 161783
Blocks: 392728
  Show dependency treegraph
 
Reported: 2002-07-24 04:43 PDT by Alexander Rabtchevich
Modified: 2009-10-12 01:12 PDT (History)
68 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
Various differently sized download.rdf files (202.67 KB, application/zip)
2003-01-07 12:11 PST, Andreas Kunz
no flags Details

Description Alexander Rabtchevich 2002-07-24 04:43:02 PDT
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.1b) Gecko/20020722
BuildID:    2002072204

Saving of already loaded page to a disk takes more than 10 seconds. The PC is
K6-166, but it is too much in any case. Previously before 1.1 branch everything
was OK.

Reproducible: Always
Steps to Reproduce:
Simply save already downloaded page to a file.
Comment 1 Adam Lock 2002-07-24 12:23:48 PDT
Can you provide a sample url? Thanks
Comment 2 Boris Zbarsky [:bz] 2002-07-24 15:54:53 PDT
Are you saving "HTML only" or "complete"?
Comment 3 Alexander Rabtchevich 2002-07-24 23:07:45 PDT
It doesn't matter what type of saving is used - complete page or html only. It
doesn't matter what page is saved. For instance, this one
(http://bugzilla.mozilla.org/show_bug.cgi?id=159107) took 10+ seconds to be
saved in both modes. 
I have enabled progress dialog opening to be able to manually intercept links to
files to enter them into download manager. So it is showing for a while even if
I save already downloaded page.
Comment 4 Boris Zbarsky [:bz] 2002-07-24 23:31:48 PDT
How big is your downloads.rdf file?
Comment 5 Alexander Rabtchevich 2002-07-25 00:31:05 PDT
downloads.rdf was 667 k. The most interesting thing is after deleteng all items
in download manager the file remains 616 k. So it is not totally cleared.
After clearing download manager page saving time reduced to 8-10 seconds.

Comment 6 Boris Zbarsky [:bz] 2002-07-25 00:40:30 PDT
What about if you remove downloads.rdf altogether
Comment 7 Alexander Rabtchevich 2002-07-25 00:58:06 PDT
Additional comments: the delay is caused by the downloads.rdf file. I've deleted
it outside Mozilla (to disable it's recovery function :-)) and page saving time
decreased to 1 second.
So the issue consists in 2 items: manualy clearing the download manager inside
it doesn't delete all items from the file. 
And the second is: no one must clear this history. The file will obviously grow
with the time. So the problem will remain.
The possible solution is to split history in multiple files and write into
latest one only (as it is done in NetVampire - download manager). If one wants
to look through the whole history - create a button in download manager to load
all files consequently. 
Comment 8 Boris Zbarsky [:bz] 2002-07-25 01:04:50 PDT
confirming and nominating... we should not be letting downloads.rdf balloon this
much and even if it's big it should not take us 10 seconds to save a page.

I thought bug 142824 was supposed to fix these issues?
Comment 9 Samir Gehani 2002-07-25 14:32:58 PDT
Nav triage team: jrgm, is this defect distinct from the one fixed by bug 142824?  
Comment 10 John Morrison 2002-07-25 16:33:02 PDT
bug 142824 was to prevent that file from growing in the future. Because it was 
landing late for 1.0 (and it had been ignored/nsbeta1- three times), I wanted 
to minimize the risk, so I limited the fix to remove those items I could 
"reach" through hasMoreElements.

However, due to a defect (I believe) in RDF (which may or may not be fixed) 
some entries were "marooned" in the file. In Alexander's case, a lot of 
entries, and I doubt he's the only one in this situation. (Note: I believe
the workaround 'delete downloads.rdf' has been added to the release notes,
although that's a workaround, not a solution).

I think we should consider adding some code to clean out these "unreachable"
entries, and that we should do it in the 1.2alpha milestone. I'll take a look
at how to do this, although I have to go learn how to do it first :-)
Comment 11 Samir Gehani 2002-07-26 10:21:59 PDT
Nav triage team: nsbeta1+/adt3
Comment 12 Andreas Kunz 2003-01-07 12:10:47 PST
*** Bug 145851 has been marked as a duplicate of this bug. ***
Comment 13 Andreas Kunz 2003-01-07 12:11:37 PST
Created attachment 110879 [details]
Various differently sized download.rdf files

The results from my tests that were announced in bug 145851:
I generated some example download.rdf files (with the number of entries in
their filename, zipped in the attachment) and measured the times for saving
with each of them on my Athlon600.

Since I was using a local file, the 'Save As' dialog appeared at once, but
after clicking OK it took quite some time until the download window ("download
in separate windows") popped up (t1). The second try (t2) was faster, maybe
because downloads.rdf was already processed or modified somehow.

Entries:  Size:     t1/sec:  t2/sec:
1000	  657 kB    2	     1.x
2000	  1317 kB   6	     4
3000	  1977 kB   11	     6
4000	  2637 kB   17	     10
5000	  3297 kB   24	     15

The times are measured by looking at a watch, so don't take "15 sec" for
"15.000 sec"...

Some related bugs:
bug 132755, bug 136054 - about more or less automatic removal of entries
bug 161783 - about a better implementation

Please note that bug 142824 comment #5 mentions that "downloads.rdf is
re-written 3 times before each download. I also could observe the file flicker
some times in Windows Explorer during my tests, which usually hints at
(re-)creating of a file.
Maybe this points to some way to a temporary fix until some of the other bugs I
mentioned are fixed.
Comment 14 Jürgen Weber 2003-01-08 02:02:41 PST
*** Bug 157864 has been marked as a duplicate of this bug. ***
Comment 15 Samir Gehani 2003-01-08 13:31:26 PST
Over to Jan.
Comment 16 johann.petrak@gmail.com 2003-03-16 23:04:58 PST
The major complaint here is about the perf problem, so DUPing against bug 161783
not bug 132755.
Not vice versa because we should have specific bugs for the perf issue and the
auto-delete issue.

*** This bug has been marked as a duplicate of 161783 ***
Comment 17 John Morrison 2003-03-17 17:30:09 PST
Actually, no, this bug (comment 10) is about post-facto cleaning up the 
marooned entries that accumulated in users downloads.rdf file during april-july 
2002. So, it's not a duplicate. But it's probably not worth worrying about at 
this point, and I'm okay if varga just wants to wontfix this one. [But bug 
161783 does need some sort of fix for buffy].
Comment 18 Samir Gehani 2003-03-21 14:43:31 PST
Nav triage team: nsbeta1-
Comment 19 Rick DeBay 2003-05-17 22:16:23 PDT
*** Bug 204648 has been marked as a duplicate of this bug. ***
Comment 20 Allen Clark 2003-08-30 17:24:24 PDT
*** Bug 217839 has been marked as a duplicate of this bug. ***
Comment 21 Michael Gabriel 2003-11-02 05:38:24 PST
*** Bug 160624 has been marked as a duplicate of this bug. ***
Comment 22 Michael Gabriel 2003-11-02 05:38:28 PST
*** Bug 208576 has been marked as a duplicate of this bug. ***
Comment 23 Michael Gabriel 2003-11-02 05:42:37 PST
*** Bug 187314 has been marked as a duplicate of this bug. ***
Comment 24 Michael Gabriel 2003-11-02 05:58:56 PST
*** Bug 185554 has been marked as a duplicate of this bug. ***
Comment 25 Boris Zbarsky [:bz] 2003-11-05 08:33:27 PST
*** Bug 224792 has been marked as a duplicate of this bug. ***
Comment 26 Christian :Biesinger (don't email me, ping me on IRC) 2003-11-15 11:38:59 PST
*** Bug 217403 has been marked as a duplicate of this bug. ***
Comment 27 Christian :Biesinger (don't email me, ping me on IRC) 2003-11-16 07:35:50 PST
*** Bug 214785 has been marked as a duplicate of this bug. ***
Comment 28 Boris Zbarsky [:bz] 2003-12-29 10:30:25 PST
*** Bug 229626 has been marked as a duplicate of this bug. ***
Comment 29 Frank Wein [:mcsmurf] 2004-03-29 11:29:10 PST
*** Bug 239086 has been marked as a duplicate of this bug. ***
Comment 30 Ali Ebrahim 2004-05-04 15:25:09 PDT
*** Bug 227346 has been marked as a duplicate of this bug. ***
Comment 31 Ali Ebrahim 2004-05-04 15:28:31 PDT
*** Bug 242604 has been marked as a duplicate of this bug. ***
Comment 32 R J Gray 2004-06-26 14:20:31 PDT
(In reply to comment #13)
>
> Maybe this points to some way to a temporary fix until some of the other bugs I
> mentioned are fixed.


A quick temporary fix I've been using since I reported the bug is to replace the
file download.rdf with a symlink to /dev/null. 

Whis will, of course only work on a linux or unix syste,
RJG.
Comment 33 Thomas Lange 2004-07-02 05:28:06 PDT
I just checked, on my Win XP box setting the read-only attribute helps also.
Thus it will probably help on all OS versions.
Comment 34 Christian :Biesinger (don't email me, ping me on IRC) 2004-08-06 18:55:49 PDT
*** Bug 254595 has been marked as a duplicate of this bug. ***
Comment 35 Christian :Biesinger (don't email me, ping me on IRC) 2004-08-06 18:57:30 PDT
*** Bug 254649 has been marked as a duplicate of this bug. ***
Comment 36 Andreas Kunz 2004-08-11 05:31:04 PDT
*** Bug 204099 has been marked as a duplicate of this bug. ***
Comment 37 Vincenzo 2004-08-14 20:24:39 PDT
This is still present in 0.9.2, and should probably be given a higher severity.
It is a high profile problem that rears its head when images are saved. I have
seen several people switch back to using IE because of this -- it is quite
nauseating to see it take 10 or 15 *seconds* to save a single image. Deleting
the downloads.rdf file will only generally fix this for a short time, under an
hour of normal browsing. I am not sure why this information needs to be kept in
that file, but if this problem can't be easily fixed probably the right thing to
do is remove the functionality altogether, and not update downloads.rdf at all. 
Comment 38 Valerio Capello 2004-08-15 00:24:52 PDT
Workarounds suggested by R J Gray (Linux) and Thomas Lange (for Windows systems,
may apply to other OSes) work fine.

As for the fix: there could be an option to limit the file to N entries. 100
entries should be enough to keep track of what you did download recently without
losing speed. Also, you can set it to 0 in case you don't want to keep a log of
the stuff you've downloaded.
Comment 39 Michael Gabriel 2004-08-21 02:40:28 PDT
*** Bug 146037 has been marked as a duplicate of this bug. ***
Comment 40 Tim Powell 2004-09-30 08:20:06 PDT
*** Bug 254114 has been marked as a duplicate of this bug. ***
Comment 41 Kevin Brosnan 2004-11-20 22:39:31 PST
*** Bug 261424 has been marked as a duplicate of this bug. ***
Comment 42 OstGote! 2004-11-25 02:58:10 PST
*** Bug 250462 has been marked as a duplicate of this bug. ***
Comment 43 darren 2004-12-18 18:01:02 PST
Just a comment to perhaps induce a higher priority... This problem causes
firefox to hang, and requires a "kill -9" to stop firefox.

When firefox is installed, and mozilla already existed, it apparently makes a
copy of all the moz files, including downloads.rdf. 

Unlike mozilla, firefox apparently isn't able to deal with these files. Running
strace(1) on firefox shows that after you click "save," it reads in the whole
downloads.rdf file, closes the file handle, and then does nothing. No other
output from strace. I admit I only waited about 40 seconds before killing it,
but moz only takes ~ 20 seconds to save a file.

Firefox version is 1.0PR in this case. The .mozilla/firefox/.../downloads.rdf
file is 3.4 Mb.

Cheers,
-Darren
Comment 44 OstGote! 2005-02-03 23:06:25 PST
*** Bug 239210 has been marked as a duplicate of this bug. ***
Comment 45 R.K.Aa. 2005-03-08 19:38:37 PST
*** Bug 285363 has been marked as a duplicate of this bug. ***
Comment 46 OstGote! 2005-05-27 10:27:05 PDT
*** Bug 295001 has been marked as a duplicate of this bug. ***
Comment 47 Michael Gabriel 2005-08-01 12:39:12 PDT
*** Bug 268328 has been marked as a duplicate of this bug. ***
Comment 48 Ryuichi KUBUKI 2005-08-09 04:51:27 PDT
*** Bug 250267 has been marked as a duplicate of this bug. ***
Comment 49 OstGote! 2005-09-11 10:46:56 PDT
*** Bug 307803 has been marked as a duplicate of this bug. ***
Comment 50 OstGote! 2005-10-12 12:51:17 PDT
*** Bug 311178 has been marked as a duplicate of this bug. ***
Comment 51 OstGote! 2005-11-21 15:17:19 PST
*** Bug 316175 has been marked as a duplicate of this bug. ***
Comment 52 Steve England [:stevee] 2006-03-26 08:15:17 PST
*** Bug 331768 has been marked as a duplicate of this bug. ***
Comment 53 tOM Trottier 2006-04-08 13:07:02 PDT
     Additional workaround. 
Clicking "Clean Up" in the downloads screen speeds up saves without having to exit and delete downloads.rdf.

tOM
Comment 54 Andreas Morawietz 2006-06-20 09:13:01 PDT
Problem is still there in Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1a3) Gecko/20060616 BonEcho/2.0a3 ID:2006061603

This should be fixed for firefox 2.0 final!
Comment 55 Ryan Flint [:rflint] (ping via IRC for reviews) 2006-07-15 14:01:48 PDT
*** Bug 297693 has been marked as a duplicate of this bug. ***
Comment 56 Nick Thomas [:nthomas] 2006-08-06 05:31:50 PDT
*** Bug 347608 has been marked as a duplicate of this bug. ***
Comment 57 Mats Lindhé 2006-08-15 12:38:24 PDT
I had a downloads.rdf file only 32k big, but somehow it hindered downloads altogether. Everything seemed OK, I was asked for name and folder, but no download occured and the downloader window never appeared. I deleted the downloads.rdf file (without restarting Firefox) and everything is working again!
Comment 58 Jerry 2006-08-17 17:10:37 PDT
Thunderbird version 1.0.7 build (20050923)
Unable to download new emails into my in-box. The in box folder is very slow to display email messages and each time I try the email counter adds more messages.
eg. presently I have 16 emails but the counter reads 211. The other file folders are fine. I can write and send new email messages no problem. A blue bar graph runs each time I attempt a download. Incoming mail server is correct (pop.telus.net). This is 3rd day of my problem.
Comment 59 paolo 2006-08-18 00:26:56 PDT
1.0.7? I'd rather give at least latest 1.0.8 a try.
Comment 60 hajma 2006-08-20 06:06:44 PDT
still here in FF 1.5.0.6 on linux.
It is really serious.
Comment 61 Adam Guthrie 2006-09-03 16:13:08 PDT
*** Bug 350162 has been marked as a duplicate of this bug. ***
Comment 62 Adam Guthrie 2006-09-03 16:14:15 PDT
*** Bug 351136 has been marked as a duplicate of this bug. ***
Comment 63 artorrijos 2006-09-04 01:08:26 PDT
I think the problem is not the time in seconds that Firefox takes to download an image/file/page. The major problem is that Firefox becomes unresponsive at the moment when the download is initiated. This has been my experience with Firefox on Windows on several computers. I think it's a pretty important problem to look into especially from user/usability view.


(In reply to comment #43)
> Just a comment to perhaps induce a higher priority... This problem causes
> firefox to hang, and requires a "kill -9" to stop firefox.
> 
> When firefox is installed, and mozilla already existed, it apparently makes a
> copy of all the moz files, including downloads.rdf. 
> 
> Unlike mozilla, firefox apparently isn't able to deal with these files. Running
> strace(1) on firefox shows that after you click "save," it reads in the whole
> downloads.rdf file, closes the file handle, and then does nothing. No other
> output from strace. I admit I only waited about 40 seconds before killing it,
> but moz only takes ~ 20 seconds to save a file.
> 
> Firefox version is 1.0PR in this case. The .mozilla/firefox/.../downloads.rdf
> file is 3.4 Mb.
> 
> Cheers,
> -Darren
> 

Comment 64 Jean-Michel Reghem 2006-09-08 03:13:37 PDT
Maybe we have to change Product "Mozilla Application Suite" to "Firefox" or a Core component or open a new bug for firefox ... as many bug concerning Firefox/Download Manager are resolved as dup of this one (for example: bug 311178), but this one is (for historical reasons) concerning Mozilla Application Suite and thus, Seamonkey ...
Is this bug really present in SeaMonkey?
Comment 65 Charles Evans 2007-03-06 11:53:14 PST
Same problem with open file - I had to cancel the file open after many minutes -
sometimes hitting X on the open dialog works, but once it locked up even another konqueror file browser window looking at the profile dir. Had to kill firefox.
could not even change dirs - dir has less than 100 files, but one is a big .iso, also 37 subdirs
I have bookmarked file:/// - that works fine! Open file, change dir instantly!
after deleting the apparently empty downloads.rdf, file open works fine.
This is a MAJOR bug until one knows the trick - I already knew to clean the downloads daily, but knowing that did not help, and finding this took awhile.
Please put a warning on the mozilla.com download page!
Comment 66 Charles Evans 2007-03-09 17:58:26 PST
IMHO this is a CRITICAL bug! 
deleting downloads.rdf, uncheck remember what I've downloaded,
worked for several hours, then extreme slowness, then total lockup!
had to kill ffox, delete the 900 byte downloads.rdf, restart.
very nearly DATALOSS - could not even crash recover my session 
normally this time - glad I had just backed up the dir minutes before
Comment 67 Adam Guthrie 2007-03-15 08:33:33 PDT
*** Bug 374032 has been marked as a duplicate of this bug. ***
Comment 68 Shawn Wilsher :sdwilsh 2007-05-14 14:52:16 PDT
Should be fixed with Bug 380250
Comment 69 Shawn Wilsher :sdwilsh 2007-06-13 14:05:31 PDT
*** Bug 372627 has been marked as a duplicate of this bug. ***
Comment 70 James.Schatzman 2007-06-24 06:46:10 PDT
Firefox 2.0.0.4 has the sometimes severe/fatal problems that

1) It tries to reload pages when you use "save as" instead of getting them from cache. Sometimes this fails (e.g., for certain types of dynamic content), and sometimes this simply represents a delay.

2) The download manager gets INCREDIBLY slow if the queue of already downloaded files is long.
Comment 71 Eep² 2007-07-10 01:33:16 PDT
This happens for me in Firefox 2.0.0.4 (and earlier versions back to 1.5.x at least) even IF I delete downloads.rdf (which mine was over 3MB before I just deleted it).
Comment 72 Jo Hermans 2008-02-01 10:44:26 PST
*** Bug 415227 has been marked as a duplicate of this bug. ***
Comment 73 Tony Mechelynck [:tonymec] 2008-04-04 17:21:45 PDT
I've had problems in the past, both in Firefox and in SeaMonkey, when my list of finished downloads was quite long. Clearing the list always cured the problem.

At the moment, my Sm download manager is empty and my downloads.rdf is 206 bytes long, as follows:

<?xml version="1.0"?>
<RDF:RDF xmlns:NC="http://home.netscape.com/NC-rdf#"
         xmlns:RDF="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <RDF:Seq RDF:about="NC:DownloadsRoot">
  </RDF:Seq>
</RDF:RDF>

which I suppose is normal.

I would expect this bug to belong in "Core" but I can't find an appropriate component. In the meantime, I notice that this bug, supposedly a "Mozilla Application Suite" bug, is also linked from the entry for downloads.rdf in http://kb.mozillazine.org/Profile_folder_-_Firefox and not only from the SeaMonkey one.
Comment 74 Shawn Wilsher :sdwilsh 2008-04-04 17:46:27 PDT
as stated in comment 68, this is fixed (for Firefox, and any Toolkit consuming applications using 1.9) by bug 380250.  Note, this fix will not be backported to branch.
Comment 75 Eep² 2008-04-04 21:11:54 PDT
(In reply to comment #71)
> This happens for me in Firefox 2.0.0.4 (and earlier versions back to 1.5.x at
> least) even IF I delete downloads.rdf (which mine was over 3MB before I just
> deleted it).

I think I have resolved the problem, which didn't have to do with Firefox but, instead, because I had "Allow Indexing Service to index this disk for fast file searching" unchecked/disabled on all my hard drives. Once I checked/enabled this option (despite the indexing service NOT running), file access was much faster when opening/saving and browsing storage devices.
Comment 76 Antti Pokki 2009-02-05 05:06:14 PST
I have intalled Firefox 3.06 today.
Please, accept non IT text below, I am no specialist.

After that downloading of a web file has become "complicated".
Downloading window and doewnloading picture is very long time on screen - perhaps forever, informing that virus checking is going on.
When I stop the process, the whole Firefox goes to end, BUT the downloaded file is on my disc.
I have never had this problem with earlier versions of Firefox.

I guess the problem may be inadequate cooperation between my virus program F-Secure (F-Secure Client Security 7.12 build 108 with all subsequent updates).

Is this a bug in Firefox or a problem in F-Secure?
Comment 77 Antti Pokki 2009-02-05 07:53:38 PST
(In reply to comment #76)
I had the below problem 2 first times when using the newest versionf of Firefox.
Now I have downloaded 3 more larger files without problems.
Therefore this bug info may be cancelled.

> I have intalled Firefox 3.06 today.
> Please, accept non IT text below, I am no specialist.
> 
> After that downloading of a web file has become "complicated".
> Downloading window and doewnloading picture is very long time on screen -
> perhaps forever, informing that virus checking is going on.
> When I stop the process, the whole Firefox goes to end, BUT the downloaded file
> is on my disc.
> I have never had this problem with earlier versions of Firefox.
> 
> I guess the problem may be inadequate cooperation between my virus program
> F-Secure (F-Secure Client Security 7.12 build 108 with all subsequent updates).
> 
> Is this a bug in Firefox or a problem in F-Secure?
Comment 78 :Hb 2009-10-12 01:12:56 PDT
VERIFIED

Note You need to log in before you can comment on or make changes to this bug.