Closed Bug 159107 Opened 18 years ago Closed 13 years ago

page saving/downloads takes too much time (is slow) ('marooned' entries in downloads.rdf)

Categories

(SeaMonkey :: Download & File Handling, defect, P3)

defect

Tracking

(Not tracked)

VERIFIED FIXED

People

(Reporter: alexander.v.rabtchevich, Assigned: janv)

References

(Depends on 1 open bug)

Details

(Keywords: perf, Whiteboard: [adt3])

Attachments

(1 file)

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.1b) Gecko/20020722
BuildID:    2002072204

Saving of already loaded page to a disk takes more than 10 seconds. The PC is
K6-166, but it is too much in any case. Previously before 1.1 branch everything
was OK.

Reproducible: Always
Steps to Reproduce:
Simply save already downloaded page to a file.
Can you provide a sample url? Thanks
Are you saving "HTML only" or "complete"?
It doesn't matter what type of saving is used - complete page or html only. It
doesn't matter what page is saved. For instance, this one
(http://bugzilla.mozilla.org/show_bug.cgi?id=159107) took 10+ seconds to be
saved in both modes. 
I have enabled progress dialog opening to be able to manually intercept links to
files to enter them into download manager. So it is showing for a while even if
I save already downloaded page.
How big is your downloads.rdf file?
downloads.rdf was 667 k. The most interesting thing is after deleteng all items
in download manager the file remains 616 k. So it is not totally cleared.
After clearing download manager page saving time reduced to 8-10 seconds.

What about if you remove downloads.rdf altogether
Additional comments: the delay is caused by the downloads.rdf file. I've deleted
it outside Mozilla (to disable it's recovery function :-)) and page saving time
decreased to 1 second.
So the issue consists in 2 items: manualy clearing the download manager inside
it doesn't delete all items from the file. 
And the second is: no one must clear this history. The file will obviously grow
with the time. So the problem will remain.
The possible solution is to split history in multiple files and write into
latest one only (as it is done in NetVampire - download manager). If one wants
to look through the whole history - create a button in download manager to load
all files consequently. 
confirming and nominating... we should not be letting downloads.rdf balloon this
much and even if it's big it should not take us 10 seconds to save a page.

I thought bug 142824 was supposed to fix these issues?
Status: UNCONFIRMED → NEW
Ever confirmed: true
Keywords: nsbeta1
OS: Windows NT → All
Hardware: PC → All
Nav triage team: jrgm, is this defect distinct from the one fixed by bug 142824?  
Whiteboard: [need info]
bug 142824 was to prevent that file from growing in the future. Because it was 
landing late for 1.0 (and it had been ignored/nsbeta1- three times), I wanted 
to minimize the risk, so I limited the fix to remove those items I could 
"reach" through hasMoreElements.

However, due to a defect (I believe) in RDF (which may or may not be fixed) 
some entries were "marooned" in the file. In Alexander's case, a lot of 
entries, and I doubt he's the only one in this situation. (Note: I believe
the workaround 'delete downloads.rdf' has been added to the release notes,
although that's a workaround, not a solution).

I think we should consider adding some code to clean out these "unreachable"
entries, and that we should do it in the 1.2alpha milestone. I'll take a look
at how to do this, although I have to go learn how to do it first :-)
Summary: page saving takes too much time → page saving takes too much time ('marooned' entries in downloads.rdf)
Whiteboard: [need info]
Nav triage team: nsbeta1+/adt3
Keywords: nsbeta1nsbeta1+
Whiteboard: [adt3]
Keywords: perf
QA Contact: sairuh → petersen
*** Bug 145851 has been marked as a duplicate of this bug. ***
The results from my tests that were announced in bug 145851:
I generated some example download.rdf files (with the number of entries in
their filename, zipped in the attachment) and measured the times for saving
with each of them on my Athlon600.

Since I was using a local file, the 'Save As' dialog appeared at once, but
after clicking OK it took quite some time until the download window ("download
in separate windows") popped up (t1). The second try (t2) was faster, maybe
because downloads.rdf was already processed or modified somehow.

Entries:  Size:     t1/sec:  t2/sec:
1000	  657 kB    2	     1.x
2000	  1317 kB   6	     4
3000	  1977 kB   11	     6
4000	  2637 kB   17	     10
5000	  3297 kB   24	     15

The times are measured by looking at a watch, so don't take "15 sec" for
"15.000 sec"...

Some related bugs:
bug 132755, bug 136054 - about more or less automatic removal of entries
bug 161783 - about a better implementation

Please note that bug 142824 comment #5 mentions that "downloads.rdf is
re-written 3 times before each download. I also could observe the file flicker
some times in Windows Explorer during my tests, which usually hints at
(re-)creating of a file.
Maybe this points to some way to a temporary fix until some of the other bugs I
mentioned are fixed.
Attachment #110879 - Attachment mime type: application/octet-stream → application/zip
*** Bug 157864 has been marked as a duplicate of this bug. ***
Over to Jan.
Assignee: blaker → varga
Priority: -- → P3
Target Milestone: --- → mozilla1.4beta
The major complaint here is about the perf problem, so DUPing against bug 161783
not bug 132755.
Not vice versa because we should have specific bugs for the perf issue and the
auto-delete issue.

*** This bug has been marked as a duplicate of 161783 ***
Status: NEW → RESOLVED
Closed: 18 years ago
Resolution: --- → DUPLICATE
Actually, no, this bug (comment 10) is about post-facto cleaning up the 
marooned entries that accumulated in users downloads.rdf file during april-july 
2002. So, it's not a duplicate. But it's probably not worth worrying about at 
this point, and I'm okay if varga just wants to wontfix this one. [But bug 
161783 does need some sort of fix for buffy].
Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---
Nav triage team: nsbeta1-
Keywords: nsbeta1+nsbeta1-
*** Bug 204648 has been marked as a duplicate of this bug. ***
*** Bug 217839 has been marked as a duplicate of this bug. ***
Depends on: 161783
*** Bug 160624 has been marked as a duplicate of this bug. ***
*** Bug 208576 has been marked as a duplicate of this bug. ***
*** Bug 187314 has been marked as a duplicate of this bug. ***
Target Milestone: mozilla1.4beta → ---
*** Bug 185554 has been marked as a duplicate of this bug. ***
*** Bug 224792 has been marked as a duplicate of this bug. ***
*** Bug 217403 has been marked as a duplicate of this bug. ***
*** Bug 214785 has been marked as a duplicate of this bug. ***
*** Bug 229626 has been marked as a duplicate of this bug. ***
*** Bug 239086 has been marked as a duplicate of this bug. ***
*** Bug 227346 has been marked as a duplicate of this bug. ***
Depends on: 233238
*** Bug 242604 has been marked as a duplicate of this bug. ***
(In reply to comment #13)
>
> Maybe this points to some way to a temporary fix until some of the other bugs I
> mentioned are fixed.


A quick temporary fix I've been using since I reported the bug is to replace the
file download.rdf with a symlink to /dev/null. 

Whis will, of course only work on a linux or unix syste,
RJG.
I just checked, on my Win XP box setting the read-only attribute helps also.
Thus it will probably help on all OS versions.
Summary: page saving takes too much time ('marooned' entries in downloads.rdf) → page saving/downloads takes too much time (is slow) ('marooned' entries in downloads.rdf)
*** Bug 254595 has been marked as a duplicate of this bug. ***
*** Bug 254649 has been marked as a duplicate of this bug. ***
*** Bug 204099 has been marked as a duplicate of this bug. ***
This is still present in 0.9.2, and should probably be given a higher severity.
It is a high profile problem that rears its head when images are saved. I have
seen several people switch back to using IE because of this -- it is quite
nauseating to see it take 10 or 15 *seconds* to save a single image. Deleting
the downloads.rdf file will only generally fix this for a short time, under an
hour of normal browsing. I am not sure why this information needs to be kept in
that file, but if this problem can't be easily fixed probably the right thing to
do is remove the functionality altogether, and not update downloads.rdf at all. 
Workarounds suggested by R J Gray (Linux) and Thomas Lange (for Windows systems,
may apply to other OSes) work fine.

As for the fix: there could be an option to limit the file to N entries. 100
entries should be enough to keep track of what you did download recently without
losing speed. Also, you can set it to 0 in case you don't want to keep a log of
the stuff you've downloaded.
*** Bug 146037 has been marked as a duplicate of this bug. ***
*** Bug 254114 has been marked as a duplicate of this bug. ***
*** Bug 261424 has been marked as a duplicate of this bug. ***
Product: Browser → Seamonkey
*** Bug 250462 has been marked as a duplicate of this bug. ***
Just a comment to perhaps induce a higher priority... This problem causes
firefox to hang, and requires a "kill -9" to stop firefox.

When firefox is installed, and mozilla already existed, it apparently makes a
copy of all the moz files, including downloads.rdf. 

Unlike mozilla, firefox apparently isn't able to deal with these files. Running
strace(1) on firefox shows that after you click "save," it reads in the whole
downloads.rdf file, closes the file handle, and then does nothing. No other
output from strace. I admit I only waited about 40 seconds before killing it,
but moz only takes ~ 20 seconds to save a file.

Firefox version is 1.0PR in this case. The .mozilla/firefox/.../downloads.rdf
file is 3.4 Mb.

Cheers,
-Darren
*** Bug 239210 has been marked as a duplicate of this bug. ***
*** Bug 285363 has been marked as a duplicate of this bug. ***
*** Bug 295001 has been marked as a duplicate of this bug. ***
*** Bug 268328 has been marked as a duplicate of this bug. ***
*** Bug 250267 has been marked as a duplicate of this bug. ***
*** Bug 307803 has been marked as a duplicate of this bug. ***
*** Bug 311178 has been marked as a duplicate of this bug. ***
*** Bug 316175 has been marked as a duplicate of this bug. ***
*** Bug 331768 has been marked as a duplicate of this bug. ***
     Additional workaround. 
Clicking "Clean Up" in the downloads screen speeds up saves without having to exit and delete downloads.rdf.

tOM
Problem is still there in Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1a3) Gecko/20060616 BonEcho/2.0a3 ID:2006061603

This should be fixed for firefox 2.0 final!
*** Bug 297693 has been marked as a duplicate of this bug. ***
*** Bug 347608 has been marked as a duplicate of this bug. ***
I had a downloads.rdf file only 32k big, but somehow it hindered downloads altogether. Everything seemed OK, I was asked for name and folder, but no download occured and the downloader window never appeared. I deleted the downloads.rdf file (without restarting Firefox) and everything is working again!
Thunderbird version 1.0.7 build (20050923)
Unable to download new emails into my in-box. The in box folder is very slow to display email messages and each time I try the email counter adds more messages.
eg. presently I have 16 emails but the counter reads 211. The other file folders are fine. I can write and send new email messages no problem. A blue bar graph runs each time I attempt a download. Incoming mail server is correct (pop.telus.net). This is 3rd day of my problem.
1.0.7? I'd rather give at least latest 1.0.8 a try.
still here in FF 1.5.0.6 on linux.
It is really serious.
*** Bug 350162 has been marked as a duplicate of this bug. ***
*** Bug 351136 has been marked as a duplicate of this bug. ***
I think the problem is not the time in seconds that Firefox takes to download an image/file/page. The major problem is that Firefox becomes unresponsive at the moment when the download is initiated. This has been my experience with Firefox on Windows on several computers. I think it's a pretty important problem to look into especially from user/usability view.


(In reply to comment #43)
> Just a comment to perhaps induce a higher priority... This problem causes
> firefox to hang, and requires a "kill -9" to stop firefox.
> 
> When firefox is installed, and mozilla already existed, it apparently makes a
> copy of all the moz files, including downloads.rdf. 
> 
> Unlike mozilla, firefox apparently isn't able to deal with these files. Running
> strace(1) on firefox shows that after you click "save," it reads in the whole
> downloads.rdf file, closes the file handle, and then does nothing. No other
> output from strace. I admit I only waited about 40 seconds before killing it,
> but moz only takes ~ 20 seconds to save a file.
> 
> Firefox version is 1.0PR in this case. The .mozilla/firefox/.../downloads.rdf
> file is 3.4 Mb.
> 
> Cheers,
> -Darren
> 

Maybe we have to change Product "Mozilla Application Suite" to "Firefox" or a Core component or open a new bug for firefox ... as many bug concerning Firefox/Download Manager are resolved as dup of this one (for example: bug 311178), but this one is (for historical reasons) concerning Mozilla Application Suite and thus, Seamonkey ...
Is this bug really present in SeaMonkey?
Same problem with open file - I had to cancel the file open after many minutes -
sometimes hitting X on the open dialog works, but once it locked up even another konqueror file browser window looking at the profile dir. Had to kill firefox.
could not even change dirs - dir has less than 100 files, but one is a big .iso, also 37 subdirs
I have bookmarked file:/// - that works fine! Open file, change dir instantly!
after deleting the apparently empty downloads.rdf, file open works fine.
This is a MAJOR bug until one knows the trick - I already knew to clean the downloads daily, but knowing that did not help, and finding this took awhile.
Please put a warning on the mozilla.com download page!
IMHO this is a CRITICAL bug! 
deleting downloads.rdf, uncheck remember what I've downloaded,
worked for several hours, then extreme slowness, then total lockup!
had to kill ffox, delete the 900 byte downloads.rdf, restart.
very nearly DATALOSS - could not even crash recover my session 
normally this time - glad I had just backed up the dir minutes before
Duplicate of this bug: 374032
Should be fixed with Bug 380250
Duplicate of this bug: 372627
Firefox 2.0.0.4 has the sometimes severe/fatal problems that

1) It tries to reload pages when you use "save as" instead of getting them from cache. Sometimes this fails (e.g., for certain types of dynamic content), and sometimes this simply represents a delay.

2) The download manager gets INCREDIBLY slow if the queue of already downloaded files is long.
This happens for me in Firefox 2.0.0.4 (and earlier versions back to 1.5.x at least) even IF I delete downloads.rdf (which mine was over 3MB before I just deleted it).
Duplicate of this bug: 415227
I've had problems in the past, both in Firefox and in SeaMonkey, when my list of finished downloads was quite long. Clearing the list always cured the problem.

At the moment, my Sm download manager is empty and my downloads.rdf is 206 bytes long, as follows:

<?xml version="1.0"?>
<RDF:RDF xmlns:NC="http://home.netscape.com/NC-rdf#"
         xmlns:RDF="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <RDF:Seq RDF:about="NC:DownloadsRoot">
  </RDF:Seq>
</RDF:RDF>

which I suppose is normal.

I would expect this bug to belong in "Core" but I can't find an appropriate component. In the meantime, I notice that this bug, supposedly a "Mozilla Application Suite" bug, is also linked from the entry for downloads.rdf in http://kb.mozillazine.org/Profile_folder_-_Firefox and not only from the SeaMonkey one.
as stated in comment 68, this is fixed (for Firefox, and any Toolkit consuming applications using 1.9) by bug 380250.  Note, this fix will not be backported to branch.
Status: REOPENED → RESOLVED
Closed: 18 years ago13 years ago
Resolution: --- → FIXED
(In reply to comment #71)
> This happens for me in Firefox 2.0.0.4 (and earlier versions back to 1.5.x at
> least) even IF I delete downloads.rdf (which mine was over 3MB before I just
> deleted it).

I think I have resolved the problem, which didn't have to do with Firefox but, instead, because I had "Allow Indexing Service to index this disk for fast file searching" unchecked/disabled on all my hard drives. Once I checked/enabled this option (despite the indexing service NOT running), file access was much faster when opening/saving and browsing storage devices.
I have intalled Firefox 3.06 today.
Please, accept non IT text below, I am no specialist.

After that downloading of a web file has become "complicated".
Downloading window and doewnloading picture is very long time on screen - perhaps forever, informing that virus checking is going on.
When I stop the process, the whole Firefox goes to end, BUT the downloaded file is on my disc.
I have never had this problem with earlier versions of Firefox.

I guess the problem may be inadequate cooperation between my virus program F-Secure (F-Secure Client Security 7.12 build 108 with all subsequent updates).

Is this a bug in Firefox or a problem in F-Secure?
(In reply to comment #76)
I had the below problem 2 first times when using the newest versionf of Firefox.
Now I have downloaded 3 more larger files without problems.
Therefore this bug info may be cancelled.

> I have intalled Firefox 3.06 today.
> Please, accept non IT text below, I am no specialist.
> 
> After that downloading of a web file has become "complicated".
> Downloading window and doewnloading picture is very long time on screen -
> perhaps forever, informing that virus checking is going on.
> When I stop the process, the whole Firefox goes to end, BUT the downloaded file
> is on my disc.
> I have never had this problem with earlier versions of Firefox.
> 
> I guess the problem may be inadequate cooperation between my virus program
> F-Secure (F-Secure Client Security 7.12 build 108 with all subsequent updates).
> 
> Is this a bug in Firefox or a problem in F-Secure?
VERIFIED
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.