142824 - Downloads.rdf keeps on growing

Status: VERIFIED FIXED

(SeaMonkey :: Download & File Handling, defect)

Description

[Frederic Bezies]

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.0+)
Gecko/20020507
BuildID:    2002050705

I download about 20 or 30 files a day. My download.rdf files keeps on growing,
even after I clear downloaded file list.

This behaviour happens since 2 or 3 days.

Reproducible: Always
Steps to Reproduce:
1.Download some files
2.clear downloaded files list


Actual Results:  download.rdf keeps on growing

Expected Results:  download.rdf should be more little in size as file list is
cleared.

Comment 1

[Andrew Hagen]

Downloads.rdf can be viewed in a text editor. It stores a reference to all saved
files. It never clears. 

Is anyone seeing this on a platform/OS other than Windows?

Comment 2

[Peter Trudelle]

nsbeta1- per nav triage team

Comment 3

[John Morrison]

pulling back for retriage. 

blaker: how much work is it to remove resources that are no longer on the 
active (UI-visible) list of downloads?

Comment 4

[Samir Gehani]

Nav triage team: nsbeta1-

Comment 5

[Marco Lamberto]

Downloads.rdf it's re-written 3 times while saving a file!!
This stuff slows down (freezes) the whole browser!!!

Comment 6

[Marco Lamberto]

This problem affects Mozilla 1.0.0 too.

Comment 7

[John Morrison]

I have a patch that removes the resource as well when it is removed from the 
Seq in RemoveDownload.

Removing nsbeta1- for re-re-triage. I don't see how it is preferable to just
leave dead content on disk forever.

Comment 8

[John Morrison]

Attachment: RDF flailings; unassert resource arcs when removing from Seq

I have a suspicion that there might be a simpler way (e.g., an existing utility
method) to do this, but this does the job.

(It doesn't address cleaning up existing downloads.rdf, and I'm not sure 
what (if) to do about that).

Comment 9

[Paul Wyskoczka]

adding nsbeta1- per nav triage team.  Let's fix it for the next release.

Comment 10

[John Morrison]

I don't think you appreciate the risk and consequences of _not_ removing 
stale resource entries from downloads.rdf.

Consider a user who has ever downloaded 1000 items with the download manager
ever in the lifetime of their use of the browser. That is not what a typical
user will do, but it is not out of the realm of possible use. (Note that this
bug report begins with the assertion that the user downloads 20 to 30 files 
per day, and at that rate, this is less than 2 months of usage).

Now, even when the user removes the items from the visible UI, they are not
removed from the downloads.rdf and these dead resources will continue to be
maintained by the RDF datasource (i.e., read from and written to disk file and 
manipulated in memory).

With 1000 dead entries in downloads.rdf, on my 500MHz/128MB/win2k system, it 
now takes 3 seconds longer to open the download manager with noticeable lockup 
of the UI, it takes over one second to delete an item (batching or no batching) 
and it consumes an additional 1.0 megabytes of RAM.

There is zero benefit to the end user, there is measurable cost for the end 
user, and there is an unknown amount of stability risk in _not_ removing stale 
entries. Fixing this in the next release is too late.

Comment 11

[Ben Goodger (use ben at mozilla dot org for email)]

Comment on attachment 87889 [details] [diff] [review]
RDF flailings; unassert resource arcs when removing from Seq

sr=[e-mail]

Comment 12

[Blake Ross]

Comment on attachment 87889 [details] [diff] [review]
RDF flailings; unassert resource arcs when removing from Seq

r=blake

Comment 13

[Samir Gehani]

Nav triage team: nsbeta1+, adt3 rtm

Comment 14

[Blake Ross]

fixed on trunk.

Comment 15

[Asa Dotzler [:asa]]

Comment on attachment 87889 [details] [diff] [review]
RDF flailings; unassert resource arcs when removing from Seq

a=asa (on behalf of drivers) for checkin to the 1.0.1 branch.

Comment 16

[Asa Dotzler [:asa]]

please check this into the 1.0.1 branch and change the mozilla1.0.1+ keyword to
"fixed1.0.1" 

Comment 17

[Blake Ross]

Fixed on branch.

Comment 18

[sairuh (rarely reading bugmail)]

vrfy'd fixed using trunk commercial builds (2002.06.20.0x) on linux rh7.2, win2k
and mac 10.1.5.

1. remove downloads.rdf for the profile being tested.
2. start browser, save a couple of .html files.
3. view downloads.rdf, observe that there are RDF:Description for each file.
4. go to download manager window and delete both file entries.
5. go back and view downloads.rdf again.

results (expected): RDF:Description for each file deleted has been removed.

Comment 19

[Matthias Versen [:Matti]]

*** Bug 157491 has been marked as a duplicate of this bug. ***

Comment 20

[sairuh (rarely reading bugmail)]

i'll wait till bug 153480 is fixed on the branch (keep getting erratic crashes
because of it)...

Comment 21

[sairuh (rarely reading bugmail)]

vrfy'd fixed on the BRANCH using 2002.07.22 commercial branch builds on linux
rh7.2, win2k and mac 10.1.5. tested used steps in comment 18.

Comment 22

[Matthias Versen [:Matti]]

*** Bug 161578 has been marked as a duplicate of this bug. ***

Comment 23

[Oliver Kluge]

downloads.rdf is also growing if you do not use the download manager. I verified
this on Mozilla/5.0 (OS/2; U; Warp 4.5; en-US; rv:1.1) Gecko/20020827.

I wondered for a long time why downloads kept going slower and slower. I used
download manager only for (few) very large files, I never realized that each and
every download gets listed there (why?).

For some time I believed the slow speed was due to my large and fragmented HPFS
disks which were nearly full. But after upgrading to 27 GByte of LVD Ultra-SCSI
my disks should not be a problem any longer, but still downloads were slow.

Finally it took up to two seconds after launching a download that the Save As
dialog came up and after confirming of download location the machine spent up to
six seconds at 100 percent CPU load (on a Pentium-III 450 MHz with 256 MByte
RAM) which of course locked up the entire machine.

Today I suspected download manager and found several hundred files in there
(yes, this is achievable by normal people...). Marking one and deleting it took
two seconds at 100 percent CPU load, marking all of them and deleting them (I
miss a button to do that!) was something that I stopped by killing Mozilla after
20 minutes (!) at 100 percent CPU load.

So while the bug as described here initially may be fixed (items removed from
the UI actually disappear from downloads.rdf) I suggest added a size or number
of entries limitation into Edit | Preferences | Downloads. Furthermore I suggest
adding a "delete all entries" button and I suggest that only downloads done in
download manager be listed at all. Otherwise, all Mozillas out there will
eventually run out of power...

Comment 24

[John Morrison]

Yes, you raise a valid concern. There is a bug open which proposes three 
solutions to this problem, including (essentially) two of the suggestions
you've made here. See bug 161783, which is scheduled to be fixed.