261885 - Long delay &Heavy HDD use: slows complete system, when opening download manager after many Large file downloads

Status: RESOLVED DUPLICATE of bug 240525

(Toolkit :: Downloads API, defect)

Description

[Chris]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20040913 Firefox/0.10
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20040913 Firefox/0.10

Dowloading many large files in a row, creates horribly fragmented files which
results in long system delays. 

When i restarted Firefox, opened the Tools->Download box, the hdd churned
constantly for 60 seconds before any content appeared in the box.
After removing the completed download from the window (by clicking on trash
cans) the window would open instantly, obviously because it didn't have to
access my downloads directory for file information.

Most of the time I was only downloading 2 files at the same time, but I
downloaded approx 12 of them (game demos).


Reproducible: Always
Steps to Reproduce:
1.Download 6 to 10 large game demo files, 100MB+, only 2 files downloading at
any one time.
2.Close Download Manager, reopen Download Manager. Delay, Delay, Delay...Wait
for a long time....
3.Open volume/drive in "My Computer". Wait for long time....

Actual Results:  
Results in heavily fragmented files, causing system delays:

Volume DVD (F:)
    Volume size                                = 12.59 GB
    Cluster size                               = 8 KB
    Used space                                 = 9.91 GB
    Free space                                 = 2.68 GB
    Percent free space                         = 21 %


--------------------------------------------------------------------------------
Fragments       File Size       Most fragmented files
595             284 MB          \cmr2005-demo-installer.exe
386             247 MB          \CV_DEMO.EXE
262             166 MB          \cmr2005-mp-demo-installer.exe
340             171 MB          \SP2DEMOB.EXE
25              22 MB           \HEDZ.ZIP
53              319 MB          \dow_demo_install.exe
183             154 MB          \xpandrallyspseptdemo.exe
18              5 MB            \mortyr2_trailer.zip
360             231 MB          \medievallords_demo_eng.exe
170             223 MB          \trs2004_us_demo.exe
326             292 MB          \sorm_sp_demo.exe
34              43 MB           \MMO_lore_demo_v1_1_2.exe
684             510 MB          \t72_beta_demo.exe
341             374 MB          \tribesv_spdemo_en.exe
501             148 MB          \canned_with_vo_web.wmv
100             159 MB          \wowdemoengsetup.exe
133             83 MB           \shade_demo_1.0.exe
89              43 MB           \lore_demo_v1_1_2.exe
926             301 MB          \fsw_pc-demo.exe
1,138           534 MB          \moh_pacific_assault_demo.exe
366             185 MB          \rct3_demo.exe

----------------------------------------------------------------
Slowed down the computers operation every time it accessed volume F:

Opening "My Computer" and opening drive F: delayed for 1 minute just to show
contents.

(this problem would be worse with that indexing windows xp runs in the
background, which i have OFF.)
Any other background scanners would just cause more delays to system operation.

Expected Results:  
Less fragmentation, some fragmentation is always expected though.

IE saves file in temp dir, then copies the file to the destination after
downloading it, and now I realize why.


Centrino 1.5ghz Laptop
5400 rpm 40gig hdd
Cable 700KBytes/ps connection speed

Attempt at some sort of solution: 
After the file has been fully downloaded, 
the same can be copy and saved to real memory in large chunks, 
from and to the same folder, and this would reduce the fragments.

I will resort to using Getright, as they have some wat to beat this problem.

Comment 1

[Tom Leski]

I confirm - after a period of time, when many pictures are saved and the
downloaded file list grows large, there's a huge gap between the moment you
click "save" and the time the download manager actually saves the file and gives
you back the control (during this gap the system is freezed).

Comment 2

[Tom Leski]

How come it's unconfirmed yet? It's so obvious to check it out.

BTW: this would never happened if the download manager was disabled when saving
smaller files. What's the point to use download manager to save a 500K file? Or
even a 1 meg one? 

For me working with this download manager is only waste of time. 

Comment 3

[Alan Eliasen]

Confirmed too on Windows 2000.  During this time, CPU goes to 100% (for about a
minute) and locks all Firefox windows and makes all other applications unusable
during this time.  Memory use also skyrockets.

It doesn't get better after restarting Firefox.  The Downloads window still
takes forever to open.

Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0

Comment 4

[Alan Eliasen]

Further information.  Opening the "download" window took about 60 sec of CPU
time, and increased memory usage by about 30 MB.

Clicking the "clean up" button in the download window took 6 minutes of CPU
time, at which time the CPU was pegged and the system was unusable.

After "clean up" finished, the "clean up" button was disabled, but now the
download window showed a new and different list, which apparently contained
hundreds of downloads, probably everything I've downloaded or tried to download
and failed in the past year or so, some indicating that they were "starting". 
Since the "clean up" button was disabled, I had to click "cancel" for each one
(which took about 5 minutes.)  A huge number of files.

God knows what files got overwritten if those really were resuming downloads.

Comment 5

[Jo Hermans]

see the discussion in bug 251876 (which was WONTFIX'ed)

Comment 6

[Alan Eliasen]

Disk fragmentation would likely *not* cause 100% CPU usage like in this bug.  As
a general rule, heavy fragmentation causes *low* CPU usage while the system is
simply waiting for the drive to seek to the right position.

It's much more probable that some bad algorithm is taking all the time.

Comment 7

[Greg Edwards]

(In reply to comment #5)
> see the discussion in bug 251876 (which was WONTFIX'ed)

This seems to be a mixed description of two different bugs: The fragmentation
issue tagged as 'wontfix' (which could be resolved by saving a copy when the
download finishes and deleting the old version; a makeshift defrag) and another
bug, where, when the Download Manager is full of files, it take a very long time
to load. It took 17 seconds to display on my system before I clicked the Clean
Up button. An option should be added to Firefox to automatically clean up files
older than a certain date, or when a maximum number is reached. (Additionally,
there must be some severe inefficiency in the download list rendering code to
cause this extent of slowdown.)

Comment 8

[Elmar Ludwig]

There are actually three separate issue here:

- disk fragmentation as discussed in bug 251876 (WONTFIX)
- performance issues with growing number of items (bug 240525, bug 161783)
- memory usage/leak caused by the download manager (bug 273244)

Given that the first issue is WONTFIX and this report is about "Dowloading many large files in a row", I'm going to mark this as a duplicate of bug 240525 now.

*** This bug has been marked as a duplicate of 240525 ***