285490 - Lockup, cpu usage at 99%, high memory at 245MB or greater when downloading many files.

Status: RESOLVED DUPLICATE of bug 251380

(Toolkit :: Downloads API, defect)

Description

[Nate Homier]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.6) Gecko/20050223 Firefox/1.0.1
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.6) Gecko/20050223 Firefox/1.0.1

When I go to http://hubblesite.org/newscenter/newsdesk/archive/releases/
And I select a News releases by year: any year.  I proceed to select the images
that are astronomical not others like illustration or art.  I normally select
the high resolution .tif files and if available the desktop wallpaper versions
at the size of 1024x768. I also have Download sort 2.5.0 set to filter .tif and
.jpg files to a specific folder. What happens is that I started to download
every .tif and sometimes .jpg files to a specific folder by right clicking and
selecting save image as... in the context menu. When I select the save image
as... in the context menu Download Sort 2.5.0 then sorts the files. Then the
download manager pops up and then I minimize it. I continue to download file
after file concurrently with as many as 16 going on at once. Year 2001 and 2002
gave me the biggest trouble.  After downloading for a while Firefox seems to
become sluggish, like when I use my Intellimouse Explorer 4.0 wired side keys to
go forward and back there seems to be a delay of 1 3 seconds. If I try to save a
file a second time then the save as dialog box also is sluggish 1 to 5 seconds.
 Eventually Firefox or the download manager locks up with 99% cpu usage and with
a memory foot print of 245MB or larger up to 260MB. When the browser stops
responding I will try to minimize Firefox and look at the download manager and
all I see is a white or grey space along with the Dos like icon in the uper left
corner. If I try to close the download manager then windows will popup with
"this program is not responding" and if I terminate the download manager then
Firefox is terminated as well. When I terminate Firefox I lose any remaining
downloads or the downloads are corrupted. The cpu and memory claims are taken
from Microsoft windows task manager. I will terminate Firefox when this happens
and then restart. Then I will proceed as usual but this time I will keep the
number of concurrent downloads at 8 or fewer. This seems to help greatly. I also
use Tweak Network 1.0 and I have it set to Max connetions 60, Max connetions per
server 24, Max persistent connetions per server 24, Pipelining is checked,
Pipelining maxrequest 30.  Also My Windows XP SP2 global connections limit is
set to 500. My ADSL connection is 1.5mb. 

Reproducible: Always

Steps to Reproduce:
1.Go to http://hubblesite.org/newscenter/newsdesk/archive/releases/ and in the
News releases by year: box select 2001 or 2002.
2. in the box (Thumbnails: Displays images only) check it. Then at the bottom of
the page in the box (Results per page:) select 100. Then proceed to download
every .tif file and when available desktop wallpaper at 1024x768
3. Do this as fast as you can. Keep the number of concurrent downloads at 15 or
higher. Don't stop until you have gone through all of 2001 and 2002. This should
cause your Firefox browser to become slugish and eventually result in a lockup
with 99% cpu usage and 245MB memory foot print. You will lose any remaining
downloading files or they will be corupted.

Actual Results:  
Firefox became very slugish 2/3 thirds of the way through 2002 and then locked
up.  I proceeded to minimize Firefox and look at the download manager but the
download manager box was all white or grey, my eyes can't be sure. I tried to
click the  x close button in the upper right coner but Microsoft windows poped
up with a box that said "this program has stopped responding" I cliked to
terminate the program and it terminated the download manager and Firefox. The
remaining downloading files were lost or corupted.

Expected Results:  
Firefox should have downloaded normally. Meaning with a reasonable memory
footprint of less than 150MB and cpu usage of less that 75% at least. Firefox 
should not become slugish. The download manager window should not become a blank
white or grey box with a Dos like icon in the upper left corner.

Default Theme, Windows XP SP2, 1.8AGHZ Intel P4, 1GB RAM,

Comment 1

[Darin Fisher]

I wonder if this might be "Core: File Handling" instead.

Comment 2

[Nate Homier]

(In reply to comment #1)
> I wonder if this might be "Core: File Handling" instead.

Well I thought it should be download manager because it seems to be fine on
fewer concurrent downloads. I might be suggesting that Firefox's cache or
pagefile writes are too few in between maybe? Instead of writing the data at
regular intervals to keep RAM memory low, it keeps it in RAM too long? Then you
have the cpu dealing also with the massive RAM data and just trying to
coordinate everything. I thought file handling refered to external programs for
certain Internet protocols or on how to handle them internally?

Comment 3

[Nate Homier]

I want to clarify my supposed bug report. When I download the images,
specifically (.tif) (.jpg) I "left Click" on the (.tif) files and then "Download
Sort 2.5.0" sorts (.tif) into the appropiate windows folder.  (.jpg) files I use
"left click" which then causes Firefox to begin downloading and displaying the
(.jpg) file. However, I do not wait for the (.jpg) file to complete downloading
and display, instead as soon as I see a sliver of the (.jpg) file (image) I
"right click" select save image as... from the context menu. Download Sort 2.5.0
then sorts the (.jpg) file in the correct windows folder. The key thing is as
soon as I have clicked I move on. So I left click on the .tif file then right
away I hit the mouses back button etc. Every thing else in the bug report
remains the same.

Comment 4

[Tom Maddox]

I have what appears to be additional data on this bug.  I had to grab a bunch of
RPMs from different rpmfind.net mirrors.  Eventually, Firefox hung during
download; the whole application became unresponsive, and I had to kill it from
the Windows Task Manager.  Before I did so, however, I noticed that Firefox's
memory usage was at about 220 MB, and its CPU usage was at 99%.  I kept the task
manager open when I relaunched Firefox and started downloading again.  Memory
usage climbed rapidly as I downloaded more files, quickly passing 70 MB. 
Closing the Download Manager got me back down to under 60 MB, and reopening it
didn't cause the memory usage to spike again, so I would surmise that Firefox is
caching the downloaded files in RAM as long as the Download Manager is open. 
Also note that I'm generally downloading the files serially instead of in parallel.

OS is Windows XP SP2, running on an IBM ThinkPad R51 with 512 MB of RAM, Firefox
version is 1.0.1.

Comment 5

[Nate Homier]

I have tested Bug 285490 on the trunk build March 15, 2005:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b2) Gecko/20050315
Firefox/1.0+

I still have the same problem as described in bug 285490. I will test "trunk
build March 16, 2005" when it is released using the conditions described in
comment #4 as posted by Tom Maddox.

Comment 6

[Nate Homier]

Hi everybody, I have tested some more. I tested on March 16, 2005 using the
trunk build March 16, 2005 Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
rv:1.8b2) Gecko/20050316 Firefox/1.0+

I tried to test like I said I would in comment #5 in response to comment #4 but
the site rpmfind.net was very slow it seemed on ftp. I did not test the http
download at rpmfind.net. So instead I created a new test again at
http://hubblesite.org/newscenter/newsdesk/archive/releases/ But this time I
completely erased Firefox from my computer meaning the profile and programs
folder. Then I reinstalled the March 16 build in a completely default mode with
no extentions. This means I accepted whatever the installer defaulted too, like
install locations and packages ect. Then I said no to importing IE favorites.
The only thing I changed was in Tools > Options > Downloads > Dowload Folder =
save all files to this folder E:\test space . Otherwise every thing else was
default install. Then I started to download like I described in the original bug
285490 but this time it took all of the tif files from the years 2002 & 2001
before it hanged. I made sure this install was clean and default every thing
except as noted above. I will see if I can add some screen shots using the
attachment tool. The screen shots show the hang as well as the read outs of XP
Taskmanager tabs Applications, Processes, Performance. I think I have done all I
can as a end-user to help the developers. However if you think of something else
I can provide let me know! I will continue to monitor supposed bug 285490 and if
I ever get more useful info I will pass it on. I just want to say that as a
result of all of this I will from now on only be using the nightly trunk builds
of Firefox. I have found computing on the edge far too exciting!! I even tried
to compile my own build today using the info from
http://gemal.dk/mozilla/build.html I failed of course will try again. Thank you.
 Sincerly Nate Homier.

Comment 7

[Nate Homier]

Attachment: Screenshot of Firefox

Notice the DOS icon in extreme upper left hand corner.

Comment 8

[Nate Homier]

Attachment: Screenshot XP Task manager

Shows XP Task Managers tab processes

Comment 9

[Nate Homier]

Attachment: Screenshot XP Task manager tab performance

Showing tab performance in XP Task Manager

Comment 10

[Ali Ebrahim]

Nate, screenshots of the task manager are quite unnecessary.

Comment 11

[Nate Homier]

I am very sorry for the attactments. My bad. I tried to delete them but I gues
you can't once you upload. No more attachment I promise!

Comment 12

[Tristor]

This appears to be a dupe of the download manager history bug.  Reporter:  Have
you tried clearing your download manager history occasionally as you download? 
Does this keep the RAM/CPU usage from spiking significantly?

Comment 13

[Tom Maddox]

(In reply to comment #12)
> This appears to be a dupe of the download manager history bug.  Reporter:  Have
> you tried clearing your download manager history occasionally as you download? 
> Does this keep the RAM/CPU usage from spiking significantly?

Clearing the download manager history appears to have no effect on the memory
consumption.  Closing the download manager window does cause memory consumption
to drop, however.

Comment 14

[Ryuichi KUBUKI]

*** This bug has been marked as a duplicate of 251380 ***