the bug url is a bit extreme, ~65k html file which re
oops ... -quires 885MB of [virtual] memory in nc4.75/ie5.5 moz needed 1300+MB. I will generate a more reasonable testcase. http://wam.umd.edu/~soref/mstress.html ~1K, ie5.5: 26832K/19544K moz: 42604K/35224K mem/vm
talkback info, albeit sparse: Incident ID 16672285 Trigger Time 2000-09-01 01:48:33 Email Address timeless@bemail.org User Comments Build ID 2000082815 Product ID Netscape6 Platform ID Win32 Stack Trace nsFileTransport::Process [d:\builds\seamonkey\mozilla\netwerk\base\src\nsFileTransport.cpp, line 511]
here's a better trace, ignore the one above. Incident ID 16672002 Trigger Time 2000-09-01 01:38:39 Email Address timeless@bemail.org User Comments please attach to http://bugzilla.mozilla.org/show_bug.cgi?id=51028 Build ID 2000082815 Product ID Netscape6 Platform ID Win32 Stack Trace nsFileTransport::Process [d:\builds\seamonkey\mozilla\netwerk\base\src\nsFileTransport.cpp, line 511] nsFileTransport::Run [d:\builds\seamonkey\mozilla\netwerk\base\src\nsFileTransport.cpp, line 363] nsThreadPoolRunnable::Run [d:\builds\seamonkey\mozilla\xpcom\threads\nsThread.cpp, line 694] nsThread::Main [d:\builds\seamonkey\mozilla\xpcom\threads\nsThread.cpp, line 96] _PR_NativeRunThread [pruthr.c, line 421] 0x006f006d KERNEL32.DLL + 0x37cd (0x77e837cd)
the crashes happen by loading http://wam.umd.edu/~soref/ostress.html ~2k [well mozilla "finish"es but doesn't render it all], and then clicking reload.
Just a guess from the stack trace ->networking:cache
-->gagan
9/13-12: on my large testcase: <1100mb vm needed which is only 23% worse than nc4.75/ie5.5 (that's a 200+mb savings) 24772/31704K (4mb saved), we still don't load all images on mstress. Gagan, if you can't reproduce the crash, kick this back to imagelib or layout or compositor; I can't.
this is really an overall memory usage issue. ->warren.
taking
Changing component. It doesn't look cache specific.
libpr0n fixes this, right pav?
Lib pr0n makes this much wrose :-( i was at ~1.95GB of ram for the large testcase [119% worse than competition, and browser doesn't actually respond] this test run was done in a w2k terminal session w/ [peak mem usage 2510864, current 550456 - after killing mozilla] @256 colors.
There is definitely something wrong here, even at 24bit color we shouldn't be eating that much memory.
removing 'crash' keyword. This is an out of memory problem potentially resulting in a crash, but not a crash due to anything specific.
Is the huge memory uptake and long load time (I haven't let it finish yet on my machine before killing mozilla) on http://www.12see.de/hpm_design01.cgi?user=funnylady (Select the "Gif's Gif's Gif's" link) this bug? Loads quickly and takes only 11MB in Opera 5.0 for Linux, takes 100% CPU usage and over 70MB in Mozilla 2001060408-trunk
I am using Mozilla Build ID: 2001060703 on NT4.0. The start up memory image of this build is 17346KB while that of IE6.0 (the beta release) is only 6000 and odd KB. I had mozilla on last night and when I came into to work this morning, I found that it had used up 76MB of memory space! Now that is an astronomical figure, in contrast, IE rarely takes more than 20MB. Someone needs to look at the way mozilla handles memory and why it isn't flushing unused memory blocks.
Windows 2000 P3 500 128 MB IE 5.5 mem: 82 M VM: 866 M IE 6 mem: 35 M VM: 1,317 M Mozilla 2001062708 (0.9.2) mem: 53 M VM: 1,381 M IE6 and Mozilla are very close... Anyone have XP?
NS 4.7something on win2k: mem: 37 M VM: 798 M
http://wam.umd.edu/~soref/mstress.html Testing build 2001070321 (gnucc-295) on Linux 2.4.x gives: PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 24055 kmaraas 9 0 53624 49M 9392 S 0,0 26,4 2:14 mozilla-bin Which amounts to ~40 MB?
Fizilla build from 0_9_2 branch under Mac OS X 10.0.4: 1,890,164 kbytes IE 5.whatevercomeswith Mac OS X 10.0.3 : 1,806,004 kbytes
http://wam.umd.edu/~soref/mstress.html on Windows 2000, 16 bit color, RAM = 256MB NN4.7 10M IE6 (beta) 25M M 0.9.2 44M It's still the problem.
I agree. Moz 0.9.2+ freezes up for many seconds after the browser dispenses with one page and fills with another. It's really annoying. IE 5.0 doesn't do this. I prefer open source.
i am using build 2001082608 on WinNT and tried to load the pages mentioned http://wam.umd.edu/~soref/stress.html http://wam.umd.edu/~soref/mstress.html there was nothing displayed whatsoever, this was even if i tried to reload the said pages
didn't work for me either the first time, but the second attempt in a new window worked. Odd
qa -> tpreston
*** Bug 98930 has been marked as a duplicate of this bug. ***
Additionally, if you use many browser windows (control-N) in IE, memory usage with IE (any version 5 or above, I'm using 6) is much lower than Mozilla (currently using 0.9.5). In fact, just browsing with about 6 Moz windows open uses up all of my 192M within about 1/2 hour to 1 hour (W2K) while I can browse all day with IE and have no problems. Sorry about the lack of a test case, but this is really obvious if you do it for a while.
bug 104999 (that has just been fixed) should help with this abit.
46% ? i would say 900% ! in bugzilla, request ALL unconfirmed bugs ... ie and ns both use like 40 megs, mozilla 0.9.8 requires 400! megs. thats just hillarious
as per my measurements, as described in http://bugzilla.mozilla.org/show_bug.cgi?id=124441 when loading http://bugzilla.mozilla.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&email1=&emailtype1=substring&emailassigned_to1=1&email2=&emailtype2=substring&emailreporter2=1&bugidtype=include&bug_id=&changedin=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&short_desc=&short_desc_type=allwordssubstr&long_desc=&long_desc_type=allwordssubstr&bug_file_loc=&bug_file_loc_type=allwordssubstr&status_whiteboard=&status_whiteboard_type=allwordssubstr&keywords=&keywords_type=anywords&field0-0-0=noop&type0-0-0=noop&value0-0-0=&cmdtype=doit&order=Reuse+same+sort+as+last+time 2002020803 160M 1:45 minute CPU time (when placed into background the memory goes up to 800M ???) 0:25 minutes when closing program IE6 102M 0:35 minutes CPU time (not fully loaded?) 0:05 when closing program
for 0.9.8: as you know windows 2000 will swap mozilla out overnight leading to extremely slow response each morning. today I was watching the task manager display as mozilla was being paged in. it showed VM size for mozilla.exe at 66MB. mozilla was totally unresponsive (no screen repaint, no nothing) until the Memory Usage field got up to 45MB. As I loaded the bugzilla.mozilla.org it got to 55MB. This seems to show that mozilla has a huge working set (perhaps the heap gets fragmented over time). Then after checking some mails, moving them aroung, doing a few bugzilla queries to find this bug it shows both Memory at 118MB and VM at 121MB The machine has 512 MB RAM and shows 256MB used in task manager.
sorry guys, you want some other bug, this is an *IMAGELIB* bug
dp, cathleen: can you guys help/comment on this one? [adt needinfo]
We need a new valid test case for this bug! Otherwise, this bug is useless. these links do not work http://wam.umd.edu/~soref/mstress.html http://wam.umd.edu/~soref/ostress.html comment #30 http://bugzilla.mozilla.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&email1=&emailtype1=substring&emailassigned_to1=1&email2=&emailtype2=substring&emailreporter2=1&bugidtype=include&bug_id=&changedin=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&short_desc=&short_desc_type=allwordssubstr&long_desc=&long_desc_type=allwordssubstr&bug_file_loc=&bug_file_loc_type=allwordssubstr&status_whiteboard=&status_whiteboard_type=allwordssubstr&keywords=&keywords_type=anywords&field0-0-0=noop&type0-0-0=noop&value0-0-0=&cmdtype=doit&order=Reuse+same+sort+as+last+time is not an imagelib issue, this is a query generating a bugzilla list of 23461 bugs, which took a huge load of memory. There is already an bug filed, see bug 49539, which has nothing to do with image. The URL link provided on top of bug report http://timeless.haque.net/mozilla/stress.html works. However, nothing gets rendered after page gets done loading. This is same with IE. So, I saved the page, and look at all image links in page source, and found there is no valid access to those image files. I will attatch stress.html for people to view.
Created attachment 72658 [details] stress.html saved from http://timeless.haque.net/mozilla/stress.html no access to any of the images specified in this page. --> test invalid.
aww that sucks. ok, i think i found someone with a webserver who can host the files. i should have more info in 24hrs
ok, the files are now up on the server in the url, if you can mirror them instead of constantly hitting the server, that'd probably be good. index.html mstress.html ostress.html slurp-stress.html slurp.html stress.html you should be able to use wget or netscape composer to slurp the files.
Created attachment 72809 [details] another HTML showing the problem (using Yahoo images) another HTML showing the problem (using Yahoo images) Mozilla takes twice of IE's footprint to display this HTML file BTW JPEG images seem to be worse than GIF images
I did a spacetrace analysis using attachment 72809 [details] and it showed 100 calls to imglib, which holds around 120MB. task manager showed us using 120+MB to load this page and IE5 took 42+MB to load. saari is going to take a look at this. he guessed implementing paletted gifs will help, 3 to 1 ratio.
Those images are PNGs and the ones on the old testcase were JPEGs. GIF fixes are irrelivant to this.
Palettized image storage in general then. I'm guessing PNG uses a palettized format anyway, and if you look at the content of the images, you can see where IE is getting the win; it is an obvious 3:1 ratio. We can teach each decoder to go directly to native paletized formats when appropriate (my choice), or we can take a temporary decode to full 3 byte per pixel, keep color statistics, and then make a decision on if it should be palettized (< 256 colors). My vote is the first option; more work but better results. I'm willing to punt on JPEGs as they're meant to be used when GIF isn't the best choice: large amounts of colors often seen in photographs where LZW starts falling down anyway.
If GIF-s are not relevant to this, should a new bug to filed for the GIF problem?
*** Bug 116257 has been marked as a duplicate of this bug. ***
Bug 143046 is the windows only (currently) GIF solution to this bug. 1 bit/8 bit PNGs should also use the framework that Bug 143046 is trying to set up.
*** Bug 122581 has been marked as a duplicate of this bug. ***
(from comment 16 of bug 122581:) Page with 141 1280x960 images, each shrunk to 160x120 using WIDTH and HEIGHT attributes in IMG tag, requires 1GB RAM (allocated to Mozilla while decoding, X afterwards). Netscape 4.05 requires only 50MB (Netscape+X11 combined) for the same page, and loads 2-3 times faster as well. Shouldn't Mozilla only keep the shrunk image in memory, not the full one?
adt: nsbeta1-
Can someone please update this bug with more example web pages? Thank you much.
Please slurp only once. I will try to retest stress locally, but i currently have a mozilla which crashed due to OOM after using about 700mb of vm, so I don't think mozilla has a chance of loading this page.
I set up a small (~250kB including images) test page at http://achurch.org/51028/ . NC4.78 doesn't even break a sweat; Mozilla chews through memory like crazy without displaying anywhere near all of the images (and dies from OOM on both a 512MB Windows box and a 1.5GB Linux box, both Mozilla 1.6a).
Using Mozilla 1.4 (20030821) on Linux I tried the site mentioned in comment #50. I had a window open showing top (sorted according to mem usage) and noticed it wasn't mozilla-bin eating my memory but X.
Since Netscape doesn't cause X to take up such huge amounts of memory, I'd argue that Mozilla is at fault.
This affects Windows to, and on Windows it does bloat Mozilla process memory, so I'd agree Mozilla is at fault.
andrew church: could you get a stack trace w/ gdb? :) I will try to fix any oom crashes for which you provide a stack. You're welcome to file such stack traces in new bugs and reference them here, since we probably shouldn't clutter this bug with them (although I did cause some clutter in comments 2 and 3).
Unfortunately, it's killed by the OOM killer, so I can't get any stack traces.
Any progress on that?
I think the URL example results in a "memory cache overflow". Each image is stored decoded in the memory cache even if the memory cache is already full. While loading the page about:cache increases and "Storage in use" will exceed "Maximum storage size". I have seen this also in bug 213391 comment 5. If bug 213391 is a dupe of this then other bugs with this memory cache behaviour (like bug 105370) could also be dupes of this bug.
The account from timeless had moved to our new server so I changed the URL.
I affected by the problem also, and IE don't suffer from this, may be we should put the image in disk rather than at Memory??
Created attachment 194498 [details] Another testcase: png image (72 KB) with 10k-15k size PNG Image as Testcase that shows huge memory usage (over 400 MB). Memory Usage of IE with same PNG is far less. Not really sure this is the right Bug, there are a lot Bugs about images and huge memory usage: is Bug 240369 only about .bmp? Tested with: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b4) Gecko/20050831 Firefox/1.0+ ID:2005083106
even worse for me, moz 1.8b on win2k, while rendering memory usage goes over 1.1 gig, then calms down to 550megs once image is shown. stalls the system for a good minute tho...
This bug has been quiet for a while, with no apparent solution in sight. This analysis seems relevant: http://primates.ximian.com/~federico/news-2005-11.html While I think the scheme suggested by the author is sound enough, I would also suggest memory-mapping the in-memory images to the cache, at least for non-visible/inactive tabs; in my experience it's a good idea to let the kernel dynamically distribute memory to apps. Bug 240369 looks like a dupe of this bug, can anyone confirm?
(In reply to comment #62) > This bug has been quiet for a while, with no apparent solution in sight. This > analysis seems relevant: > > http://primates.ximian.com/~federico/news-2005-11.html see bug 259672
(In reply to comment #63) > see bug 259672 Not talking about X; the excessive memory usage problem also exists on Windows.
When trying to view the attachment "Another testcase", I get this error message : The image “https://bugzilla.mozilla.org/attachment.cgi?id=194498” cannot be displayed, because it contains errors. Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3
for reference, bug 296818 has significantly improved things. someone will comment here with numbers.
All with emptied disk cache set to 404800 running the stress test; Page was loaded, then slowly scrolled to the bottom. Camino Version 2007102701 (2.0a1pre) 294MB real, 656MB virtual. Cache: Storage in use: 130418 KiB Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.9a9pre) Gecko/2007102804 Minefield/3.0a9pre 323MB real, 681MB virtual. Cache: Storage in use: 130093 KiB
For completeness: Safari Version 2.0.4 (419.3) 625.75MB real, 1.03GB virtual. Oh, with a beach ball of death that lasted nearly 10 minutes.
Note that the numbers may vary over time. EG, when I ran the testcase with Minefield the VM size was ~1.1GB immediately after running the test. It then dropped to ~700MB after a bit (296818's work), and then scrolling through the whole page increased it back again.
The original testcase requires 885MB of [virtual] memory in nc4.75/ie5.5, but now in Firefox 3.5b4pre it requires less than 450K (with some other tabs open and a lot of addons active). So, I would say the original reported problem that that page requires more memory than nc4.75 is no longer applicable...
or
