crash, memory-leak, testcase
2.02 KB, image/gif
364 bytes, text/html
85.20 KB, image/png
85.22 KB, image/png
103.76 KB, application/zip
Update, also crashes Mozilla Firebird on Linux. Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4b) Gecko/20030516 Mozilla Firebird/0.6
Please test a current build of mozilla -- some image memory leak issues were fixed after 1.4.
I tested with the latest build ( Mozilla/5.0 (Windows; U; Win98; et-EE; rv:1.5b) Gecko/20030722 ) I still do see the leak.
I tested on older builds, this is a regression. The leak has been introduced between 1.3alpha and 1.3beta. I hope it helps.
It seems that to much memory is allocated in the memory cache: > Memory cache device > > Number of entries: 97 > Maximum storage size: 13312 k > Storage in use: 195922 k > Inactive Storage: 0 k 85 of the 97 entries are images. Each image has a cache size of 2.25 MByte, what really results in the 195 MByte. The memory is freed if I close the page.
Assignee: general → darin
Severity: major → critical
Component: Browser-General → Networking: Cache
QA Contact: general → cacheqa
Unnecessary entries should be evicted by nsMemoryCacheDevice::EvictEntriesIfNecessary but this is never done because all entries are in use: http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/netwerk/cache/src/nsMemoryCacheDevice.cpp&rev=1.51&root=/cvsroot#343 Maybe the entry should be doomed if entries should be evicted but could not. Same is done in nsMemoryCacheDevice::OnDataSizeChange for the softlimit: http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/netwerk/cache/src/nsMemoryCacheDevice.cpp&rev=1.51&root=/cvsroot#285
Summary: Crash caused by a memory leak → Failure of evicting cache entries will increase memory usage [@ nsMemoryCacheDevice]
Testcase reloads the previous test image every 25 ms with a different cache key. Store this file and http://bugzilla.mozilla.org/attachment.cgi?id=138533 in the same (local) folder and load it in the browser. Cache will increase until whole computer memory is used.
>is never done because all entries are in use why are they in use?
*** Bug 225977 has been marked as a duplicate of this bug. ***
*** Bug 211377 has been marked as a duplicate of this bug. ***
The testcases in bug 98835 also produces a 'cache overflow' like this => DUPLICATE *** This bug has been marked as a duplicate of 98835 ***
Status: NEW → RESOLVED
Last Resolved: 15 years ago
Resolution: --- → DUPLICATE
Sorry wrong duplicate => REOPEN
Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---
The observed cache issue is bug 126445 => mark as DUPLICATE Sorry for the spam *** This bug has been marked as a duplicate of 126445 ***
Status: REOPENED → RESOLVED
Last Resolved: 15 years ago → 15 years ago
Resolution: --- → DUPLICATE
Bad monday. Maybe it will help if I did not only understand the bug symptom but also the bug description too. Bug 126445 also stores it's data in the memory cache even when there is not enough memory free. But the problem of that bug is the gif animation. => reopening again
Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---
Just chiming in my experiences... WinXP, Moz1.7b (but been experiencing this since 1.6), 232Mb physical memory. Viewing many pages with lots of images (say...www.fark.com Photoshop contests), eventually the Windows page file goes through the roof (commit charge approaches max). Mozilla's about:cache reports: Memory cache device Number of entries: 477 Maximum storage size: 13312 KiB Storage in use: 149138 KiB Inactive storage: 0 KiB Eventually, the whole machine bogs down until I quit Mozilla.
*** Bug 256139 has been marked as a duplicate of this bug. ***
>Gary: If the images are not used anymore (because the tab or the window is >closed), then also the cache is freed. this is wrong, what gave you the idea?
Some tests: 1) The URL from bug 51028 2) The testcase in the attachment In both cases is the cache emptied to it's normal size (read: below Maximum storage size). I assumed that a) not used objects are removed from the cache if b) new memory is needed for a new object and c) to much cache memory is in use. a) is what I expect when I close those the tabs and windows b) is what I forgot to mention c) is what Gary's (about:cache) has described. OK, I forgot b) Is there something else not correct with my conclusions?
I'm having difficulty understanding why memory cache is allowed to go above the maximum allowed limit. Surely it should be using some form of FIFO system? Nominating for 1.8b. Would love to nominate for 1.8a6 but I don't think enough info is available for a quick patch.
(In reply to comment #22) > I'm having difficulty understanding why memory cache is allowed to go above the > maximum allowed limit. Surely it should be using some form of FIFO system? My understanding is that the memory cache capacity is a soft limit that can temporarily be exceeded, and in addition there is a hard limit that cannot be exceeded. The soft limit is exceeded because the cache is storing all images on all open tabs. Here's another way to get the memory cache size to increase: 1. Set browser.cache.memory.capacity to 4096 2. Open a new tab and go to about:cache 3. Open another new tab and go to http://www.google.com/imghp?hl=en&tab=wi&q= 4. Do a Google image search for "cats" 5. Scroll down the page, right click on the Next link, and select Open Link in New Tab 6. Repeat step 5 until you have 20 open tabs with images of cats (meow!) 7. Make the about:cache tab active and reload the page, noting memory cache usage 8. Right-click on the about:cache tab and select Close Other Tabs 9. Reload about:cache tab noting memory usage When I follow the above steps with Mozilla 1.8a5 on Windows XP, I get 15-16 MB memory cache usage when 20 tabs of cats are open. After closing all 20 tabs, the memory usage hardly goes down at all. The same happens if I close the tabs without making them active in some other way, such as right-clicking each tab and selecting Close Tab. But if I make each tab active before I close it, the memory is released as soon as the tab is closed. Finally, no matter how I close the tabs, the memory is released when I close the window the tabs were in. It seems that for some reason, cache memory is not immediately released for inactive tabs that get closed.
> 1. Set browser.cache.memory.capacity to 4096 As far as I understand is this (4 MB) the new hard limit. And it seems that the softlimit is 90% of the hardlimit and is only used for streaming data. > When I follow the above steps with Mozilla 1.8a5 on Windows XP, I get 15-16 MB > memory cache usage when 20 tabs of cats are open. After closing all 20 tabs, So you will exceed your hard limit.
>1. The crash is the result of a hugh memory consumption. that makes no sense. apps should not crash just because there is no memory left. please file such bugs, preferrably with talkback IDs or stacks.
dupe of or dependant on bug 265028 ??
Hi all, I tested Netscape 7.02, Mozilla 1.7.5 and Firefox 1.0 on an NT4. All browser versions exceeded the memory-cache max. storage size. I simply opened a number of sites (newspapers, business-sites and others) in tabs. I did the tests because I want to reduce the memory-cache capacity in order to run Eclipse (which consumes a lot of memory) together with Mozilla (or Firefox) on a (SuSe Linux, KDE or FWM2) notebook with 500 MB RAM. Reducing the cache size is one of useful options in such a situation. For that reason, regardless of any reported crashes, I recommend to give this problem more attention. From a design perspective I would suppose that a browser shifts a piece from memory to disk in case of the memory-cache is full.
(In reply to comment #25) > The crash is the result of a hugh memory consumption. (btw. the URL is > death, the testcase still works) The testcase does not cause any more memory to be consumed for me using Mozilla trunk build 2005022405 on Windows XP. I can't see any image in the "Simple plain test image." attachment, so maybe this is why the testcase doesn't work for me. Can we get a new testcase or new steps to reproduce? > Should it possible to exceed the hard limit? Yes, after reading the cache code it's clear that all images on currently displayed pages are stored in the cache, regardless of the size of the cache. It seems like there are several issues all lumped together in this one bug report. One is the fact that a crash occurs on excessive memory use. The others are ideas on how we can limit the amount of memory used for images -- store them compressed instead of decompressed, or move them out of the memory cache if they're displayed only on tabs that aren't visible, or stop loading images if we start running out of memory.
I hope this might help. The URL http://marius-slc.c7obs.net/timeto/, contains a few simple image-based counters. Although the total size of the images is small, due to the fact that they are reloaded, Firefox seems to reload the images every time and, after a few hours, consumes very much memory. I have not been able to do a simpler test case yet, but I am working on it.
I can't reproduce the problem using the new test case using Mozilla trunk build 2005022805 on Windows XP SP2, even with the page open all night. The VM Size did not appear to increase, and the size of the memory cache also did not increase. Can someone reproduce it using a recent trunk nightly of Firefox?
I can confirm the leak with firefox-2005030600.en-US.linux-i686-gtk2+xft-xforms-svg-cairo . With http://marius-slc.c7obs.net/timeto/ and an about:cache tab opened: memory usage (both virtual and resident) grows by about 330kb/s, memory cache usage (mesured both by about:cache and x pixmaps in xrestop) doesn't change.
It's a bit more complicated: the second time I visit the page, it doesn't always work, sometimes the memory stays level.
Comment #6: Maybe the entry should be doomed if entries should be evicted but could not. Same is done in nsMemoryCacheDevice::OnDataSizeChange for the softlimit: http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/netwerk/cache/src/nsMemoryCacheDevice.cpp&rev=1.51&root=/cvsroot#285 A: Could be something to do, at least for the images, but the corresponding imgRequest object is then not freed... Possibly, when the memoryCache want to clear an imgRequest, the mImage of imgRequest should be cleared, and mImageStatus reset, so that when the page needs that image, it is again decoded (from the original file) into the internal format. Comment #19: Why is it possible to store something in the cache even if there is no space left in it? Are *all* objects *allways* cached? Maybe mozilla should stop loading a page (or the objects on it) if a quantity of memory (like the memory cache) is exceeded. A: Only for cache elements with IsStreamData are refused when they are too large for the memory cache (larger than mSoftLimit). As even very large image objects are still needed to draw them, the cache cannot refuse this. But it should be possible to clear this very large object as soon as possible, even before the page where the image is used is gone... See above for how this could work... In summary, This is not a bug within the memory cache, but in the way that the internal format of images is kept in the memory cache as long as the page with them is visible. So, the memory cache could request the imageCache to request imgRequest to destroy the internal image object, as it can rebuild it from the cached file when needed. But unsure how the Gecko engine would react to that...
Status: REOPENED → ASSIGNED
Summary: Failure of evicting cache entries will increase memory usage [@ nsMemoryCacheDevice] → Necko/Gecko needs to enforce a maximum for total size of image/memory cache
I think I see a similar problem in SeaMonkey 1.0a. After opening many big images, the max storage size for memory cache is exceeded even over the size of used memory displayed in Windows taskmanager.
On my box firefox 1.5 final is grabbing >1,2 GB RAM after opening and closing many high-res images. This is sick.
Yes, I agree. Something needs to be done about this. I suspect the large memory usage in your case is attributable to the back/forward cache. The documents in its cache reference image objects, which are stored in the memory cache until the document releases its reference on them.
Target Milestone: --- → mozilla1.9alpha
(In reply to comment #39) > Created an attachment (id=206045)  > Extreme memory usage It looks normal to me. Yes, the cache shows 1.2 GB, but the attachment shows 836 images in cache. That's a very reasonable average of 1.7 MB per image, or slightly larger than a 600x800 24-bit image. We need to be clear here about what the problem is. I think there are two problems. First, it can become extremely unresponsive as memory resources approach zero. Second, there is no way to terminate the JS execution except by closing the tab or window (Stop button is grayed out). That appears to be bug 51139.
(In reply to comment #41) > (In reply to comment #39) Clarification: If attachment refers to the testcase, yes, that's outrageous. If it's just loading many images at once, it could be normal. I reiterate that part of the problem is that the user cannot press "Stop" to terminate the loop in the testcase (bug 51139).
I created a set of static files and images to demonstrate the problem (resulting in 900MB memory cache usage and about 1GB virtual memory allocated with Firefox 184.108.40.206 on Windows XP). Basically, what Alfred Kayser said, the problem is with the fact that all bitmaps of all images in all tabs in all windows (and the ones in the fastback cache) are kept in memory, until the tab or window is closed (or, presumably, until it expires from the fastback cache, but I haven't verified that). Image bitmaps that are not currently visible (scrolled out of view, in an inactive tab, in an inactive window...) must be allowed to be cleared from the cache on demand, instead of when closing the tab. The performance drop resulting from having to re-decode images is negligible compared to the amount of swapping (and general unresponmsiveness of the whole operating system). The testcase consists of an index page and 4 pages, each containing 100 "unique" images in 1024x768, 400 images in total (the individual files are quite small, 532 bytes each). Each image occupies 2,25MB of memory (1024*768*3 bytes, assuming a 24bit truecolor display), each page therefor occupies 225MB. All pages together occupy 900MB of memory. Similiar memory requirements are needed for 25 open tabs, with 30 800x600 images in each tab. To reproduce, open each image set in a separate tab/window (closing the tab/window correctly frees the memory in FF 220.127.116.11), or follow the next/previous links (fastback).
Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9a1) Gecko/20060208 Firefox/1.6a1 ID:2006020806 With the testcase from comment 43 firefox does indeed consume a rather large chunk of memory (many 100s MBs) where both IE6 and Opera7.0 TP2 remain calm with ~20MB used.
*** Bug 326962 has been marked as a duplicate of this bug. ***
Bug #259672 is related with this one (depends i think), please add to the list
*** Bug 330406 has been marked as a duplicate of this bug. ***
*** Bug 321223 has been marked as a duplicate of this bug. ***
I also ran into this problem, however I don't excatly know how related my problem is to this. When opening the URL: http://nenya.ms.mff.cuni.cz/projects/mono/benchmarks/rtests.phtml mozilla allocates tons of pixmaps on the XServer, way too much when running on a machine with just 256mb: "15 0 222m 210m 4436 S 0.7 42.0 1:08.99 X" lg Clemens
Just a small note - it is possible for extensions to "lock" the images in the memory cache so they don't get freed up even after the tabs or even the windows are closed (this happens for example in Adblock Plus 0.7.0.2 but there are probably many more). While this is clearly something that can be fixed by extension authors, I think it should not be possible for this to happen in the first place.
that's not this bug. let's please use one bug per issue.
*** Bug 292993 has been marked as a duplicate of this bug. ***
(In reply to comment #49) > When opening the URL: > http://nenya.ms.mff.cuni.cz/projects/mono/benchmarks/rtests.phtml Just ran this testcase using Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20060918 Firefox/2.0 under WinXP and experienced the following strange behavior: - *Opening* the above mentioned page in a new tab makes VM rise by about 180MB (from 21MB freshly started FF to 200MB) - *Closing* tab does not release memory (?) - *Re-opening* tab does not increase memory bloat (?) - *Scrolling* through opened page (thereby displaying images) frees memory gradually (?), until VM size reaches about 40MB again - Closing the tab frees a bit more of VM
(In reply to comment #53) > - *Closing* tab does not release memory (?) >... I see the same behavior, but only with Adblock Plus (0.7.1.2) enabled (Using Safemode or Adblock Plus disabled via Addons Manager, memory IS released after closing Tab) So this not-releasing behavior is probably part of another Bug, maybe http://bugzilla.mozdev.org/show_bug.cgi?id=14445 ? This seems also to be somehow related to History. After a restart of Fx, memory is now released, you only see Bug again after clearing Browsing History. Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20060927 BonEcho/2.0 ID:2006092710
(In reply to comment #54) About:cache says that cache memory used for this page is 146518 KiB, way above the maximum limit (14336 KiB). This page is another test case for the bug into discussion.
(In reply to comment #54) > I see the same behavior, but only with Adblock Plus (0.7.1.2) enabled I am using Adblock Plus as well, same (newest) version. > (Using Safemode or Adblock Plus disabled via Addons Manager, memory IS released > after closing Tab) Cannot confirm for 18.104.22.168 (my work machine), just opened page in tab (memory ~192MB) with Adblock Plus disabled, closed tab, memory still high. Memory DOES get freed after navigating to different site in another (new) tab, so I agree on browser history relation. Will re-test on 2.0rc2. Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:22.214.171.124) Gecko/20060909 Firefox/126.96.36.199
(In reply to comment #54) > (Using Safemode or Adblock Plus disabled via Addons Manager, memory IS released > after closing Tab) Stebs is right, when I disable Adblock Plus in the *Extension Manager*, then memory gets released ASA tab is closed. In 1.5, I only disabled Adblock Plus itself (this extension has an option to turn it off, but it remains loaded in FF). So forget my last comment. Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061003 Firefox/2.0
PLEASE DO NOT ADD MORE COMMENTS unless you are fixing this bug. The issue is very well known, very well understood, and assigned to a competent developer (see comment 40). Additional reports of behavior in various situations will not help get this bug fixed.
Whiteboard: FURTHER COMMENTS ARE NOT NEEDED. See comment 58 for details.
-> reassign to default owner
Assignee: darin.moz → nobody
Status: ASSIGNED → NEW
(In reply to comment #58) > PLEASE DO NOT ADD MORE COMMENTS unless you are fixing this bug. The issue is > very well known, very well understood, and assigned [...] With the bug now again unassigned, is this six-month-old comment moratorium on this four-year-old bug still desirable?
Yes, adding more comments (that are not about coding) will not achieve anything.
[quote="chob"] People have filed such bugs, as long ago as 2003, and still there has been no action on them:[list][*][url=https://bugzilla.mozilla.org/show_bug.cgi?id=194296]#194296[/url] [Core:Networking: Cache]-minimize memory cache size and growth [All] [*][url=https://bugzilla.mozilla.org/show_bug.cgi?id=213391]#213391[/url] [Core:Networking: Cache]-Necko/Gecko needs to enforce a maximum for total size of image/memory cache [All] [*][url=https://bugzilla.mozilla.org/show_bug.cgi?id=327280]#327280[/url] [Core:History: Session]-Don't keep decoded images in the bfcache pages [All][/list]Especially regards the first two, At first I thought Fx 1.5 would have such fixes, but no. So my hope turned to Fx 2.0 to include such fixes. But no, so i thought that Fx 3 much get such fixes. But no. So now where do I pin my hopes to? Fx 3.5? Fx 4? Fx 5? Personally I don't see such memory problem as a big deal, but I close my browser regularly. My main complaint is that I'm sick to **** of hearing about firefox's memory usage ;) and it is a common complaint on forums and sullies Firefox's reputation in some quarters. By leaving such bugs untouched over years, Mozilla really sending out the message 'we don't care'. But it's worse than that - it's not like they don't care because they don't understand the problem or know how to address it; in the bugs filed there are technical discussions and ideas of what to do, and it seems that they still don't care anyway! Bug 213391 was even assigned for a while before being set back to NEW. But anyway, my main point is there /are/ bugs filed that suggest specific ways firefox could better manage its memory, and absolutely nothing has been done for such bugs. Firefox's memory usage is a definite mark against it when comparing it to other browsers, and it needn't be such a big issue. Mozilla need to give these bugs some serious consideration when the next release cycle is coming around and give people another reason to be proud of firefox.[/quote]
Patrick, your comment would have been better place in a blog entry somewhere but it is still and interesting read. Thanks.
It's probably too late now to bring this on the table for 1.9, but while trying to support users this is the single most annoying bug related to memory handling that comes up again and again. Unfortunately, I am not competent at all to work on this. Isn't there anybody else who could do it?
bug 296818 goes a long way towards addressing this. If you try the testcase in a new nightly and wait 45s after loading one of the pages with lots of images you'll get close to 225mb back. So if you load lots of images in quick succession you'll still hit the max, but if you wait intervals between the uncompressed images will get purged from the cache. As a side not to comment 63, there has been a tremendous effort put forward to improve memory usage in gecko 1.9 (witnessed by 296818 getting fixed, gif's as 8bit, 150+ mlk bugs, weekly memory meetings, etc). I know it can be hard to tell because there are so many different bugs open on things - but this is definitely getting looked at. Given the work elsewhere to reduce memory usage I'm going to take this bug off the 1.9 blocker list.
Flags: blocking1.9? → blocking1.9-
Whiteboard: FURTHER COMMENTS ARE NOT NEEDED. See comment 58 for details. → FURTHER COMMENTS ARE NOT NEEDED. See comment 58 for details. [sg:dos oom]
Whiteboard: FURTHER COMMENTS ARE NOT NEEDED. See comment 58 for details. [sg:dos oom] → FURTHER COMMENTS ARE NOT NEEDED. See comment 58 for details. [sg:dos oom], [MemShrink]
Looks like our image work has paid off; all the testcases here wfm.
Status: NEW → RESOLVED
Last Resolved: 15 years ago → 8 years ago
Resolution: --- → WORKSFORME
It looks like this bug was never resolved. I frequently see memory consumption go far beyond what is specified in "browser.cache.memory.capacity" when browsing web pages with many large images. I understand that this limit is a "soft", temporary limit. Unfortunately, it's temporary. Image cache keeps growing. The only way to address this is to completely exit Firefox from memory; which isn't a reasonable solution. Is it really that hard to limit image caching? This problem has been around for as long as I've used Firefox. It's no wonder why so many people are switching to Chrome.
Edit: Please disregard my previous comment. The problem goes away *IF* I change the browser.cache.memory.capacity setting to -1. I am using that in conjunction with browser.sessionhistory.max_total_viewers=5 (max 512MB of memory dedicated to cached pages).
mkanet: - browser.cache.memory.capacity is used to control the size of the memory cache, which is used to speed up loading a website that you've visited before. It has nothing to do with the memory that is used to display the pages currently on your screen. And it's a hard limit. - the same remark about browser.sessionhistory.max_total_viewers ; that's used to speed up going backwards and forwards. And a viewer is only a page (roughly), it's not equivalent to 100MB.
Thanks for replying Jo Hermans. So, how do I limit how much memory for firefox to use? The problem is that the firefox.exe process keeps growing over time; and, never returns back the memory it took. If it is indeed really a hard hard limit; it doesnt really make any sense; especially since a single hard-set setting can't possibly be ideal for for PCs that sometimes run low on memory; hence, there should be a user-configurable setting for this. I end up having to restart the firefox.exe process every morning so it's not so sluggish. Hopefully, there's a better way to handle this without having to change hardware on the PC.
Sorry for the grammar errors above! Hopefully, I was able to get my point across.
I think you should focus on the sluggishness as the problem. Why do you think RAM usage is causing the sluggishness? The only way I can think that would happen is that if Firefox uses up most of the RAM on your computer and it starts swapping memory to disk. How much RAM does your computer have and how much is Firefox using when it becomes sluggish?
As one of its functions is a web server, I keep my 4GB 32 bit openSUSE Linux system running continuously, without an enabled swap partition. I also keep KDE3 running for upwards of a week, often more than a month, without restarting. Within KDE I keep 4 Gecko instances running for days or a week or more at a time, with a 5th that I restart daily so I can backup its profile while it is closed. Individual instances are: FF rv:188.8.131.52 from mozilla.org FF rv:184.108.40.206 from Mozilla Legacy repo (Flash is enabled only for this one) FF latest (now rv20.0) from mozilla.org SM next (now rv21.0) browser and IRC; from mozilla.org SM latest (now rv20.0) browser and email; from mozilla.org only because updates are so slow to arrive in openSUSE repos. This one has the most open tabs, and NoScript installed. I see no sluggishness or delays like I see reported by others, except occasional ones I invariably attribute to network issues outside my LAN. RAM usage as reported by top for all Gecko instances appears to be completely reasonable. I estimate that my average open tab count among the 5 browser windows and 1 IRC window is upwards of 200. In email I don't use tabs. This bug is definitely worksforme. However, I have MOZ_NO_REMOTE=1 set, and I wonder of having this unset is some kind of enabler for otherwise unexperienced trouble. I suspect most developers are using either -no-remote on startup or have MOZ_NO_REMOTE=1 set to facilitate using a regular release for normal web activity while simultaneously using one or more of various pre-releases for testing development.
Hey, guys. I appreciate that you're trying to fix problems that you experience, but please bear in mind that every comment here sends an e-mail to more than 83 people, some of whom monitor bugzilla closely as part of their daily job. At the very least, if you find anything technically interesting in this discussion, you're going to be ignored. It's much better to file a new bug and talk in there.
You need to log in before you can comment on or make changes to this bug.