Closed Bug 631494 Opened 13 years ago Closed 13 years ago

Massive memory leak with Adblock Plus 1.3.3 installed

Categories

(Core :: General, defect)

Product:
Core
Component:
General
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
VERIFIED FIXED
Milestone:
mozilla2.0b12
Tracking Flags:
Tracking Status
blocking2.0 --- final+

People

(Reporter: jwkbugzilla, Assigned: smaug)

References

Details

(Keywords: regression, Whiteboard: [hardblocker])

Attachments

(6 files, 1 obsolete file)

Minimal content policy for testing
1.49 KB, application/x-xpinstall
Details
Comment #40 - Memory usage After1stRUN
440.28 KB, image/jpeg
Details
Comment #40 - Memory usage After2ndRUN
446.07 KB, image/jpeg
Details
Comment #40 - Memory usage After3rdRUN
442.72 KB, image/jpeg
Details
Comment #40 - Crash report (at a memory usage of 1.7 GB)
2.53 KB, text/plain
Details
Olli's try patch, version that applies on top of 3ea2b5a7c9c8
26.91 KB, patch
Details | Diff | Splinter Review 
A user reported a memory leak on trunk if Adblock Plus is installed. Steps to reproduce:

1. Create a new Firefox profile, install Adblock Plus 1.3.3 from https://addons.mozilla.org/addon/adblock-plus/.
2. On restart, when asked to choose a filter subscription choose EasyList (the bug is not reproducible if nothing is blocked).
3. Go to http://www.askmen.com/specials/top_99_women/ (sorry, that's the testcase I got). Open task manager or something else to monitor Firefox memory usage.
4. Now click the red "Start with No.99" button and flip through the pages clicking "Next" until you get to number 80 (20 pages in total).
5. Wait a few seconds for garbage collection to run and write down the Firefox memory usage.
6. Now repeat steps 3-5.

Expected results:
During the first run the memory usage increases and stabilizes then. The memory usage after the first and the second run isn't significantly different.

Actual results:
The memory usage continues to grow and increases by 150-300MB between first and second run.

I tried finding the regression range. It seems that the issue first occurred in 2011-01-30 build (regression range http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=94a51a3b64d4&tochange=336d5906cb0f). However, the memory increase was smaller in that build, it became really bad in the 2011-02-01 build. So I am not entirely sure about this regression range.

Leak gauge and Leak Monitor don't show anything, according to them there is no leak. I've been testing on Windows 7 x64, the original reporter is on OS X 10.6.
Requesting blocking2.0 - releasing Firefox 4 with a huge memory leak would be bad.
blocking2.0: --- → ?
I am not sure this is really to do with Adblockplus, with all addons disabled manually, I still get to 625MB private bytes by number 80 which is way above what I get in safe mode with 234MB in use by 80. Given safe mode disabled jaegermonkey as well as all addons I think this must be the culprit. With all addons, including ABP enabled by number 80 I get 740MB private bytes in use.

Mozilla/5.0 (Windows NT 6.1; rv:2.0b12pre) Gecko/20110203 Firefox/4.0b12pre
Narrowed it down to two filters that can be added instead of subscribing to EasyList:

/show_ads.js
||wrapper.askmen.com^

Both block JavaScript files, still no idea why blocking these particular scripts causes a leak.
(In reply to comment #2)
> I am not sure this is really to do with Adblockplus, with all addons disabled
> manually, I still get to 625MB private bytes by number 80

Absolute memory usage numbers are not very meaningful. Sure, this site seems to require tons of memory for some reason but that might be ok and the memory will be freed later. The tendency that going through the same pages increases memory use significantly every time is an indication of a memory leak however.
I can also reproduce this issue with a minimal content policy implementation that will only block these two addresses. Attaching the extension, it can be used instead of Adblock Plus.
Attached file Leak Gauge log from test URL (obsolete) — 
Leak Gauge found some leaks from my 03-01 nightly with Adblock dev version and the test URL.
Comment on attachment 509741 [details]
Leak Gauge log from test URL

When running such tests, please disable all unrelated extensions (in particular, NoScript and Firebug that appear prominently in this log). As I mentioned above, leak gauge doesn't show any leaks here. I just verified again, whatever the leak is here it doesn't appear to be DOM objects.
Attachment #509741 - Attachment is obsolete: true
I rechecked the regression range. It seems that 2011-01-30 build only shows abnormally high memory use but no leak. The first build where memory use actually increases is 2011-01-31. So the real regression range is http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=336d5906cb0f&tochange=732a38102733.
(In reply to comment #4)
> (In reply to comment #2)
> > I am not sure this is really to do with Adblockplus, with all addons disabled
> > manually, I still get to 625MB private bytes by number 80
> 
> Absolute memory usage numbers are not very meaningful. Sure, this site seems to
> require tons of memory for some reason but that might be ok and the memory will
> be freed later. The tendency that going through the same pages increases memory
> use significantly every time is an indication of a memory leak however.
I don't think the requiring of a ton of memory is normal- look at the usage in safe mode 234MB (with 4 other tabs open) vs normal mode with all addons disabled 625MB vs usage in SR Iron (Chrome) 8, similar to FF4 safe mode 240MB. So nearly 400MB more in use with the only difference being jaegermonkey is enabled, with no ABP enabled at all.
I think FF4 just has a problem with this askmen site, there have been other problems with their sister site ign.com

For example, go here http://www.askmen.com/specials/2010_great_male_survey/

check memory usage, then click one of the tabs (dating & sex, lifestyle, men in 2010), memory usage increases by 90-100MB, no such increase is seen in FF 3.6.13 or SR Iron (Chrome) 8.
Memory usage increases continuously with just having that webpage opened, on an *extensionless* profile.
I am doing some tests here relating to these two askmen pages mentioned above, I do not believe the problem is the cause of ABP Wladimir so you can rest easy. There was a change between Beta 6 and 7 that caused a doubling of the memory used by a new askmen page, and a second change between 11Pre nightly 26 Jan-30 Jan that caused memory to not fallback causing the ever increasing memory problem you have seen in recent builds. I am currently trying to determine the exact builds which caused these issues.
orangezilla and JK: Please refrain from off-topic comments. I posted very clear steps to reproduce in the description of this bug. For these steps there is a regression range. Anything else, particularly the high memory usage on this page which could already be observed *before* the regression, are *different* issues that don't belong into this bug (feel free to file separate bug reports). The more comment spam is added here, the less likely this bug will be fixed.
Yes I intended to file a separate bug for the earlier change, but I must say that I and JK, I think agree that this issue is not caused by ABP as it occurs without it or any other extensions installed at all. I agree that the 2011-01-29 11Pre build did not have the issue, and the faulty change was from the 30th onwards, as on the other askmen page below the memory did not fall 80MB after rising 90MB or so after 35 seconds as usual, but instead rose after 60 seconds or so by 5MB and continued to rise

http://www.askmen.com/specials/2010_great_male_survey/
--> Core::General

Can someone please update the summary, as well, if it's not specific to AdBlock?
Product: Firefox → Core
There clearly is a bigger leak if ABP is installed.
The issue I am seeing between 29th & 30th builds is from 30th build onwards FF is not giving up memory after loading the page below as I have found all previous builds have done from FF4 Beta 1-11Pre within around 35 seconds 
http://www.askmen.com/specials/2010_great_male_survey/
I will file this as a separate bug
Bug posted here on other askmen page
https://bugzilla.mozilla.org/show_bug.cgi?id=631733
(In reply to comment #4)
> (In reply to comment #2)
> > I am not sure this is really to do with Adblockplus, with all addons disabled
> > manually, I still get to 625MB private bytes by number 80
> 
> Absolute memory usage numbers are not very meaningful. Sure, this site seems to
> require tons of memory for some reason but that might be ok and the memory will
> be freed later. The tendency that going through the same pages increases memory
> use significantly every time is an indication of a memory leak however.
Using previous builds to 30th, there is not the high usage problem with all addons disabled or enabled, while cycling through the pictures, it settles for me at about 265MB pb, with short peaks to 310MB at the most, nothing like the 625MB pb and rising usage I see in 30th builds & after.
QA Contact: general → general
OK I have done a comparison test between latest nightly build and 26th build recording memory with each click- as you can see, although ABP adds 20MB or so extra, with or without ABP enabled, the behaviour is the same on both builds, clealy showing on the latest build and from 30th onwards, the memory steadily increases instead of dropping back with 29th build and previous eventually reaching 660MB with 20 or so clicks through the pictures, ABP disabled, 712MB with ABP enabled

using: this bugpage, the askmen top 99 women tab
http://www.askmen.com/specials/top_99_women/

and http://input.mozilla.com/en-US/beta/search?q=&product=firefox&version=4.0b10&date_start=&date_end= :

FF4 Beta 11pre 2011-01-26 2 tabs + askmen tab ABP disabled, 

after 1 minute 56.6MB, open askmen top 99 women, peak 101MB, 

fall to 88.2MB, click start with 99, peak 127MB, fall to 

106.1MB, click next rise to 209.4MB, then 222.4MB, then 224.8MB, 

then 224.8MB, then 215.1MB, then 225MB, then 224.8MB, then 220.0MB, then 220.1MB, then 232MB, then 226.5MB, then 236.3MB, then 236.5MB, then 234.4MB, then 236.1MB, then 226.3MB, then 234.2MB, then 235.3MB, then 227.6MB, after 80 seconds fall to 145.9MB


FF4 Beta 11pre 2011-01-26 2 tabs + askmen tab ABP enabled w/e-l, 

after 1 minute 81.2MB, open askmen top 99 women, peak 120MB, 

fall to 106MB, click start with 99, peak 143MB, fall to 125.3MB, 

click next rise to 223.8MB, then 229.6MB, then 237.7MB, then 235.4MB, then 239.8MB, then 243.9MB, then 250.3MB, then 265.1MB, 

then 251.0MB, then 252.4MB, then 248MB, then 251.1MB, then 

263.8MB, then 251.4MB, then 255.1MB, then 252.2MB, then 255.8MB, 

then 263.3MB, then 250.3MB, after 80 seconds fall to 153.3MB


FF4 Beta 12pre 2011-02-05 2 tabs + askmen tab ABP disabled, 

after 1 minute 59.6MB, open askmen top 99 women, peak 115.2MB, 

fall to 111.3MB, click start with 99, peak 165MB, fall to 

158.9MB, click next rise to 230.4MB, then 244.9MB, then 281.2MB, 

then 305.9MB, then 327.7MB, then 353.4MB, then 377MB, then 

398.4MB, then 424.8MB, then 448.4MB, then 471.5MB, then 495.4MB, 

then 522.6MB, then 543.1MB, then 565.9MB, then 590.2MB, then 

612.4MB, then 635.7MB, then 659.5MB, after 100 seconds no change


FF4 Beta 12pre 2011-02-05 2 tabs + askmen tab ABP enabled w/e-l, 

after 1 minute 84.2MB, open askmen top 99 women, peak 126.2MB, 

fall to 121.8MB, click start with 99, peak 163.2MB, fall to 

134.3MB, click next rise to 251.1MB, then 274.8MB, then 302.3MB, 

then 329.0MB, then 354.5MB, then 379.64MB, then 404.8MB, then 

431.9MB, then 456.2MB, then 482.7MB, then 509.4MB, then 538.3MB, 

then 559MB, then 585.2MB, then 611.6MB, then 637.7MB, then 

662.8MB, then 689.4MB, then 712MB, after 100 seconds no change
Nothing in the comment 8 regression range jumps out at me.  Could we double-check that?
I think the problem first occured with the 30th build, so the regression range in comment 1 would the correct one in that case

http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=94a51a3b64d4&tochange=336d5906cb0f
I have discovered two other News Corp sites affected by the 29th-30th change, Sky TV shop, and News of the World, where the memory continues to increase with browsing & is never released, probably others also affected not tested, have also done Askmen tests with 29th and 30th builds to show the change definitively:

FF4 Beta 11pre 2011-01-29 2 tabs + askmen tab ABP disabled, after 1 minute 56MB, open askmen top 99 women, peak 101.3MB, fall to 85.5MB, click start with 99, peak 123.6MB, fall to 107.7MB, click next rise to 204.9MB, then 213.5MB, then 221.5MB, then 226.7MB, then 221.7MB, then 221MB, then 229.2MB, then 224.4MB, then 211.3MB, then 211.9MB, then 221.5MB, then 217.3MB, then 212.6MB, then 213.6MB, then 219.3MB, then 205.7MB, then 213MB, then 214.7MB, then 213.4MB, after 80 seconds fall to 130.9MB

FF4 Beta 11pre 2011-01-29 2 tabs + askmen tab ABP enabled w/e-l, after 1 minute 79.9MB, open askmen top 99 women, peak 111.5MB, fall to 100MB, click start with 99, peak 141.2MB, fall to 123.9MB, click next rise to 213.6MB, then 204.2MB, then 208.6MB, then 238.4MB, then 216.6MB, then 232.5MB, then 230.6MB, then 230.5MB, then 233.6MB, then 242.4MB, then 254MB, then 242.9MB, then 228.9MB, then 226.5MB, then 231.1MB, then 241.8MB, then 235.6MB, then 230.8MB, then 234.6MB, after 80 seconds fall to 151.8MB

FF4 Beta 11pre 2011-01-30 2 tabs + askmen tab ABP disabled, after 1 minute 58.4MB, open askmen top 99 women, peak 116.2MB, fall to 112.4MB, click start with 99, peak 164.1MB, fall to 159MB, click next rise to 256.6MB, then 281.3MB, then 305.2MB, then 329MB, then 352MB, then 375.8MB, then 399.9MB, then 423.7MB, then 446.2MB, then 470.4MB, then 494MB, then 517.5MB, then 541.6MB, then 563.4MB, then 586.5MB, then 611.2MB, then 635.8MB, then 660.1MB, then 682.4MB, after 100 seconds no change


FF4 Beta 11pre 2011-01-29 2 tabs + Sky TV shop ABP disabled
http://www.sky.com/shop/tv/
after 1 minute 65.4MB, open Sky TV shop, peak 113.2MB, fall to 101.2MB, click Entertainment tab, cycle through sub tabs, and other TV tabs & sub tabs, i.e. Sky Sports, Movies, HD TV, Anytime, Sky 3D, Extra Channels, Included Channels, stays around 190-210MB


FF4 Beta 11pre 2011-01-30 2 tabs + Sky TV shop ABP disabled
http://www.sky.com/shop/tv/
after 1 minute 62MB, open Sky TV shop, peak 112.7MB, fall to 109.7MB, click Entertainment tab, jumps to 190MB, cycle through sub tabs, and other TV tabs & sub tabs, i.e. Sky Sports, Movies, HD TV, Anytime, Sky 3D, Extra Channels, Included Channels, steadily increases with each click to reach 450MB, memory is never released


FF4 Beta 11pre 2011-01-29 2 tabs + NOTW ABP disabled, after 1 minute 64.6MB, search News of World in Bing, open site
http://www.newsoftheworld.co.uk/notw/public/home/
rise to 86.6MB, click categories, News, Showbiz, Sport etc. and subcategories, get taken to register page, click back to main site top left, jumps to 200MB, repeat for 3-4 minutes, memory stays around 190-210MB, after stop browsing drops to 101.1MB after 15 seconds

FF4 Beta 11pre 2011-01-30 2 tabs + NOTW ABP disabled, after 1 minute 70MB, search News of World in Bing, open site
http://www.newsoftheworld.co.uk/notw/public/home/
rise to 84.3MB, click categories, News, Showbiz, Sport etc. and subcategories, get taken to register page, click back to main site top left, repeat for 3-4 minutes, memory rises to 430MB, never releases
This seems to be a much more general problem caused by the 29th-30th change, definitely needs to be hardblocker, some sites have steady memory increase and no release of memory, some do release after steady increase but have much lower steady usage in 29th Jan build:


4 12Pre 0206 any http://abclocal.go.com site - steady increase in browsing to 500MB+, no memory released

4 11Pre 0129 any http://abclocal.go.com site - browsing uses 200-258MB, no steady increase, drop to 135MB if stop browsing


4 12Pre 0206 abc local slideshow
http://abclocal.go.com/wtvg/gallery?section=news&id=7942061&photo=1 rise to 444MB hw on no drop in memory

4 11Pre 0129 abc local slideshow
http://abclocal.go.com/wtvg/gallery?section=news&id=7942061&photo=1 stay around 230MB throughout slideshow

FF4 12Pre 0206 abc slideshow
http://abcnews.go.com/International/slideshow/egyptian-protesters-clash-police-12785329
start at 200MB, start clicking through slideshow rise to 390MB, drop to 270MB, start to rise again until stop cycling through images, plateau at 348MB, eventually drop to 280MB 

FF4 11Pre 0129 abc slideshow
http://abcnews.go.com/International/slideshow/egyptian-protesters-clash-police-12785329
stays around 190-210MB for entire slideshow, no steady rise, memory eventually drops to 124.7MB if navigate from tab, if switch back to tab, rise to 194MB.

4 12Pre 0206 National Gallery 
http://nationalgallery.org.uk/paintings/collection-overview/
with persistent browsing of paintings, zooming, steady rise to 426MB, eventually drop to 343MB, fall to 218MB, then 180MB

4 11Pre 0129 National Gallery
http://nationalgallery.org.uk/paintings/collection-overview/
with persistent browsing of paintings, zooming, stay betwen 130MB-160MB, constantly releasing memory
Again, can someone please double-check the regression ranges here?  So far there's lots of mostly-unreadable data (attachments would work better for that sort of thing), and two disagreeing regression ranges...
Boris, please ignore orangezilla, he insists on investigating an entirely different bug and keeps posting off-topic comments.

I checked the regression range once again and it looks like I need to correct myself - build 2011-01-31 shows high memory usage but that the memory use is stable. It increases while browsing but then goes back to the same value. Build 2011-02-01 actually shows signs of a leak then, it continuously adds to used memory (I see more than 1 GB after three passes and it doesn't go back even after waiting a long time). Unfortunately, the two scenarios are not always easy to distinguish.

The regression range is http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=732a38102733&tochange=8b5cb26bbb10 then. This regression range has a TM merge which probably means that I need to find a regression range on the TM branch. Bug 625248 is another suspect.
Now I see why I had trouble finding the regression range - the issue isn't reliably reproducible. Right now I can no longer reproduce it in build 2011-02-01, for whatever reasons. Maybe the website changed. I am afraid that the regression range above might be incorrect as well.
I have tried 2011-02-01 build, and you are correct that on the 2nd run, it frees up some memory, but the memory usage is still way way higher than 29th build and it never releases after browsing as 29th and before builds do (see 2nd & 3rd tests in comment 24). I think really we are getting tied up on this bug with the issue of the 1st vs 2nd run memory usage, when as I stated in comment 25 the issues arising from the 29th-30th change don't necessarily exhibit no memory release >at all<, but what they do all show is steadily increasing memory usage with ABP enabled or no addons enabled at all, whereas 29th and before builds constantly release memory and do not show steady increase to excessively high levels. Sorry if my tests have been a little difficult to read, they were simply to show how private bytes memory usage increases with each click 'next' of the slideshow  (i.e. I state then xMB, means one click).
I verified the memory usage on  the following builds (OS: Windows XP) using the STR's in the Description:

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 -> 
Mem.Usage aprox. 140 MB (on first run) locks at (after first run) 130 MB

Mozilla/5.0 (Windows NT 5.1; rv:2.0b7) Gecko/20100101 Firefox/4.0b7 -> 
Mem. Usage aprox. 250 MB (on first run) locks at (after first run) at 160 MB

Mozilla/5.0 (Windows NT 5.1; rv:2.0b8) Gecko/20100101 Firefox/4.0b8 -> 
Mem. Usage aprox. (300 MB) (on first run) locks at (after first run) 180 MB

Mozilla/5.0 (Windows NT 5.1; rv:2.0b10) Gecko/20100101 Firefox/4.0b10 -> 
Mem.Usage aprox. (360 MB) (on first run) locks at (after first run) 170 MB

Mozilla/5.0 (Windows NT 5.1; rv:2.0b12pre) Gecko/20110207 Firefox/4.0b12pre ->
Mem.Usage (500 MB) (on first run) locks at (after first run) 300 MB
will run a test with a debug build + trace refcnt to find out whats going on.

- Tomcat
Olli, this is probably due to the GC/CC scheduling changes, which you're fixing...
Assignee: nobody → Olli.Pettay
blocking2.0: ? → final+
Whiteboard: [hardblocker]
There was a change to GC/CC scheduling 2011-02-08. Did that perhaps
help here?
I was using the test content policy and run the steps-to-reproduce and
after second time memory usage was actually 25MB less than after first run.
Vladimir, could you perhaps re-test.
Bug 630947 could have fixed the problem.
Unfortunately, I can no longer reproduce the issue, not even with build 2011-02-02. I tried build 2011-02-08 now and get very similar behavior there: 550 MB after first run, 725 MB after second run and 590 MB after third. So memory use seems to grow until at some point all the extra memory is released. 550 MB seems to be the lower bound all way along, so no recognizable leak.
AndreiD, could you perhaps try a newer build?
I reproduced the bug on the following build:
Mozilla/5.0 (Windows NT 6.1; rv:2.0b12pre) Gecko/20110208 Firefox/4.0b12pre

*Results 
1. After a 1st run the memory usage for the browser was 760 MB
2. After a 2nd run the memory usage for the browser was 974 MB
3. After a 3rd run the memory usage for the browser was 1.28 GB
4. During the 4th run Firefox crashed when reaching 1.7 GB . When trying to
report the crash I got the message "Oh Noes! This archived report could not be
located"

**Notes
1.Attached you can find screenshots of the first 3 cases mentioned above.
2.Fortunately i copied the report. You can find it attached.

IMO, having tried also without ADBLOCK addon installed and still getting a
crash when reaching 1.7 GB mem. usage, the slideshow in the example is a too
heavy load for Firefox.
AndreiD, could you try 20110209?
I think 20110208 doesn't have the patch for Bug 630947.
The patch for bug 630947 does not appear to stop this issue from happening. The memory still increases with today's build. I was however unable to crash Firefox. Memory usage topped out at 2.4 GB before it OS X started swapping stuff out to disk (! GB in total before I stopped). When memory usage grew to beyond 1 GB (as seen in activity monitor) Firefox became visibly sluggish.
Ok, next thing is perhaps to try with the patch for Bug 614347.

I wish I could reproduce. I'll try harder.
Tried a build with that patch and it didn't help, the memory still leaks.
I don't know if it is the same bug or not, but I see a big increase in memory usage when visiting the In Focus photo blog (run by the guy who started The Big Picture at Boston.com):

http://www.theatlantic.com/infocus/
We will try to get a Mozmill test up which hopefully can reproduce this issue.
HG bisected this (from range in comment 0, not comment 27, that is) to
https://hg.mozilla.org/mozilla-central/rev/3ea2b5a7c9c8
which originates from bug 624549

Seems the GC or CC doesn't run at all, when ABP is enabled, at least the memory stays 1GB+, even after a couple of minutes doing nothing or browsing something else. Without ABP the memory stays low the entire time; you can even see the memory drops because of the GC in the Task Manager.

I built up 1GB+ mem usage with ABP enabled and .garbageCollect() (via Error Console) will indeed free the memory again... But it is not freed automatically.
Nils, thanks!
Blocks: 624549
Keywords: regressionwindow-wanted
I'm sure not familiar with the CC code, but shouldn't this line:
https://hg.mozilla.org/mozilla-central/rev/3ea2b5a7c9c8#l2.114
   2.112 +  if (nsContentUtils::XPConnect() &&
   2.113 +      (aForceGC ||
>  2.114 +       (!GetGCRunsSinceLastCC() &&
   2.115 +        sCCSuspectedCount > NS_COLLECTED_OBJECTS_LIMIT))) {

instead be something like:
   2.112 +  if (nsContentUtils::XPConnect() &&
   2.113 +      (aForceGC ||
>  2.114 +       (GetGCRunsSinceLastCC() > NS_MAX_GC_COUNT &&
   2.115 +        sCCSuspectedCount > NS_COLLECTED_OBJECTS_LIMIT))) {
Depends on: 633149
Ok, I can reproduce that mem usage goes up quite a bit.
But it goes up even if I back out Bug 630947 and bug 624549.
Investigating.
I have created a Mozmill endurance test for this issue. You can see the results of running the steps to reproduce repeatedly on my Mozmill dashboard.

20 iterations, browsing the top ~20 pages with Adblock Plus installed:
http://mozmill.blargon7.com/#/endurance/report/aa40f05493d5cee5fc40cc7c15005c11

20 iterations, browsing the top ~20 pages without Adblock Plus installed:
http://mozmill.blargon7.com/#/endurance/report/aa40f05493d5cee5fc40cc7c150060f7

The endurance tests project hasn't landed yet, but we have enough to be able to replicate this sort of issue. Hopefully we'll land the initial prototype of the endurance tests by the end of this week or early next week.
A correction to my previous comment, the results show just 5 iterations and not 20.

I have now run this on a build from 29th January with Adblock Plus installed as requested, the results can be seen here: http://mozmill.blargon7.com/#/endurance/report/aa40f05493d5cee5fc40cc7c15006acd

This shows that the leak has been introduced since this build: Firefox 4.0b11pre (2.0b11pre, en-US, 20110129030338)
I see mention of private browsing, I cc ehsan.
Would anyone have time to test a tryserver build?
The builds are not ready yet, but will be here
http://ftp.mozilla.org/pub/mozilla.org/firefox/tryserver-builds/opettay@mozilla.com-d019f19e91ae/
Tried that build, but it doesn't appear to change much of anything. The memory usage still goes up as before but now some of it is released (~80MB of 650MB, down to 570MB (visited 10 pages of link in comment #1)).

Leak must be someplace else.
about:memory shows this:

Memory mapped:                    365,035,520
Memory in use:                    242,977,184
          
malloc/allocated                  242,981,392
malloc/mapped                     365,035,520
malloc/zone0/committed            242,976,208
malloc/zone0/allocated            361,889,792
js/gc-heap                         78,643,200
js/string-data                      5,087,386
js/mjit-code                       57,275,843
storage/sqlite/pagecache            5,340,656
storage/sqlite/other                1,661,424
images/chrome/used/raw                      0
images/chrome/used/uncompressed       276,324
images/chrome/unused/raw                    0
images/chrome/unused/uncompressed           0
images/content/used/raw                     0
images/content/used/uncompressed            0
images/content/unused/raw                   0
images/content/unused/uncompressed          0
layout/all                          4,002,103
layout/bidi                                 0
gfx/surface/image                  11,977,392
d.a. did you run exactly the steps mentioned in the comment 0?
This bug is about memory usage going up after the first
load-20-pages cycle.
I was watching the memory usage in activity monitor after every click and by the time I reached got to the 10th click my memory usage was 650MB, before I started it was ~250MB.
Well, could you please try the exact steps mentioned in the comment 0
and report what is the memory usage after the first 20 pages
and after the second 20 pages?
Perhaps someone should create a tryserver build with bug 625305's about:compartments info so this can be narrowed down successfully. The minimal JS info in comment 60 clearly doesn't account for the high memory use.
Well, if bug 624549 caused this, it is quite like that the tryserver
build fixes the problem (and it does seem to fix it at least locally).
The tryserver build has the following patch
https://bugzilla.mozilla.org/attachment.cgi?id=511444
Start point: 250MB
After first 20: 850MB
After second 20: 1GB
Thanks!
May I ask what kind of results you get using a nightly build 2011-01-29 ?
I went to the site I wrote about in comment 49 and it will release nearly all of the memory used by that site.
(In reply to comment #58)
> Would anyone have time to test a tryserver build?
> The builds are not ready yet, but will be here
> http://ftp.mozilla.org/pub/mozilla.org/firefox/tryserver-builds/opettay@mozilla.com-d019f19e91ae/

I did my own build with m-c tio with d019f19e91ae on top, as the windows release build didn't arrive yet (or is broken).
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b12pre) Gecko/20110210 Firefox/4.0b12pre
Core i7, 4 GB RAM box

- After start: 
win32/privatebytes 86,712,320
win32/workingset   93,650,944

- initial top99 front page:
win32/privatebytes 118,018,048
win32/workingset   127,557,632

- first 20page circle (immediately afterwards)
win32/privatebytes 733,773,824
win32/workingset   742,948,864

- first 20page circle (after memory stabilizes, ~1min)
win32/privatebytes 539,115,520
win32/workingset   547,340,288

- reloading top99 front page:
win32/privatebytes 557,805,568
win32/workingset   565,665,792

- second 20page circle (immediately afterwards)
win32/privatebytes 908,992,512
win32/workingset   916,918,272

- second 20page circle (after memory stabilizes, ~1min)
win32/privatebytes 785,264,640
win32/workingset   798,855,168

- reloading top99 front page:
win32/privatebytes 783,364,096
win32/workingset   796,958,720

- third 20page circle (immediately afterwards)
win32/privatebytes 1,044,586,496
win32/workingset   1,053,691,904

- third 20page circle (after memory stabilizes, ~1min)
win32/privatebytes 892,792,832
win32/workingset   906,878,976

- executing .garbageCollect() (via error console) until nothing changes anymore:
win32/privatebytes 851,566,592
win32/workingset   864,235,520

This is better than before, as memory is freed again. But it's still far from perfect.


- same build, without ABP, the memory stays stable around:
win32/privatebytes 159,858,688
win32/workingset   167,862,272
(+/- 40,000,000)


- 20110129 nightly with ABP, the memory stays stable between runs at about:
win32/privatebytes 398,979,072
win32/workingset   413,429,760
While still running the tryserver build I decided to test one more thing, not sure if it is related to this bug or not:

By simply running the V8 benchmark V6 the browser used up 880MB of memory, 550,000,000 (bytes?) or so were in js/gc-heap and it took me browsing around a few pages before this was released and now js/gc-heap contains only 50,000,000, total memory usage is 410 MB
(In reply to comment #65)
> Well, if bug 624549 caused this, it is quite like that the tryserver
> build fixes the problem (and it does seem to fix it at least locally).
> The tryserver build has the following patch
> https://bugzilla.mozilla.org/attachment.cgi?id=511444

I actually hg bisected to the corresponding changeset. I can recheck the changeset and compare it to it's parent, if you want me to, to exclude any human errors I may have made.
However there might be two or even more different issues at play here, where bug 624549 just added one.
And actually the try-build makes the situation better, hence to me it seems likely that there are different issues and the try-build fixed one of them.
With the 20110129 nightly with ABP:

about:memory says this a few minutes after I ran the test:

Memory mapped:            231,636,992
Memory in use:            110,493,488

malloc/allocated / malloc/mapped and malloc/zone0/committed / malloc/zone0/allocated show more or less the same.

At it's peak the run with ABP reached 500 MB of total memory, without it 400 MB, now it uses 357 MB / 290 MB without ABP.
(In reply to comment #57)
> I see mention of private browsing, I cc ehsan.

Sorry for the misdirection there, the private browsing mentioned in the report is simply another test that is being run. The report shows the results of the three endurance tests currently in existence, the first test runs the steps mentioned in comment 0 to replicate this issue. I left the other tests in the test run, as I thought it may be useful to see if the memory usage recovers.
@ comment 71

Using the hourly archive here: http://hourly-archive.localgho.st/hourly-archive2/

I can confirm that changeset: http://hg.mozilla.org/mozilla-central/rev/3ea2b5a7c9c8 does cause the bug

The next older build uses changeset: http://hg.mozilla.org/mozilla-central/rev/ba84db82eed5 which doesn't have the bug
> Memory mapped:            231,636,992
> Memory in use:            110,493,488

Don't trust "memory mapped" -- Firefox may be correctly calling free()/delete on heaps of blocks but the malloc implementation might be holding onto its pages (which isn't necessarily a bad thing;  their presence alone isn't necessarily costly).  "Memory in use" is the in-use heap measurement, and reflects better what Firefox is doing (even though it doesn't count code space used by the JITs, sigh).
Ok, I will investigate this some more. And I'll try with a leak patch...
Bug 630072 just got a patch that fixed one memory leak, but there isn't any tryserver build for it yet. I'd like to try a build with both patches included to see if there is an improvement.
(In reply to comment #76)
> Ok, I will investigate this some more. And I'll try with a leak patch...

I tested your try-server changeset directly on top of the changeset causing the regression (comment 51), as opposed to on top of the tip, and there is no apparent memory leak with that combination. The memory usage has a higher peak than before and takes longer to drop again, but it eventually drops.

So the try changeset fixes the issue I bisected in comment 51, but there is at least another (unrelated?) regression.
Thanks Nils, that is very useful information!
Thanks Nils again. I can certainly reproduce what you see.
Now need to find out what has caused the other memory usage problem.
Depends on: 611653
Should this be a beta 12 blocker instead of just final? Not my area, but seems like this would be a scary change to take between the last beta and RC...
(In reply to comment #81)
> Thanks Nils again. I can certainly reproduce what you see.
> Now need to find out what has caused the other memory usage problem.

Do you want me to bisect with your patch on top of each build? I can certainly try that.

Other than this, I suggest you fix the already bisected bug in m-c.
Any other issues might have been fixed in the meantime without us knowing, so an official nightly build with the fix will help checking if the other issue(s) are still present or already vanished in the mean time.
Looks like this is resolved in today's nightly...

Firefox 4.0b12pre (2.0b12pre, en-US, 20110215030353)

without any add-ons:
http://mozmill.blargon7.com/#/endurance/report/aa40f05493d5cee5fc40cc7c15010493

with AdblockPlus 1.3.3:
http://mozmill.blargon7.com/#/endurance/report/aa40f05493d5cee5fc40cc7c1500f7be

Yesterday's nightly:

Firefox 4.0b12pre (2.0b12pre, en-US, 20110214030347)

without any add-ons: 
http://mozmill.blargon7.com/#/endurance/report/aa40f05493d5cee5fc40cc7c1501261c

with AdblockPlus 1.3.3:
http://mozmill.blargon7.com/#/endurance/report/aa40f05493d5cee5fc40cc7c15010fb5
I'm still seeing this on my machine, although it takes a little longer to build up the RAM usage.
With 15th build I can confirm the ever increasing issue seems to be solved, although it does not release memory as quickly as before, so it will rise higher but then drop while browsing i.e. browsing the National Gallery site, zooming & panning images only took 130-160MB, constantly releasing memory with 29th and previous builds, but with 15th Feb build it rises to 270MB, but then has a sharp drop of 100MB, and will then rise again if continuing to browse.
http://nationalgallery.org.uk/paintings/collection-overview/

An issue that is still not fixed for askmen, Sky TV shop, News of the World, ABC Local or the National Gallery site is that memory is never released if you stop browsing, and even if you switch tab, it will only 5-10MB at the most, whereas with 29th and previous builds it would drop 80-100MB. If you have leave many sites open this will get to be a significant problem.
Depends on: 630932
Works For Me on:
 Mozilla/5.0 (Windows NT 6.1; rv:2.0b12pre) Gecko/20110216 Firefox/4.0b12pre

During a manual testing on askmen.com, nationalgallery.org.uk, and abc local the memory usage did not go above 400 MB (with AdBlock installed and on)
IMO, there is no more massive memory leak.
With the latest hourly memory peaks around 440 MB on my profile with ABP, and is able to release some of it when the tab is closed (currently using 345MB). So it appears that the ABP leak is fixed.

If I however add NoScript to the mix, it will leak, albeit not as much as before with Adblock Plus alone. After doing 5 runs memory peaked at around 1 GB, and released about 100 MB of it once the tab was closed.

I ran NoScript with the default settings while allowing scripts from AskMen.com to run.
d.a. could you file a new bug for that NoScript issue.
And please could you add exact steps to reproduce. Thanks.

I'm marking this one fixed.
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Filed bug 634855.
(In reply to comment #88)
> Works For Me on:
>  Mozilla/5.0 (Windows NT 6.1; rv:2.0b12pre) Gecko/20110216 Firefox/4.0b12pre
> 
> During a manual testing on askmen.com, nationalgallery.org.uk, and abc local
> the memory usage did not go above 400 MB (with AdBlock installed and on)
> IMO, there is no more massive memory leak.
Is that not still high, 400MB? If the 29th and previous builds used only 130-160MB while browsing the National Gallery site and now it uses more than 100MB+ more, is this problem really fixed?
Olli, Nils, d.a. - thanks a lot!
Reopened bug 631733 marked as duplicate as the memory never being released issue is not fixed as per comment 87.
Marking as verified fixed based on the results from comment 85 and other confirmations.
Status: RESOLVED → VERIFIED
Keywords: qawanted
Target Milestone: --- → mozilla2.0b12
I'm getting an impressive memmory leak while using ABP and latest nightlies. The difference between the same session and time with and without ABP is about 200-300MB, from 400 to 700MB of RAM used.

When this patch checked in, the memory usage decreased, but still uses a lot of memory and it is usually not released.

Should I file another bug with more info?
Yes, please do as soon as possible.
Maybe related to bug 637782?
No longer blocks: mlk2.0
[Tracking Requested - why for this release]:
blocking-b2g: --- → 2.0M?
tracking-firefox32: --- → ?
Flags: needinfo?(bugs)
Flags: needinfo?(bugs)
This is an old bug. If you see leaks with the most recent ABP, please file a new bug.
blocking-b2g: 2.0M? → ---
tracking-firefox32: ? → ---
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: