780331 - 2.5MiB/sec memory leak with 32 add-ons installed

Status: RESOLVED INVALID

(Firefox :: Untriaged, defect)

Description

[Stephan Sokolow]

Attachment: ram_usage.txt

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:16.0) Gecko/16.0 Firefox/16.0
Build ID: 20120726103218

Steps to reproduce:

Used my browser normally with my normal loadout of 32 extensions.


Actual results:

The memory usage ticked upward at a steady rate of 2.5MiB per second. As of this posting, it's at 5692MiB resident.


Expected results:

It should only increase when navigation of tabs to new URLs leaves behind the old URLs as zombie compartments. (Which, given my 32 extensions and the slow rate of leakage, I can't spare the time to diagnose by simply enabling and disabling addons to do a binary search)

Comment 1

[Stephan Sokolow]

Attachment: Output of Help > Troubleshooting Information

Comment 2

[Nicholas Nethercote [inactive]]

There's not enough info for us to go on, but bug 778318 might be relevant -- GreaseMonkey has a known bad leak and there's a development version that hopefully will fix it.

Comment 3

[Stephan Sokolow]

I do use YousableTubeFix but the vast majority of zombie compartments aren't on sites with anything YouTube related. (eg. comment pages on http://addventure.bast-enterprises.de/

I was aware of the issue with Greasemonkey but, since the original 0.9.21 XPI was broken and a.m.o wasn't making their fixed XPI public, I'd forgotten about it. An 0.9.22 update has been pushed to force a.m.o to serve a non-broken XPI, so I'll give that a try.

What additional details should I generally provide for something like this?

(Assuming someone can give sufficiently detailed instructions, I'd be happy to putter around with lower-level tools like GDB as long as I don't have to spend days without my addons while waiting for things to leak.)

Comment 4

[Nicholas Nethercote [inactive]]

> What additional details should I generally provide for something like this?
> 
> (Assuming someone can give sufficiently detailed instructions, I'd be happy
> to putter around with lower-level tools like GDB as long as I don't have to
> spend days without my addons while waiting for things to leak.)

Showing about:support, which lists all your add-ons, is great.  If you're lucky, someone might recognize an add-on in your list that is known to leak.  (In this case, GreaseMonkey is known to have problems, as mentioned;  Reddit Enhancement Suite has also been reported to cause high memory consumption but we've never been able to reproduce the problem.)

Otherwise, if you have a problem when 32 add-ons are enabled and you aren't willing to selectively disable them to narrow down the culprit(s), there's little chance of progress being made -- it's highly unlikely that any Firefox developer would have the time or patience to install the same 32 add-ons and bisect themselves :/

Comment 5

[Stephan Sokolow]

I did include the output of about:support. It's the attachment named "Output of Help > Troubleshooting Information". Is there another way to attach it that would be better?

Interestingly, I do also have Reddit Enhancement Suite though, given how I almost never visit Reddit these days, turning it off shouldn't be too big a deal.

I'm surprised there's no way to track down misbehaving addons besides a disgustingly manual binary search. How am I supposed to track down slow leaks like the old "leak up to 2+ GiB over the course of a 4-7 days" problem I've had since at least Firefox 4? (I'd be more willing to switch to Chrome than to give up things like NoScript for long enough to notice whether or not the leak is still present... and I hate using Chrome for day-to-day browsing.)

Is there no "you have to be skilled, but..." option? (I have next to no experience with languages like C/C++ which don't manage your memory for you and "bt full" is the limit of my GDB experience, but I'm willing to learn if it means that I can check off Firefox as the last hold-out on my "make normal use fit in 1GiB of RAM" TODO list.)

Comment 6

[Nicholas Nethercote [inactive]]

(In reply to Stephan Sokolow from comment #5)
> I did include the output of about:support. It's the attachment named "Output
> of Help > Troubleshooting Information". Is there another way to attach it
> that would be better?

Sorry if I was unclear;  by saying it's "great" I was intending to indicate that you had done the right thing.

> I'm surprised there's no way to track down misbehaving addons besides a
> disgustingly manual binary search. How am I supposed to track down slow
> leaks like the old "leak up to 2+ GiB over the course of a 4-7 days" problem
> I've had since at least Firefox 4?

I wish there was a good answer to that question :(  Slow leaks are always the hardest leaks to diagnose.  And configurations that vary significantly from standard (e.g. 32 add-ons) also make investigation harder.

Having said that, about:memory is the obvious starting point, so thanks for including that.  Your "heap-unclassified" is unusually high:

├──2,132.50 MB (37.61%) ── heap-unclassified

but digging into that is really hard, because it requires using DMD, which makes the browser run 20x slower than usual, which is hopeless for very slow leaks :(


You have tons of ghost windows, which are indicative of a certain kind of leak, very likely due to an add-on:

│  │  ├────562.52 MB (09.92%) ++ ghost


Also, AdBlock Plus is not behaving very nicely:

│  │  │  ├────346.91 MB (06.12%) -- compartment([System Principal], chrome://adblockplus-modules/content/RequestNotifier.jsm)

That looks like bug 764258, which was fixed in ABP 2.1, but you're running 2.0.3 -- you should update.  (I wonder why it hasn't auto-updated already...)


> Is there no "you have to be skilled, but..." option?

There are some additional hard-to-use tools.  I've CC'd mccr8, who knows more about them than I do.

Comment 7

[Andrew McCreight [:mccr8]]

The first thing to do is definitely to update the troublesome addons and see if that helps.

Potentially, dumping the JS heap may let you figure out why those leaking compartments are alive. But most of the tools we have are centered around figuring out a leak once it has been reduced to a minimal case. Hundreds of megabytes of JS is going to be really hard to get any comprehensive data out of.

It is frustrating, though.  For a while, I'd get a very slow leak in Gmail that would take four or five days to get bad.  Eventually it went away, but I was never sure if Gmail fixed a problem or it was a Firefox problem.

Comment 8

[Stephan Sokolow]

Looks like quite a few extensions weren't updating for some reason. When I manually checked for updates, I was left with AdBlock Plus, Canadian English Dictionary, Firebug, Lazarus: Form Recovery, NoScript, and StumbleUpon wanting a restart and, for all I know, there could have been some restartless ones updated too.

I'm leaving RES on to limit the number of variables being changed, but I'll let you know how updating everything to newest (including a manual update to Greasemonkey 0.9.22) affects things.

Oh, since I reported this, I've restarted twice and I've noticed that, until the leak gets up into the multi-gigabyte range, it's not as recognizable. When it's down in the 1.X GiB range things leak slowly enough and resident size (as reported by MemChaser) jumps up and down wildly enough that you can only notice the leak by watching what happens over the course of an hour or so. (My Firefox starts in the 1.X GiB range when I restore my session, so I don't know how it behaves with a lower resident size)

Comment 9

[Andrew McCreight [:mccr8]]

If you could also list the Greasemonkey scripts you run with that could be useful.

Comment 10

[Stephan Sokolow]

Actually, I really need to clean my userscripts out. Half of these, I'm not even sure if they work anymore because I don't go to the sites they modify anymore. (And, since they're supposed to only run on domains I never visit anyway, the cleanup keeps slipping my mind)

4chan Cleaner
anonib image fixer
Autocomplete Always On
/b/ackwash reloaded 5.5.7
BugMeNot
Codeshelver
Diff for gist.github
Download YouTube Captions
Experts Exchange
FFCMS Integration Helper 0.0.2 (Part of one of my own creations that's not yet public)
GitHub Markdown Preview 0.3.5
Google Book Downloader
Google books ad remover
Google Cache Continue Redux
Google Groups Redirect cut
Google Search AJAX Killer 1.0
ImageFAP direct images++
Linkify
Megaupload auto-fill captcha++ 0.2.2
Popup Window Fixer
Scrub Google Redirect Links
torrentz.com_More_Comments
Torrentz.eu: magnet and direct links 2.22
Try This Search On
Userscripts Updater
YousableTubeFix 2012.6.9

+ 4 disabled userscripts

Comment 11

[Stephan Sokolow]

Whoops. Forgot that Firefox sometimes "remembers" things it shouldn't when I reload the page.

Comment 12

[Stephan Sokolow]

Unrelated events have forced me to restart Firefox far more often than usual in the last few days, but I think forcing my addons to update might have solved this.

(So far, all I'm seeing is roughly 900 leaked about:blank compartments at 0.40MiB each and I think they appear with navigation rather than time)

I'll keep an eye on it for another few days and let you guys know.

Comment 13

[Nicholas Nethercote [inactive]]

Please let us know if you get more info!  Thanks.

Comment 14

[Stephan Sokolow]

I actually thought I'd marked it as resolved with an "it seems to have been fixed by the forced extension updates" but, looking back, it turned out the changes failed with "you last refreshed this tab more than 3 days ago. Changes rejected."