677212 - Many immortal zombie compartments with 70 add-ons enabled; ABP, GreaseMonkey, QuietURL, Redirect Remover are implicated

Status: RESOLVED INCOMPLETE

(Core :: General, defect)

Description

[IU]

Attachment: about:memory output of memory leak

Attached is verbose about:memory output showing lots of zombie compartments persisting and consuming memory, hours after their respective tabs had been closed -- even though I had used about:memory to manually force GC, GC+CC, and Minimize Memory Usage.  Total memory consumption for firefox.exe remains at 510-512 MB and will never get released (short of restarting); thus, this is a leak.  I've experienced the same issue with memory consumption of 700+ MB persisting.

In the attachment, the only tabs remaining open are the ones with the following URLs:
https://bugzilla.mozilla.org/enter_bug.cgi?product=Core
http://wiki.mikrotik.com/wiki/How_PCC_works_%28beginner%29

The remaining URLs are for tabs long closed [NOTE: I have edited out part or all of some of those URLs for privacy reasons].

It would be nice if Firefox had some way of tracking when a compartment should be destroyed (if that's even the right expression) -- maybe even having a "time to live" timer, for each compartment, that is triggered once the objects for a given compartment cease to exist.  I don't know.  Just throwing out ideas.

However, it is very clear that this is a leak and has nothing to do with performance, since the memory is never released: neither during extended idle periods nor when GC,CC, etc is forcibly requested.

Comment 1

[Justin Lebar (not reading bugmail)]

Can you please include the contents of about:support?

It would also be helpful to know whether you can reproduce this in safe mode, which will disable your extensions.  Many extensions are known to create zombie compartments.

Comment 2

[Boris Zbarsky [:bzbarsky]]

> maybe even having a "time to live" timer, for each compartment, that is
> triggered once the objects for a given compartment cease to exist.

Compartments are garbage-collected, and go away precisely when the all the objects for the compartment have ceased to exist.  If the compartments are still around, that means that someone is holding on to objects from those compartments.

Comment 3

[IU]

Since it can be known that something is holding onto those object, is it then possible to extend about:memory diagnostics to identify precisely what is holding onto the objects?

In my case, the surviving tabs had no cross-site/third-party objects associated with the zombies, as any such potential objects were blocked by the Request Policy extension.  I suppose, if not the browser itself, then it is possible some other extension(s) could have been holding onto the objects for some reason.  But then that raises the following question:

Is there much danger in destroying a compartment when its owner dies, regardless of what may be holding onto the objects?  If an extension tries to access the object, it could receive an exception.  If a user closes a tab, the user most likely deems those objects to be no longer of immediate importance.

I don't understand what happens in the third-party case, but maybe the objects can be relocated to the the third-party compartment, so memory can be freed up.

Comment 4

[IU]

(In reply to Justin Lebar [:jlebar] (out 8/12 - 8/21) from comment #1)
> Can you please include the contents of about:support?

Attaching...  Yes I have a lot of extensions.
 
> It would also be helpful to know whether you can reproduce this in safe
> mode, which will disable your extensions.  Many extensions are known to
> create zombie compartments.

I'll try tonight, but Firefox without some of my extension is painful. :-)

Comment 5

[IU]

Attachment: about:support

Comment 6

[Justin Lebar (not reading bugmail)]

Holy cow, that's a lot of extensions.  Your problem is almost certainly caused by one or more of them, so I'm marking this as MemShrink:P3, which is where we've been putting leaks caused by extensions.

> Is there much danger in destroying a compartment when its owner dies, 
> regardless of what may be holding onto the objects?

Yes, there's great danger: If the extension holds a strong reference to the object and we delete the object's compartment, then when the extension uses the reference, we'll have a use-after-free.

It's definitely problematic that extensions can cause leaks.  The only plan of ours to address this (that I'm aware of, anyway) is Jetpack.

Comment 7

[Jo Hermans]

AdBlock Plus is definitely a factor here. See bug 672111 for one (and I think there are more).

Comment 8

[IU]

(In reply to Jo Hermans from comment #7)
> AdBlock Plus is definitely a factor here. See bug 672111 for one (and I
> think there are more).

Thanks.  I don't use it much, so I'll disable it.

Comment 9

[Nicholas Nethercote [inactive]]

By my count you have 70 add-ons enabled and 141 disabled.  That's the highest I've ever seen.  Your about:addons page must be fun to navigate.

As Justin says, when there are that many installed the chances of them all being reliable is low, and the chances of them interacting in bad ways is high.  Although AdBlock Plus has been intimated as causing problems in bug 672111, it's one of the most widely used add-ons and is well-written.  I bet one or more of the other 69 are the problem.

Comment 10

[Nicholas Nethercote [inactive]]

IU:  Can you try selectively disabling add-ons in an attempt to pinpoint one or a few that are responsible.  The fastest way would be a binary search -- disable half, if the problem persists disable half of the remainder, etc.

I realize that's a pain but I guarantee you this bug won't go anywhere otherwise, because you're running a ton of non-Mozilla code in Firefox :(

Comment 11

[IU]

I started, but it will be a while before I get anywhere concrete. It seems there is some improvement with Adblock Plus and two other extensions disabled, but I need more testing to be sure.

Comment 12

[IU]

After some testing, I've been able to identify 4 of my extensions responsible for some of the zombie compartments (specifically those that were identified by site/page URL):

- QuietUrl
- Redirect Remover
- Adblock Plus
- Greasemonkey

The behavior of Adblock Plus and Greasemonkey are about the same in that they cause zombies for compartments associated with sites that matched user defined filters.

Of course, there are other compartments that accumulate over time, resulting in creeping memory increases that are never freed; however, the way those are identified (e.g. compartment([System Principal], 0xc8b6800)) is not helpful to me in tracking down any more possible causes.

Comment 13

[Nicholas Nethercote [inactive]]

IU: thanks for the info, that's very helpful.  What scripts are you running with GreaseMonkey?

Comment 14

[Nicholas Nethercote [inactive]]

BTW, I run ABP in my dev builds and have never noticed problems with zombie compartments.

Comment 15

[IU]

The Greasemonkey scripts are:
- Beautiful Meta Refresh Blocker
- Google Search Filter Plus
- Javascript refresh blocker (settimeout)
- YousableTubeFix
- YouTube JXS

Comment 16

[Nicholas Nethercote [inactive]]

ABP may be causing zombie compartments for people in some locations, but not in others.  See bug 679675.

Comment 17

[IU]

Thanks.  Looks like Wladimir is having difficulty reproducing with the supplied test case.  I'll comment to that bug once I come up with an easily reproducible test case.

Comment 18

[SpeciesX]

Attachment: about:memory2

same here, most of this pages i closing hours ago, but they are still in memory.

Comment 19

[Nicholas Nethercote [inactive]]

speciesx: please don't pile on in this bug -- it just confuses things, because your zombie compartments might have a different cause.  Can you please file a new bug, and list your add-ons when you do so?

Comment 20

[Nicholas Nethercote [inactive]]

See bug 672111 for more about AdBlock Plus and zombie compartments.

Comment 21

[Wladimir Palant]

Note that bug 672111 can explain *one* zombie compartment - Adblock Plus only holds a reference to at most one DOM node meaning that one compartment cannot be released. It will still be released as soon as a different web page makes a request.

Comment 22

[Nicholas Nethercote [inactive]]

IU, are you still seeing these zombie compartments?  Both ABP and GreaseMonkey have had zombie compartment fixes land lately.

Comment 23

[Wayne Mery (:wsmwk)]

(In reply to Nicholas Nethercote [:njn] (away until August 8) from comment #22)
> IU, are you still seeing these zombie compartments?  Both ABP and
> GreaseMonkey have had zombie compartment fixes land lately.

Comment 24

[Justin Lebar (not reading bugmail)]

Resolved INCOMPLETE because we haven't heard from the reporter in many months.  IU, please re-open this bug if you still have problems.