Last Comment Bug 764258 - Adblock Plus RequestNotifier.jsm compartment gets very large with Mibbit
: Adblock Plus RequestNotifier.jsm compartment gets very large with Mibbit
Status: VERIFIED FIXED
[MemShrink:P2]
:
Product: Tech Evangelism
Classification: Other
Component: Add-ons (show other bugs)
: unspecified
: x86 Windows 7
: -- normal with 1 vote (vote)
: ---
Assigned To: Wladimir Palant
:
:
Mentors:
Depends on:
Blocks: LeakyAddons 780331
  Show dependency treegraph
 
Reported: 2012-06-12 20:38 PDT by Hugh Nougher [:Hughman]
Modified: 2012-08-07 18:44 PDT (History)
9 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
Multi-compartment Adblock Plus 2.1 build (730.24 KB, application/x-xpinstall)
2012-06-28 02:15 PDT, Wladimir Palant
no flags Details

Description Hugh Nougher [:Hughman] 2012-06-12 20:38:24 PDT
Previously when using Mibbit I have seen growing memory usage in my Firefox and now with FF15 its easier to work out what is doing it. Yay!

This is a snippet of by about:memory
│  ├──116,869,048 B (13.27%) -- compartment([System Principal], chrome://adblockplus-modules/content/RequestNotifier.jsm)
│  │  ├───58,292,272 B (06.62%) ── string-chars
│  │  ├───29,638,656 B (03.36%) -- gc-heap
│  │  │   ├──17,866,960 B (02.03%) -- objects
│  │  │   │  ├──17,840,592 B (02.03%) ── non-function
│  │  │   │  └──────26,368 B (00.00%) ── function
│  │  │   ├──11,583,568 B (01.31%) ── strings
│  │  │   ├─────142,416 B (00.02%) -- arena
│  │  │   │     ├──115,776 B (00.01%) ── headers
│  │  │   │     └───26,640 B (00.00%) ── unused
│  │  │   ├──────37,720 B (00.00%) -- shapes
│  │  │   │      ├──21,720 B (00.00%) ── tree
│  │  │   │      └──16,000 B (00.00%) ── base
│  │  │   └───────7,992 B (00.00%) ── sundries
│  │  ├───27,988,544 B (03.18%) -- objects
│  │  │   ├──23,173,056 B (02.63%) ── slots
│  │  │   └───4,815,488 B (00.55%) ── elements
│  │  ├──────786,432 B (00.09%) ── cross-compartment-wrappers
│  │  ├──────131,072 B (00.01%) ── analysis-temporary
│  │  ├───────22,528 B (00.00%) ── shapes-extra/compartment-tables
│  │  └────────9,544 B (00.00%) ── other-sundries

If I reload the Mibbit tab this memory suddenly all gets freed.

Filter Lists are:
- Fanboy's List
- Adversity Adult
- Antisocial

I have full verbose about:memory before/after and about:support saved and on hand if more info is needed.
Comment 1 Wladimir Palant 2012-06-13 02:02:08 PDT
For reference: the URL for testing is http://chat.mibbit.com/. It performs a bunch of image requests per second for each user displayed in the chat but I cannot see why there would be a leak here - data about these requests will be garbage collected along with the image nodes. So far that's what I see - memory use of the Adblock Plus compartment grows constantly but that memory is completely reclaimed when I press "Minimize memory usage". If I don't click it the memory use stabilizes with roughly 130 kB of memory that could be reclaimed. Then again, I didn't manage to find a chat room where a conversation would be going on.
Comment 2 Justin Lebar (not reading bugmail) 2012-06-14 08:58:47 PDT
> Then again, I didn't manage to find a chat room where a conversation would be going on.

Try #ubuntu and ##linux on irc.freenode.net.
Comment 3 Hugh Nougher [:Hughman] 2012-06-18 22:18:45 PDT
(In reply to Justin Lebar [:jlebar] from comment #2)
> Try #ubuntu and ##linux on irc.freenode.net.

Freenode is out since mibbit is banned there.

(In reply to Wladimir Palant from comment #1)
> memory use of the Adblock Plus compartment grows constantly but that memory
> is completely reclaimed when I press "Minimize memory usage".

Nothing changes when pressing "Minimize memory usage" for my tests.

This time have to ran a test in a new profile of
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:15.0) Gecko/20120617 Firefox/15.0a2
with just Adblock Plus 2.0.3 Fanboy addon against mozilla #firefox, #developers, #jsapi and #gfx (fairly active channels). Compartment started around 0MB and reached 75MB in 1 hour.
Comment 4 Nicholas Nethercote [:njn] 2012-06-26 16:21:57 PDT
Wladimir: can you try #developer on irc.mozilla.org?  That has a reasonable amount of traffic.
Comment 5 Wladimir Palant 2012-06-26 22:32:38 PDT
(In reply to Nicholas Nethercote [:njn] from comment #4)
> Wladimir: can you try #developer on irc.mozilla.org?  That has a reasonable
> amount of traffic.

Not in my time zone - I actually tried that when I wrote my comment. But I will try it once more.
Comment 6 scientes 2012-06-26 22:36:54 PDT
try #debian on irc.oftc.net
Comment 7 scientes 2012-06-26 22:37:08 PDT
irc://irc.oftc.net/debian
Comment 8 Kris Maglione [:kmag] 2012-06-27 09:26:20 PDT
I can reproduce this quite easily. I joined #developers and #firefox on irc.mozilla.org and a couple of big channels on Rizon. The only channel with any activity is #developers (and it's really not a lot of activity), and after a few minutes that compartment is up to 46MB and still growing. Minimize memory makes only a minor dent.

│  ├───46,512,568 B (14.62%) -- compartment([System Principal], chrome://adblockplus-modules/content/RequestNotifier.jsm)
│  │   ├──21,323,776 B (06.70%) -- gc-heap
│  │   │  ├──12,369,952 B (03.89%) ── strings
│  │   │  ├───8,595,680 B (02.70%) -- objects
│  │   │  │   ├──8,585,760 B (02.70%) ── non-function
│  │   │  │   └──────9,920 B (00.00%) ── function
│  │   │  ├─────340,920 B (00.11%) -- arena
│  │   │  │     ├──166,592 B (00.05%) ── headers
│  │   │  │     ├──134,168 B (00.04%) ── padding
│  │   │  │     └───40,160 B (00.01%) ── unused
│  │   │  ├───────8,984 B (00.00%) ── sundries
│  │   │  └───────8,240 B (00.00%) ── shapes/tree
│  │   ├──14,358,336 B (04.51%) ── string-chars
│  │   ├──10,543,904 B (03.32%) -- objects
│  │   │  ├───8,250,752 B (02.59%) ── slots
│  │   │  └───2,293,152 B (00.72%) ── elements
│  │   ├─────196,608 B (00.06%) ── cross-compartment-wrappers
│  │   ├──────82,944 B (00.03%) ── shapes-extra/compartment-tables
│  │   └───────7,000 B (00.00%) ── other-sundries
Comment 9 Justin Lebar (not reading bugmail) 2012-06-27 09:33:16 PDT
> The only channel with any activity is #developers (and it's really not a lot of 
> activity), and after a few minutes that compartment is up to 46MB and still growing.

How big was the compartment initially?
Comment 10 Kris Maglione [:kmag] 2012-06-27 09:36:42 PDT
Relatively small. ~95K:

│   ├──────96,136 B (00.09%) -- compartment([System Principal], chrome://adblockplus-modules/content/RequestNotifier.jsm)
│   │      ├──69,632 B (00.07%) -- gc-heap
│   │      │  ├──38,536 B (00.04%) ── arena/unused
│   │      │  ├──21,176 B (00.02%) ── sundries
│   │      │  └───9,920 B (00.01%) ── objects/function
│   │      ├──18,312 B (00.02%) ── other-sundries
│   │      └───8,192 B (00.01%) ── cross-compartment-wrappers
Comment 11 Kris Maglione [:kmag] 2012-06-27 12:45:07 PDT
It's probably worth noting that memory usage returns to normal once the Mibbit tab is closed.
Comment 12 Wladimir Palant 2012-06-27 12:51:20 PDT
Kris, please test with the release candidate for Adblock Plus 2.1 as well (https://adblockplus.org/devbuilds/adblockplus/). I tried doing that myself today but Mibbit stopped connecting to servers. This sounds like it is an issue with user data - Adblock Plus 2.1 uses weak maps instead.
Comment 13 Kris Maglione [:kmag] 2012-06-27 13:23:15 PDT
The problem seems somewhat improved with the RC, as far as I can tell. Memory growth seems slower, and more of it is reclaimed on 'minimize memory', but it still grows. The reporting is less fine grained now that you use a single compartment, but I'm now seeing, after 10 minutes or so on #developers, the footprint of your bootstrap compartment grow from ~32MB to ~46MB after 15 minutes or so.

│  ├───46,435,912 B (18.29%) -- compartment([System Principal], file:///home/kris/review-profiles/firefox15/extensions/%7Bd10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d%7D/bootstrap.js)
│  │   ├──36,831,232 B (14.51%) -- gc-heap
│  │   │  ├──19,501,184 B (07.68%) -- arena
│  │   │  │  ├──19,003,720 B (07.49%) ── unused
│  │   │  │  ├─────287,744 B (00.11%) ── headers
│  │   │  │  └─────209,720 B (00.08%) ── padding
│  │   │  ├───7,236,160 B (02.85%) ── strings
│  │   │  ├───7,203,536 B (02.84%) -- objects
│  │   │  │   ├──7,105,424 B (02.80%) ── non-function
│  │   │  │   └─────98,112 B (00.04%) ── function
│  │   │  ├───2,822,600 B (01.11%) -- shapes
│  │   │  │   ├──2,667,480 B (01.05%) ── dict
│  │   │  │   ├────107,520 B (00.04%) ── tree
│  │   │  │   └─────47,600 B (00.02%) ── base
│  │   │  ├──────64,056 B (00.03%) ── scripts
│  │   │  └───────3,696 B (00.00%) ── sundries
│  │   ├───6,341,360 B (02.50%) -- objects
│  │   │   ├──4,689,344 B (01.85%) ── slots
│  │   │   └──1,652,016 B (00.65%) ── elements
│  │   ├───1,686,896 B (00.66%) ── string-chars
│  │   ├───1,216,256 B (00.48%) -- shapes-extra
│  │   │   ├──1,053,088 B (00.41%) ── dict-tables
│  │   │   ├─────98,304 B (00.04%) ── compartment-tables
│  │   │   ├─────42,688 B (00.02%) ── tree-tables
│  │   │   └─────22,176 B (00.01%) ── tree-shape-kids
│  │   ├─────213,400 B (00.08%) ── script-data
│  │   ├─────131,072 B (00.05%) ── analysis-temporary
│  │   ├───────8,192 B (00.00%) ── cross-compartment-wrappers
│  │   └───────7,504 B (00.00%) ── other-sundries
Comment 14 Justin Lebar (not reading bugmail) 2012-06-27 15:00:16 PDT
The 32mb to 46mb growth is measured before or after minimizing memory usage?

If memory usage increases, then falls back to "normal" levels after minimizing memory usage, that's likely not an ABP bug.
Comment 15 Kris Maglione [:kmag] 2012-06-27 15:04:09 PDT
The 46MB number is after minimizing memory. It was, as I recall, around 57MB before, and I'd minimized memory periodically while I was testing, so I can't say for certain how high it would have gone if I hadn't.
Comment 16 Wladimir Palant 2012-06-28 02:15:24 PDT
Created attachment 637419 [details]
Multi-compartment Adblock Plus 2.1 build

Added a multi-compartment build that makes testing a bit easier. For some reason I got kicked out again after only a few minutes but I could see requestNotifier.js memory going up slowly (roughly 100 kB per minute) when connected to #developers on irc.mozilla.org. This doesn't actually seem to be related to chat activity, rather being caused by the user list constantly reloading.
Comment 17 Wladimir Palant 2012-06-28 03:09:19 PDT
I tried the CC analyzer from bug 726346 (collecting all traces requires inserting a checkbox with ID alltraces via Inspector and checking it) and got some very strange results. The memory is occupied by structures like this:

> 0x3281e8e0 [gc.marked] JS Object (Array)
>     > 0xc8fa020 type_proto
>     > 0xc8f7040 parent
>     > 0x3281e8b0 element[0]
>     > 0x24ed35e0 element[1]
> 0x3281e8b0 [gc.marked] JS Object (Object)
>     > 0xca015b0 type_proto
>     > 0xc8f7040 parent
> 0x24ed35e0 [gc.marked] JS Object (Object)
>     > 0xca015b0 type_proto
>     > 0xc8f7040 parent

0xca015b0 points to RequestEntry prototype - as expected, these are remembered requests. 0xc8f7040 points to the requestNotifier.js sandbox. The strange thing is that there are no references to 0x3281e8e0 - nothing is preventing this array from getting garbage collected. Normally a reference to the array is being held by a WeakMap. I'm not sure whether this is simply a limitation of the cc analyzer or WeakMap actually leaking memory.
Comment 18 Wladimir Palant 2012-06-28 04:47:37 PDT
Never mind, the above is indeed a limitation of the cc analyzer. I eventually figured out that this site makes Adblock Plus remember the same request multiple times because it resets the src property of image elements regularly. I fixed this: https://hg.adblockplus.org/adblockplus/rev/85abc71766cc
Comment 19 Wladimir Palant 2012-06-28 08:18:41 PDT
Adblock Plus 2.1 has been released: https://addons.mozilla.org/addon/adblock-plus/versions/#version-2.1

Note You need to log in before you can comment on or make changes to this bug.