Closed Bug 752865 Opened 14 years ago Closed 14 years ago

CPG causes AdBlock Plus to use lots of cross-compartment wrappers

Categories

(Core :: JavaScript Engine, defect)

Product:
Core
Component:
JavaScript Engine
Platform:
x86
Windows 7
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED WORKSFORME
Tracking Flags:
Tracking Status
firefox15 - ---

People

(Reporter: speciesx, Unassigned)

References

Details

(Whiteboard: [js:waitingforinfo][MemShrink:P1])

Attachments

(2 files)

about:memory for current Adblock Plus development build
12.58 KB, text/plain
Details
about:memory for special Adblock Plus build
14.32 KB, text/plain
Details
User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) Gecko/15.0 Firefox/15.0a1 Build ID: 20120507030511 Steps to reproduce: Since Bug 650353 landed ABP use on average 100MB more memory. Actual results: In my test case i loaded 5 different pages + about:addons ABP enabled CPG build | non CPG build 442-467MB | 305-329MB ABP disabled CPG build | non CPG build 266-270MB | 236-246MB
Odd. Is ABP causing more compartments to be created where we didn't use compartments before? Seems weird.
Blocks: cpg
Tracking for investigation.
tracking-firefox15: --- → +
Adblock Plus core consists out of 20 JavaScript modules (23 on Fennec) - each of those has its own compartment now. And there is plenty of communication between them, particularly on startup. I wonder whether most of that memory is wasted on cross-comparment-wrappers... Side note: I am working on making Adblock Plus restartless, part of that work would be getting rid of JavaScript modules. That will bring us back to one compartment again.
When you say modules, are we talking XPCOM components, or subscripts, or something else? Presumably not sandboxes, since those had their own compartments all along....
We are talking about JSMs (JavaScript code modules as MDN calls them) - the kind of thing loaded by Components.utils.import().
From comment 3, it sounds like CPG would impose an initial fixed overhead but no additional per-tab overhead. Is that right? Does that match with what we are seeing?
Whiteboard: [MemShrink]
I tested a single-compartment Adblock Plus build (currently no UI integration but otherwise fully functional) - I can indeed see 50 MB less memory used after starting up Firefox with about:home as the only tab. There are no significant differences between the two builds and the overlay (which isn't being applied in the single-compartment build) shouldn't need 50 MB. So I strongly suspect that cross-compartment wrappers are the reason for the memory increase - the current release creates filter objects in one compartment but stores a reference to them in another.
(In reply to Wladimir Palant from comment #7) > So I strongly suspect that cross-compartment wrappers are the reason for the > memory increase - the current release creates filter objects in one > compartment but stores a reference to them in another. How many wrappers are we talking about? 50 MB is a _lot_ of wrappers...
Something above 30k wrappers (assuming at least one wrapper per filter).
(In reply to Wladimir Palant from comment #9) > Something above 30k wrappers (assuming at least one wrapper per filter). That would imply that a single cross-compartment wrapper has a >1KB overheard, which seems pretty high. Luke might correct me, but AFAIK a cross-compartment wrapper should be relatively small, on the order of 50 bytes. There's also a corresponding entry in the cross-compartment wrapper map, but again, a few bytes.
Agreed. Could you post the about:memory for the two? Perhaps we have a shape-splosion or some other secondary effect.
about:memory?verbose immediately after starting 2012-05-10 nightly, with a regular Adblock Plus 2.0.4b.3465.
This is about:memory for a build loading all code into a single sandbox (%7Bd10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d%7D.xpi!/bootstrap.js). The difference is smaller with a new profile, probably due to fewer filters (only EasyList rather than three different lists in the regular profile).
A few questions "Side note: I am working on making Adblock Plus restartless, part of that work would be getting rid of JavaScript modules. That will bring us back to one compartment again." will it not make it less effective? (FF major power is strong addons but restartless are not that powerful, are they?) "I tested a single-compartment Adblock Plus build (currently no UI integration but otherwise fully functional) - I can indeed see 50 MB less memory used after starting up Firefox with about:home as the only tab." Still it is on the higher side & can adblock plus use even less ram than it uses in FF13? P.S- No idea really so excuse me or educate me
Whiteboard: [MemShrink] → [MemShrink:P1]
(In reply to Wladimir Palant from comment #7) > I tested a single-compartment Adblock Plus build (currently no UI > integration but otherwise fully functional) How did you do this? Did you just dump all of your JS code into a single file? > So I strongly suspect that cross-compartment wrappers are the reason for the > memory increase - the current release creates filter objects in one > compartment but stores a reference to them in another. I have some patches to add measurement of cross-compartment wrappers in bug 755581. That'll let us test that hypothesis.
Depends on: 755581
(In reply to Nicholas Nethercote [:njn] from comment #15) > How did you do this? Did you just dump all of your JS code into a single > file? > It would appear so based on comment #13: >> This is about:memory for a build loading all code into a single sandbox (%7Bd10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d%7D.xpi!/bootstrap.js). >>
(In reply to Nicholas Nethercote [:njn] from comment #15) > How did you do this? Did you just dump all of your JS code into a single > file? No but that's the effect - subscript loader loads files into plain objects (not sandboxes) in bootstrap.js. That's what Element Hiding Helper and my other extensions are doing. I haven't checked in this work for Adblock Plus yet, still have to finish a few things.
This bug is for FF right(JavaScript Engine) So why does ABP need a rewrite? Plus will it affect other addons(relying on JavaScript Engine)
Whiteboard: [MemShrink:P1] → [js:t][MemShrink:P1]
my about memory on a fresh start with just a google tab open: Main Process Explicit Allocations 129.34 MB (100.0%) -- explicit ├───95.57 MB (73.89%) -- js │ ├──25.43 MB (19.66%) ── gc-heap-decommitted │ ├──17.40 MB (13.45%) ++ (142 tiny) │ ├──15.35 MB (11.86%) -- compartment([System Principal], chrome://adblockplus-modules/content/FilterClasses.jsm) │ │ ├──10.47 MB (08.09%) -- gc-heap │ │ │ ├───5.27 MB (04.08%) -- arena │ │ │ │ ├──5.23 MB (04.05%) ── unused │ │ │ │ └──0.04 MB (00.03%) ── headers │ │ │ ├───3.92 MB (03.03%) -- objects │ │ │ │ ├──3.91 MB (03.02%) ── non-function │ │ │ │ └──0.01 MB (00.01%) ── function │ │ │ └───1.27 MB (00.99%) ++ (3 tiny) │ │ ├───2.40 MB (01.86%) ── string-chars │ │ ├───1.50 MB (01.16%) ── cross-compartment-wrappers │ │ └───0.98 MB (00.76%) ++ (4 tiny) │ ├───7.27 MB (05.62%) -- compartment([System Principal], chrome://adblockplus-modules/content/FilterStorage.jsm) │ │ ├──3.75 MB (02.90%) -- gc-heap │ │ │ ├──1.53 MB (01.18%) -- arena │ │ │ │ ├──1.51 MB (01.17%) ── unused │ │ │ │ └──0.01 MB (00.01%) ── headers │ │ │ ├──1.47 MB (01.14%) -- objects │ │ │ │ ├──1.46 MB (01.13%) ── non-function │ │ │ │ └──0.01 MB (00.01%) ── function │ │ │ └──0.75 MB (00.58%) ++ (2 tiny) │ │ ├──3.00 MB (02.32%) ── cross-compartment-wrappers │ │ └──0.53 MB (00.41%) ++ (4 tiny) │ ├───5.63 MB (04.35%) -- compartment([System Principal], chrome://adblockplus-modules/content/ElemHide.jsm) │ │ ├──2.54 MB (01.96%) ++ gc-heap │ │ ├──1.59 MB (01.23%) ++ (4 tiny) │ │ └──1.50 MB (01.16%) ── cross-compartment-wrappers │ ├───5.39 MB (04.17%) -- compartment(atoms) │ │ ├──3.27 MB (02.53%) ── string-chars │ │ ├──2.12 MB (01.64%) -- gc-heap │ │ │ ├──1.71 MB (01.32%) ── strings │ │ │ └──0.40 MB (00.31%) ++ (2 tiny) │ │ └──0.00 MB (00.00%) ── other-sundries │ ├───5.05 MB (03.90%) -- compartment([System Principal], chrome://adblockplus-modules/content/Matcher.jsm) │ │ ├──3.15 MB (02.43%) -- gc-heap │ │ │ ├──1.69 MB (01.31%) ++ (4 tiny) │ │ │ └──1.46 MB (01.13%) -- arena │ │ │ ├──1.45 MB (01.12%) ── unused │ │ │ └──0.01 MB (00.01%) ── headers │ │ └──1.90 MB (01.47%) ++ (5 tiny) │ ├───3.41 MB (02.63%) -- compartment(https://www.google.co.nz/) │ │ ├──1.90 MB (01.47%) ++ (6 tiny) │ │ └──1.51 MB (01.17%) ++ gc-heap │ ├───3.37 MB (02.61%) -- compartment([System Principal], chrome://adblockplus-modules/content/SubscriptionClasses.jsm) │ │ ├──1.53 MB (01.18%) -- gc-heap │ │ │ ├──1.47 MB (01.14%) -- objects │ │ │ │ ├──1.46 MB (01.13%) ── non-function │ │ │ │ └──0.01 MB (00.01%) ── function │ │ │ └──0.06 MB (00.05%) ++ (2 tiny) │ │ ├──1.50 MB (01.16%) ── cross-compartment-wrappers │ │ └──0.34 MB (00.26%) ++ (2 tiny) │ ├───3.06 MB (02.37%) -- runtime │ │ ├──2.00 MB (01.55%) ── atoms-table │ │ └──1.06 MB (00.82%) ++ (12 tiny) │ ├───2.64 MB (02.04%) ++ compartment([System Principal], about:blank) │ └───1.57 MB (01.21%) -- compartment([System Principal], chrome://adblockplus-modules/content/FilterListener.jsm) │ ├──1.50 MB (01.16%) ── cross-compartment-wrappers │ └──0.07 MB (00.05%) ++ (2 tiny) ├───17.57 MB (13.58%) ── heap-unclassified ├────4.90 MB (03.79%) -- storage │ ├──4.90 MB (03.79%) -- sqlite │ │ ├──2.97 MB (02.29%) ++ (12 tiny) │ │ └──1.93 MB (01.49%) ── other │ └──0.00 MB (00.00%) ── prefixset/all ├────4.06 MB (03.14%) -- images │ ├──3.85 MB (02.97%) -- content │ │ ├──3.73 MB (02.88%) -- used │ │ │ ├──3.37 MB (02.61%) ── uncompressed-heap │ │ │ └──0.36 MB (00.28%) ++ (2 tiny) │ │ └──0.12 MB (00.09%) ++ unused │ └──0.21 MB (00.17%) ++ chrome ├────3.41 MB (02.64%) -- window-objects │ ├──1.91 MB (01.48%) -- top(https://www.google.co.nz/, id=8)/active │ │ ├──1.91 MB (01.47%) -- window(https://www.google.co.nz/) │ │ │ ├──1.50 MB (01.16%) -- layout │ │ │ │ ├──1.33 MB (01.03%) ── style-sets │ │ │ │ └──0.17 MB (00.13%) ++ (3 tiny) │ │ │ └──0.41 MB (00.32%) ++ (2 tiny) │ │ └──0.00 MB (00.00%) ── window(about:blank)/dom [2] │ └──1.50 MB (01.16%) ++ (3 tiny) ├────2.30 MB (01.78%) ++ (7 tiny) └────1.53 MB (01.18%) ── atom-table Other Measurements 129.34 MB ── explicit 0.00 MB ── gfx-d2d-surfacecache 4.00 MB ── gfx-d2d-surfacevram 3.62 MB ── gfx-surface-image 0 ── ghost-windows 65.90 MB ── heap-allocated 81.61 MB ── heap-committed 23.80% ── heap-committed-unused-ratio 2.75 MB ── heap-dirty 67.09 MB ── heap-unused 3.37 MB ── images-content-used-uncompressed 148 ── js-compartments-system 4 ── js-compartments-user 62.00 MB ── js-gc-heap 2.93 MB ── js-main-runtime-analysis-temporary 21.35 MB ── js-main-runtime-gc-heap-allocated 15.13 MB ── js-main-runtime-gc-heap-arena-unused 0.00 MB ── js-main-runtime-gc-heap-chunk-clean-unused 0.08 MB ── js-main-runtime-gc-heap-chunk-dirty-unused 36.57 MB ── js-main-runtime-gc-heap-committed 15.22 MB ── js-main-runtime-gc-heap-committed-unused 71.26% ── js-main-runtime-gc-heap-committed-unused-ratio 25.43 MB ── js-main-runtime-gc-heap-decommitted 0.28 MB ── js-main-runtime-mjit 13.71 MB ── js-main-runtime-objects 5.14 MB ── js-main-runtime-scripts 5.95 MB ── js-main-runtime-shapes 11.14 MB ── js-main-runtime-strings 0.70 MB ── js-main-runtime-type-inference 0 ── low-commit-space-events 0 ── low-memory-events-physical 0 ── low-memory-events-virtual 195.23 MB ── private 242.25 MB ── resident 4.90 MB ── storage-sqlite 491.07 MB ── vsize 0.38 MB ── window-objects-dom 0.57 MB ── window-objects-layout-arenas 0.02 MB ── window-objects-layout-pres-contexts 1.77 MB ── window-objects-layout-style-sets 0.00 MB ── window-objects-layout-text-runs 0.68 MB ── window-objects-style-sheets Is this related?
> │ │ ├───1.50 MB (01.16%) ── cross-compartment-wrappers > │ │ ├──3.00 MB (02.32%) ── cross-compartment-wrappers > │ │ └──1.50 MB (01.16%) ── cross-compartment-wrappers > │ │ ├──1.50 MB (01.16%) ── cross-compartment-wrappers > │ ├──1.50 MB (01.16%) ── cross-compartment-wrappers > > Is this related? Yes! You have at least 9MB of cross-compartment wrappers there. I just checked my own browser, where I have ABP installed, and I only had 1.5MB of cross-compartment wrappers. Still, I think it's fair to say cross-compartment wrappers are the problem. Wladimir, I guess the fix here is your single-compartment version of ABP? (You can do your own testing now that about:memory reports on cross-compartment wrappers.) Or can we make cross-compartment wrappers smaller?
Summary: ABP uses a lot more memory after Bug 650353 landed → CPG causes AdBlock Plus to use lots of cross-compartment wrappers
With the new ABP Version 2.1b.3474 it is alot better. non CPG build 308 MB 330 MB 326 MB 328 MB VS current nightly 359 MB 363 MB 360 MB 362 MB
Adblock Plus 2.1b.3474 gets rid of most cross-compartment communication, there are only small changes otherwise. I guess that CPG is indeed the cause of increased memory use then, for whatever reason.
I'm noticing a much improved experience and much less RAM usage since the changes landed - thanks Wladimir!
while using same filters FF13 uses 140+mb whereas FF16 uses 220+mb Tried a new profile same results
What's the status of this? Comment 23 seems to say it's better with the new ABP, but comment 24 seems to say it's still a problem.
Whiteboard: [js:t][MemShrink:P1] → [js:waitingforinfo][MemShrink:P1]
Its fixed, I am thinking he might be using a non-dev version. Heres my new about:memory (I use FanBoy ultimate list, so might use more RAM then most peoples) │ ├───49,926,540 B (08.81%) -- compartment([System Principal], jar:file:///C:/Users/xxxxxxx/AppData/Roaming/Mozilla/Firefox/Profiles/xxxx.default/extensions/%xxxxx-f5b5-c8b4-a8b2-2b9879e08c5d%7D.xpi!/bootstrap.js) │ │ ├──36,843,520 B (06.50%) -- gc-heap │ │ │ ├──17,682,528 B (03.12%) -- arena │ │ │ │ ├──17,538,608 B (03.10%) ── unused [2] │ │ │ │ └─────143,920 B (00.03%) ── headers [2] │ │ │ ├───9,217,712 B (01.63%) -- objects │ │ │ │ ├──9,121,328 B (01.61%) ── non-function [2] │ │ │ │ └─────96,384 B (00.02%) ── function [2] │ │ │ ├───5,222,256 B (00.92%) -- shapes │ │ │ │ ├──4,943,280 B (00.87%) ── dict [2] │ │ │ │ ├────140,160 B (00.02%) ── tree [2] │ │ │ │ └────138,816 B (00.02%) ── base [2] │ │ │ ├───4,616,272 B (00.81%) ── strings [2] │ │ │ ├──────89,856 B (00.02%) ── scripts [2] │ │ │ ├───────8,512 B (00.00%) ── type-objects │ │ │ └───────6,384 B (00.00%) ── sundries [2] │ │ ├───5,337,616 B (00.94%) -- objects │ │ │ ├──4,380,416 B (00.77%) ── slots [2] │ │ │ └────957,200 B (00.17%) ── elements [2] │ │ ├───5,187,628 B (00.92%) ── string-chars [2] │ │ ├───1,913,152 B (00.34%) -- shapes-extra │ │ │ ├──1,781,568 B (00.31%) ── dict-tables [2] │ │ │ ├─────54,304 B (00.01%) ── tree-tables [2] │ │ │ ├─────50,176 B (00.01%) ── compartment-tables [2] │ │ │ └─────27,104 B (00.00%) ── tree-shape-kids [2] │ │ ├─────354,432 B (00.06%) ── script-data [2] │ │ ├─────262,144 B (00.05%) ── analysis-temporary [2] │ │ ├──────16,384 B (00.00%) ── cross-compartment-wrappers │ │ └──────11,664 B (00.00%) ── other-sundries [2] The fix combined apartments and now seems to use far less RAM then earlier.
Thanks, Grant. I'm marking WFM. Anyone should please reopen if they can provide solid evidence something bad is still happening (or, the badness is significantly different from the original version, please file a new bug to reduce confusion).
Status: UNCONFIRMED → RESOLVED
Closed: 14 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: