Last Comment Bug 670162 - Bugzilla tweaks uses a lot of memory in the System Principal compartment
: Bugzilla tweaks uses a lot of memory in the System Principal compartment
Status: RESOLVED FIXED
[MemShrink:P2]
:
Product: Core
Classification: Components
Component: General (show other bugs)
: Trunk
: x86_64 Linux
-- normal (vote)
: ---
Assigned To: Drew Willcoxon :adw
:
:
Mentors:
Depends on: 671159 675955
Blocks:
  Show dependency treegraph
 
Reported: 2011-07-08 09:18 PDT by Mike Hommey [:glandium]
Modified: 2012-01-14 00:21 PST (History)
14 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments

Description User image Mike Hommey [:glandium] 2011-07-08 09:18:48 PDT
I first noticed that in a session with 50 bugzilla tabs open, with the Bugzilla tweaks extension enabled, and disabling the extension made the System Principal compartment memory usage drop by around 200M.

In order to get more constrained data, I took the last nightly, created a new profile, set the default to restore last session when starting, logged in on bugzilla, and opened 10 bugs in 10 tabs (644707 and following, skipping those I don't have access to), then opened about:memory in an 11th tab.
│   ├──14,560,398 B (11.67%) -- compartment([System Principal])
│   │  ├───8,032,256 B (06.44%) -- gc-heap
│   │  │   ├──4,238,664 B (03.40%) -- objects
│   │  │   ├──2,991,872 B (02.40%) -- shapes
│   │  │   ├────574,640 B (00.46%) -- arena-unused
│   │  │   ├────124,720 B (00.10%) -- arena-padding
│   │  │   ├─────53,952 B (00.04%) -- strings
│   │  │   ├─────47,064 B (00.04%) -- arena-headers
│   │  │   └──────1,344 B (00.00%) -- xml
│   │  ├───2,555,904 B (02.05%) -- mjit-code
│   │  ├───2,322,964 B (01.86%) -- scripts
│   │  ├─────647,336 B (00.52%) -- object-slots
│   │  ├─────570,760 B (00.46%) -- mjit-data
│   │  ├─────230,312 B (00.18%) -- tjit-data
│   │  │     ├──148,000 B (00.12%) -- allocators-reserve
│   │  │     └───82,312 B (00.07%) -- allocators-main
│   │  ├─────131,072 B (00.11%) -- tjit-code
│   │  └──────69,794 B (00.06%) -- string-chars

Then, installed the addon compatibility reporter extension, then bugzilla tweaks, and restarted.
After that, memory usage looks like this:
│  ├───43,577,021 B (20.92%) -- compartment([System Principal])
│  │   ├──22,474,752 B (10.79%) -- gc-heap
│  │   │  ├───9,306,464 B (04.47%) -- arena-unused
│  │   │  ├───7,484,104 B (03.59%) -- objects
│  │   │  ├───5,050,880 B (02.42%) -- shapes
│  │   │  ├─────330,752 B (00.16%) -- arena-padding
│  │   │  ├─────169,632 B (00.08%) -- strings
│  │   │  ├─────131,688 B (00.06%) -- arena-headers
│  │   │  └───────1,232 B (00.00%) -- xml
│  │   ├──12,517,376 B (06.01%) -- mjit-code
│  │   ├───3,082,923 B (01.48%) -- scripts
│  │   ├───3,056,000 B (01.47%) -- tjit-data
│  │   │   ├──2,368,000 B (01.14%) -- allocators-reserve
│  │   │   └────688,000 B (00.33%) -- allocators-main
│  │   ├───1,201,200 B (00.58%) -- object-slots
│  │   ├─────705,180 B (00.34%) -- mjit-data
│  │   ├─────408,518 B (00.20%) -- string-chars
│  │   └─────131,072 B (00.06%) -- tjit-code

These are however (in both cases), low boundaries, because memory usage on that compartment varies a lot, in quite fast cycles, when about:memory is reloaded (up to 40M more in both cases).

Hopefully, this should be easily reproducible.
Comment 1 User image Boris Zbarsky [:bz] (still a bit busy) 2011-07-08 09:54:44 PDT
So the main jumps seem to be mjit-code and arena-unused.

Does bugzilla tweaks end up using sandboxes?  Would system sandboxes end up with separate compartments that get lumped under here, possibly?
Comment 2 User image Josh Matthews [:jdm] 2011-07-08 11:12:48 PDT
Bugzilla Tweaks is a bootstrapped add-on, which use sandboxes to run code.
Comment 3 User image Boris Zbarsky [:bz] (still a bit busy) 2011-07-08 11:15:08 PDT
Right.  Would those be in the system compartment for purposes of about:memory?  How long do they end up sticking around?
Comment 4 User image Dave Townsend [:mossop] 2011-07-08 11:28:56 PDT
(In reply to comment #3)
> Right.  Would those be in the system compartment for purposes of
> about:memory?  How long do they end up sticking around?

I'd say it is likely that they end up in the system compartment (the sandbox is created with the system principal) but maybe Blake would know better.

Incidentally if we had a way to assign these sandboxes to a named compartment then we could have a little bit of per-add-on accounting going on here.

The add-ons manager keeps a reference to the sandbox so really anything that bugzilla tweaks and keeps a reference to will stick around until the add-on is disabled or we exit.
Comment 5 User image Kyle Huey [:khuey] (Exited; not receiving bugmail, email if necessary) 2011-07-08 11:34:37 PDT
(In reply to comment #4)
> (In reply to comment #3)
> > Right.  Would those be in the system compartment for purposes of
> > about:memory?  How long do they end up sticking around?
> 
> I'd say it is likely that they end up in the system compartment (the sandbox
> is created with the system principal) but maybe Blake would know better.

I don't think they're ending up in "the" system compartment.  xpc_CreateSandboxObject passes a brand new pointer to xpc_CreateGlobalObject, so I don't think we can have a hit in the compartment hashtable.
Comment 6 User image Boris Zbarsky [:bz] (still a bit busy) 2011-07-08 12:04:17 PDT
Hence "for purposes of about:memory".  Each sandbox gets a new compartment, but the question is how about:memory tracks these.
Comment 7 User image Nicholas Nethercote [:njn] 2011-07-08 14:07:47 PDT
(In reply to comment #6)
> Hence "for purposes of about:memory".  Each sandbox gets a new compartment,
> but the question is how about:memory tracks these.

It just iterates through JSRuntime::compartments.  See http://mxr.mozilla.org/mozilla-central/source/js/src/xpconnect/src/xpcjsruntime.cpp#1519.

The arena values in comment 0 are surprising.  We have:

│   │  │   ├────574,640 B (00.46%) -- arena-unused
│   │  │   ├─────47,064 B (00.04%) -- arena-headers

and:

│  │   │  ├───9,306,464 B (04.47%) -- arena-unused
│  │   │  ├─────131,688 B (00.06%) -- arena-headers

"arena-headers" is proportional to the number of arenas in the compartment, and that's increased by 2.8x.  But the "arena-unused" space has increased by 16.2x.  So each arena is on average 5.8x less full.  Maybe Bugzilla Tweaks is causing memory to be churned at a much higher rate, ie. lots of short-lived objects?
Comment 8 User image Nicholas Nethercote [:njn] 2011-07-12 18:49:19 PDT
(In reply to comment #7)
> (In reply to comment #6)
> > Hence "for purposes of about:memory".  Each sandbox gets a new compartment,
> > but the question is how about:memory tracks these.
> 
> It just iterates through JSRuntime::compartments.

Oh, but if there are multiple compartments with the same name (and the name comes from principals->codebase) then about:memory will merge them.  I just filed bug 671159 to make them be reported separately.
Comment 9 User image Nicholas Nethercote [:njn] 2012-01-11 03:01:59 PST
I wonder if bug 672443 being fixed helps this bug.
Comment 10 User image Gabor Krizsanits [:krizsa :gabor] 2012-01-11 04:04:57 PST
Actually I think bug 687777 being fixed solved this bug already, since there were some serious memory problems with context menu items. I could not reproduce this issue with the latest bugzilla tweeks with the firefox aurora build.
Comment 11 User image Nicholas Nethercote [:njn] 2012-01-11 13:41:53 PST
Thanks for the info, Gabor!

Note You need to log in before you can comment on or make changes to this bug.