1283587 - content process worker debugger window leaks compartments

Status: NEW

(DevTools :: Debugger, defect, P3)

Description

[Ben Kelly [:bkelly, not reviewing]]

STR:

1) Open fresh browser instance
2) Open a content window with a service worker (I used blog.wanderview.com)
3) Open about:debugging#workers
4) Start the service worker if necessary by refreshing the content tab (start button has a bug)
5) Click debug to open a worker debug window
6) Wait a few seconds for window to completely initialize (some things appear lazy loaded)
7) Close debug window
8) Repeat steps 4 to 7 many times
9) Open about:memory, minimize, and measure
10) Observe in the js-main-runtime-compartments section of the content process that there are numerous sandbox compartments leaked
11) Repeat steps 4 to 7
12) Minimize and measure in about:memory
13) Observe that the number of compartments has increased

In my about memory at the moment I see this:

124 (100.0%) -- js-main-runtime-compartments
├──120 (96.77%) -- system
│  ├───91 (73.39%) ++ (91 tiny)
│  ├────5 (04.03%) ── [System Principal], [anonymous sandbox] (from: resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/builtin-modules.js:183) [5]
│  ├────5 (04.03%) ── [System Principal], [anonymous sandbox] (from: resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/builtin-modules.js:209) [5]
│  ├────5 (04.03%) ── [System Principal], [anonymous sandbox] (from: resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/builtin-modules.js:220) [5]
│  ├────5 (04.03%) ── [System Principal], [anonymous sandbox] (from: resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/builtin-modules.js:227) [5]
│  ├────5 (04.03%) ── [System Principal], Addon-SDK (from: resource://gre/modules/commonjs/toolkit/loader.js:249) [5]
│  └────4 (03.23%) ── [System Principal], [anonymous sandbox] (from: resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/server/actors/child-process.j

Note that I tried applying the patch from bug 1281040, but it did not help.

Comment 1

[Ben Kelly [:bkelly, not reviewing]]

I tried looking at the CC logs, but I don't know where to start.  What is the equivalent c++ object for a compartment?  I normally look for nsGlobalObject when windows leak.

Comment 2

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

For sandboxes, SandboxPrivate.

Comment 3

[Ben Kelly [:bkelly, not reviewing]]

Hmm, that didn't give me too much to go on:

$ /c/devel/heapgraph/find_roots.py cc-edges.9868.1467299147.log 000001CA7E9E65C0
Parsing cc-edges.9868.1467299147.log. Done loading graph.

000001CA7E76C1A0 [JS Object (Sandbox)]
    --[js::GetObjectPrivate(obj)]--> 000001CA7E9E65C0 [SandboxPrivate]

    Root 000001CA7E76C1A0 is a marked GC object.


bkelly@kosh /c/devel/tmp/cclogs/addon-loader
$ /c/devel/heapgraph/find_roots.py gc-edges.9868.1467299147.log -bro 000001CA7E76C1A0
Parsing gc-edges.9868.1467299147.log. Done loading graph.

via persistent-Object :
000001CA7E76C1A0 [Sandbox 1ca7e9e65d8]

Comment 4

[Andrew McCreight [:mccr8]]

That means there is some PersistentRooted sitting around in C++ for this sandbox. Somebody who understands how the debugger roots things probably needs to figure this out.

Comment 5

[Ben Kelly [:bkelly, not reviewing]]

Eddy, any thoughts?

Comment 6

[Ben Kelly [:bkelly, not reviewing]]

Of course, I don't know if I have the right sandbox in the CC log...  there are a lot with rc=1.

Comment 7

[Eddy Bruel [:ejpbruel]]

(In reply to Ben Kelly [:bkelly] from comment #5)
> Eddy, any thoughts?

To my knowledge, in a worker, the debugger can create sandboxes, but the debuggee cannot. Moreover, these sandboxes can never be accidentally exposed to the debuggee (we have a security wrapper that ensures this). Consequently, if we are leaking a sandbox, we must be leaking it in the debugger server.

The debugger server is written in pure JS, so it's not immediately obvious to me how it could be leaking a sandbox. The debugger server does interact with the debugger API, so if we are leaking a root somewhere, that's where I would look for it.

Unfortunately, without more information about what we're leaking, I don't really know where to go from there. Is there some way we could get more information on the nature of the leak?

Comment 8

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

(In reply to Eddy Bruel [:ejpbruel] from comment #7)
> The debugger server is written in pure JS, so it's not immediately obvious
> to me how it could be leaking a sandbox.

If only it were that easy.

One thing you might try is taking logs before starting the debugger and afterwards.  Then you'll know which Sandboxes are presumably debugger-related.

Comment 9

[Ben Kelly [:bkelly, not reviewing]]

Nick, do you have any time to look at this?  Since you are already in devtools memory land...

Comment 10

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

It would be interesting to see if this is fixed after bug 1261869 merges into m-c. Unfortunately, I don't have time to dig into this at the moment, but maybe in a couple weeks.

Comment 11

[Eric Rahm [:erahm]]

bug 1261869 merged to central 6 days ago, Ben can you try to repro again?

Comment 12

[Ben Kelly [:bkelly, not reviewing]]

This still reproduces as of nightly 50.0a1 (2016-07-14).

Comment 13

[Ben Kelly [:bkelly, not reviewing]]

My nightly was lagging quite a lot tonight.  Looking in about:memory I found this in the content process:

898.04 MB (100.0%) -- explicit
├──472.93 MB (52.66%) -- js-non-window
│  ├──453.76 MB (50.53%) -- zones
│  │  ├──448.97 MB (49.99%) -- zone(0x113133000)
│  │  │  ├──305.00 MB (33.96%) ++ compartment([System Principal], Addon-SDK (from: resource://gre/modules/commonjs/toolkit/loader.js:249))

That's a huge amount of js-non-window.  Seems it might be related to this bug:

1,362 (100.0%) -- js-main-runtime-compartments
├──1,359 (99.78%) -- system
│  ├────296 (21.73%) ── [System Principal], [anonymous sandbox] (from: resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/builtin-modules.js:185) [296]
│  ├────296 (21.73%) ── [System Principal], [anonymous sandbox] (from: resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/builtin-modules.js:209) [296]
│  ├────296 (21.73%) ── [System Principal], [anonymous sandbox] (from: resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/builtin-modules.js:24) [296]
│  ├────296 (21.73%) ── [System Principal], Addon-SDK (from: resource://gre/modules/commonjs/toolkit/loader.js:249) [296]