1278731 - devtools memory panel uses a lot of heap-uncategorized memory

Status: NEW

(DevTools :: Memory, defect, P2)

Description

[Ben Kelly [:bkelly, not reviewing]]

STR:

0) Open about:memory, minimize, measure
1) Open a complex site like gmail
2) Open devtools memory panel
3) Take a snapshot with allocation stacks
4) Note the size of the snapshot (89MB in my case)
5) Open about:memory, minimize, and measure
6) Note that heap-unclassified has jumped by about 3x the size of the snapshot (270MB in my case)

I did two snapshots on gmail and I ended up with this:

1,587.11 MB (100.0%) -- explicit
├────662.84 MB (41.76%) ── heap-unclassified

I know its a lot of data, but this seems excessive.

Comment 1

[Jim Blandy :jimb]

The snapshot itself is a serialized representation of the heap. We do build a hash table around it, I think one per node, but that shouldn't be 3x the size of the serialized data.

Nick, is this within your expectations?

Comment 2

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

It would be nice to tell about:memory about the snapshot data -- is there an easy way to hook into about:memory's reporting infrastructure via the webidl interface implementation?

(In reply to Jim Blandy :jimb from comment #1)
> The snapshot itself is a serialized representation of the heap. We do build
> a hash table around it, I think one per node, but that shouldn't be 3x the
> size of the serialized data.
> 
> Nick, is this within your expectations?

In addition to the cost of the hashtable, when the snapshot is serialized to disk it is also gzipped, so when we deserialize it, we also uncompress it.

Comment 3

[Eric Rahm [:erahm]]

Nick, is nsIMemoryReporterManager [1] what you're looking for?

[1] https://dxr.mozilla.org/mozilla-central/rev/939ecc4e9d055c263633cbe276bfb634a68fe4c5/xpcom/base/nsIMemoryReporter.idl#217-223

Comment 4

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

So I need to implement subclass of nsIMemoryReporter for heap snapshots (one for every heap snapshot or just once for all heap snapshots?), register it via RegisterStrongMemoryReporter (when?), and unregister is some other time (when?)

I searched mdn, but couldn't find a small tutorial / example of using this stuff. That would be a great thing to have! If such a page doesn't exist, is there a good existing reporter to crib from?

Comment 5

[Eric Rahm [:erahm]]

(In reply to Nick Fitzgerald [:fitzgen] [⏰PDT; UTC-7] from comment #4)
> So I need to implement subclass of nsIMemoryReporter for heap snapshots (one
> for every heap snapshot or just once for all heap snapshots?), register it
> via RegisterStrongMemoryReporter (when?), and unregister is some other time
> (when?)
> 
> I searched mdn, but couldn't find a small tutorial / example of using this
> stuff. That would be a great thing to have! If such a page doesn't exist, is
> there a good existing reporter to crib from?

I imagine the unclassified memory is on the C++ side, so I'm not sure if implementing in JS makes sense.

When to register:
- Some reporters get registered at startup (so think a thing that manages all heap snapshots, they get registered when created, removed when destroyed) [1], since this is a strong ref it's okay to let it live until the nsMemoryReporterManager dies
- Some reporters register a new instance for each thing they report on (so think a new reporter for each new snapshot) [2] and unregister when it's destroyed

There's also an mdn page [3] with more details. We can update that if you think it could use some expansion.

[1] https://dxr.mozilla.org/mozilla-central/rev/fdcee57b4e4f66a82831ab01e61500da98a858e8/dom/base/File.cpp#1147
[2] https://dxr.mozilla.org/mozilla-central/rev/fdcee57b4e4f66a82831ab01e61500da98a858e8/dom/media/webaudio/AudioBuffer.cpp#98
[3] https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Memory_reporting