1690905 - Unreported heap-unclassified memory for "nsNodeInfoManager::Allocate(unsigned long)"

Status: RESOLVED FIXED

(Core :: DOM: Core & HTML, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Running a recent Firefox Nightly build with DMD enabled shows a couple of blocks for the following unreported heap-unclassified memory:

Unreported {
  18,857 blocks in heap block record 3 of 6,548
  2,413,696 bytes (2,413,696 requested / 0 slop)
  Individual block sizes: 128 x 18,857
  1.63% of the heap (36.16% cumulative)
  2.90% of unreported (64.41% cumulative)
  Allocated at {
    #01: nsNodeInfoManager::Allocate(unsigned long) (/Applications/FirefoxNightly.app/Contents/MacOS/XUL + 0xea2ad1)
    #02: nsHtml5TreeBuilder::createElement(int, nsAtom*, nsHtml5HtmlAttributes*, void*, nsHtml5ContentCreatorFunction) (/Applications/FirefoxNightly.app/Contents/MacOS/XUL + 0xbc50f3)
    #03: nsHtml5TreeBuilder::startTag(nsHtml5ElementName*, nsHtml5HtmlAttributes*, bool) (/Applications/FirefoxNightly.app/Contents/MacOS/XUL + 0xbb3abf)
    #04: int nsHtml5Tokenizer::stateLoop<nsHtml5SilentPolicy>(int, char16_t, int, char16_t*, bool, int, int) (/Applications/FirefoxNightly.app/Contents/MacOS/XUL + 0xbbc0e9)
    #05: nsHtml5Tokenizer::tokenizeBuffer(nsHtml5UTF16Buffer*) (/Applications/FirefoxNightly.app/Contents/MacOS/XUL + 0xbac7a4)
    #06: nsHtml5StringParser::Tokenize(nsTSubstring<char16_t> const&, mozilla::dom::Document*, bool) (/Applications/FirefoxNightly.app/Contents/MacOS/XUL + 0xbae216)
    #07: nsContentUtils::ParseDocumentHTML(nsTSubstring<char16_t> const&, mozilla::dom::Document*, bool) (/Applications/FirefoxNightly.app/Contents/MacOS/XUL + 0x3168cdd)
    #08: mozilla::dom::DOMParser::ParseFromString(nsTSubstring<char16_t> const&, mozilla::dom::SupportedType, mozilla::ErrorResult&) (/Applications/FirefoxNightly.app/Contents/MacOS/XUL + 0xddbc44)
    #09: mozilla::dom::DOMParser_Binding::parseFromString(JSContext*, JS::Handle<JSObject*>, void*, JSJitMethodCallArgs const&) (/Applications/FirefoxNightly.app/Contents/MacOS/XUL + 0x3930950)
    #10: bool mozilla::dom::binding_detail::GenericMethod<mozilla::dom::binding_detail::NormalThisPolicy, mozilla::dom::binding_detail::ThrowExceptions>(JSContext*, unsigned int, JS::Value*) (/Applications/FirefoxNightly.app/Contents/MacOS/XUL + 0x10051ff)
    #11: ??? (???:???)
    #12: ??? (???:???)
    #13: ??? (???:???)
    #14: ??? (???:???)
    #15: js::jit::MaybeEnterJit(JSContext*, js::RunState&) (/Applications/FirefoxNightly.app/Contents/MacOS/XUL + 0x1d97b0c)
    #16: js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct, js::CallReason) (/Applications/FirefoxNightly.app/Contents/MacOS/XUL + 0x17d51b2)
    #17: js::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, js::AnyInvokeArgs const&, JS::MutableHandle<JS::Value>, js::CallReason) (/Applications/FirefoxNightly.app/Contents/MacOS/XUL + 0x17d7632)
    #18: js::CallSelfHostedFunction(JSContext*, JS::Handle<js::PropertyName*>, JS::Handle<JS::Value>, js::AnyInvokeArgs const&, JS::MutableHandle<JS::Value>) (/Applications/FirefoxNightly.app/Contents/MacOS/XUL + 0x1954642)
    #19: AsyncFunctionResume(JSContext*, JS::Handle<js::AsyncFunctionGeneratorObject*>, ResumeKind, JS::Handle<JS::Value>) (.llvm.6009879460456597685) (/Applications/FirefoxNightly.app/Contents/MacOS/XUL + 0x1855d0c)
    #20: PromiseReactionJob(JSContext*, unsigned int, JS::Value*) (/Applications/FirefoxNightly.app/Contents/MacOS/XUL + 0x189c2d4)
    #21: js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct, js::CallReason) (/Applications/FirefoxNightly.app/Contents/MacOS/XUL + 0x17d58b6)
    #22: JS::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, JS::HandleValueArray const&, JS::MutableHandle<JS::Value>) (/Applications/FirefoxNightly.app/Contents/MacOS/XUL + 0x19b51dd)
    #23: mozilla::dom::PromiseJobCallback::Call(mozilla::dom::BindingCallContext&, JS::Handle<JS::Value>, mozilla::ErrorResult&) (/Applications/FirefoxNightly.app/Contents/MacOS/XUL + 0xf14cfd)
  }
}

Comment 1

[Henri Sivonen (:hsivonen)]

Those allocations are supposed to be made with a DOMArena. smaug, how would those become unclassified?

Comment 2

[Olli Pettay [:smaug][[e-mail]]]

That is surprising. We do have still the same memory reporting for nodes as before and I see those in about:memory.
sefeng, does this ring any bells?

Comment 3

[Sean Feng [:sefeng]]

It doesn't ring any bells, part of the reason might be I don't know much about the memory reporter.

This looks like it was parsing an HTML document from a string by using DOMParser::ParseFromString, I am not sure who should be responsible to report the memory usage by that.

Is this similar to https://bugzilla.mozilla.org/show_bug.cgi?id=790695 which has been existing for years?

I can investigate if I know where to start..

Comment 4

[Olli Pettay [:smaug][[e-mail]]]

But the allocation is from nsNodeInfoManager::Allocate.
And yes, this is a dup of bug 790695.
https://bugzilla.mozilla.org/show_bug.cgi?id=790695#c6 is probably the issue here.

Do we need to register data documents somewhere so that they can be reported as part of about:memory?
Perhaps inner window could have a list of data documents.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Sean, was the answer from Olli enough to maybe get you started?

Comment 6

[Sean Feng [:sefeng]]

Yeah, I think so.

Andrew, I think heap-unclassifed is calculated by total heap memory - heap reported, so all I need to do is just reporting them, as something like https://searchfox.org/mozilla-central/rev/a1e81094c7c6dc5e39143caeb39cc46f36980d3e/dom/base/nsWindowSizes.h#136, and DMD is smart enough to tell these heap usages are reported? The last part just feels magical to me, how DMD tell which usages are reported?

Comment 7

[Andrew McCreight [:mccr8]]

(In reply to Sean Feng [:sefeng] from comment #6)

Andrew, I think heap-unclassifed is calculated by total heap memory - heap reported, so all I need to do is just reporting them, as something like https://searchfox.org/mozilla-central/rev/a1e81094c7c6dc5e39143caeb39cc46f36980d3e/dom/base/nsWindowSizes.h#136, and DMD is smart enough to tell these heap usages are reported? The last part just feels magical to me, how DMD tell which usages are reported?

When you gather a memory report while DMD is active, it substitutes a special MallocSizeOf that records the address of the block that was passed in, and the stack it was called from.

Comment 8

[Sean Feng [:sefeng]]

1,177.95 MB (100.0%) -- explicit
├────586.72 MB (49.81%) ── heap-unclassified
├────139.70 MB (11.86%) -- window-objects
│    ├───17.20 MB (01.46%) -- top(chrome://browser/content/browser.xhtml, id=9)/active
│    │   ├──14.34 MB (01.22%) -- window(chrome://browser/content/browser.xhtml)
│    │   │  ├───7.17 MB (00.61%) ++ layout
│    │   │  ├───3.74 MB (00.32%) ++ dom
│    │   │  ├───3.41 MB (00.29%) ++ js-realm([System Principal], about:blank)
│    │   │  ├───xxx MB (xx.xx%) ++ documents-parsed-from-string
│    │   │  │  ├───xxx MB (xx.xx%) ++ dom
│    │   │  │  │  ├──xxx MB (xx.xx%) ── element-nodes
│    │   │  │  │  ├──xxx MB (xx.xx%) ── orphan-nodes
│    │   │  │  │  ├──xxx MB (xx.xx%) ── other

Shall we make it something like this? To have a new documents-parsed-from-string sections under window, and having a subtree of dom nodes in it?

What do you think Olli?

Comment 9

[Olli Pettay [:smaug][[e-mail]]]

"documents-parsed-from-string" is weird. data documents in general aren't parsed from a string. One can create them in many ways and update like any other document.
loaded-as-data-documents would be closer to the terminology used in Gecko.

Comment 10

[Sean Feng [:sefeng]]

Attachment: Bug 1690905 - Factors DOM memory sizes out to its own struct r=mccr8

This patch factors the DOM related sizes in nsWindowSizes to its own
struct, such that callers can easily acess DOM memory sizes.

Comment 11

[Sean Feng [:sefeng]]

Attachment: Bug 1690905 - Report the DOM portion of memory usage for data documents in about:memory r=mccr8,smaug

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1b9ba76a839b
Factors DOM memory sizes out to its own struct r=mccr8
https://hg.mozilla.org/integration/autoland/rev/dc17f5154e0a
Report the DOM portion of memory usage for data documents in about:memory r=smaug

Comment 13

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/1b9ba76a839b
https://hg.mozilla.org/mozilla-central/rev/dc17f5154e0a