1878806 - Move heap overheads out of the explicit/ tree

Status: RESOLVED FIXED

(Toolkit :: about:memory, task, P1)

Description

[Paul Bone [:pbone]]

Currently heap overheads are in the explicit/ tree. Recently while working on Bug 1857841 this was surprising because that bug adds non-committed overheads and explicit memory grew as a result. I'd like to move the heap related overheads into their own tree in the memory reporter. Note that this will undo Bug 898558 but I think this will be less surprising and maybe our memory report comparison code has improved since then because I haven't found it hard to compare reports. I'll test that feature though!

I have a patch that adds a tree such as:

357.00 MB (100.0%) -- heap
├──265.85 MB (74.47%) -- committed
│  ├──257.55 MB (72.14%) ── allocated
│  ├────4.50 MB (01.26%) -- (2 tiny)
│  │    ├──3.32 MB (00.93%) ── bookkeeping
│  │    └──1.18 MB (00.33%) ── page-cache-dirty
│  └────3.80 MB (01.07%) ++ bin-unused
└───91.15 MB (25.53%) -- decommitted
    ├──64.80 MB (18.15%) ── unmapped
    └──26.35 MB (07.38%) ── clean

Which gives a nice breakdown of mozjemalloc's heap space. It's easy to see "Oh, bookkeeping accounts for about 1% of heap overheads". Note that since this depends on Bug 1857841 there is a new node in this tree heap/decommitted/clean represents memory that has never been touched or has been madvised it is part of the page cache but is distinct from page-cache-dirty.

Comment 1

[Paul Bone [:pbone]]

Attachment: Bug 1878806 - pt 1. Move mozjemalloc stats into their own tree r=mccr8

Comment 2

[Paul Bone [:pbone]]

Attachment: Bug 1878806 - pt 2. Add about:memory entries for fresh and madvised pages r=mccr8,glandium

Comment 3

[Paul Bone [:pbone]]

Attachment: WIP: Bug 1878806 - pt 3. Correct a comment

Comment 4

[Paul Bone [:pbone]]

Attachment: WIP: Bug 1878806 - pt 4. Count "double purged" memory with a seperate entry

Comment 5

[Paul Bone [:pbone]]

Attachment: WIP: Bug 1878806 - pt 5. Fold hard_purge_chunk into its ownly caller

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/adb7c40dad27
pt 1. Move mozjemalloc stats into their own tree r=mccr8
https://hg.mozilla.org/integration/autoland/rev/23f554ffaca7
pt 2. Add about:memory entries for fresh and madvised pages r=mccr8,glandium

Comment 7

[pstanciu]

https://hg.mozilla.org/mozilla-central/rev/adb7c40dad27
https://hg.mozilla.org/mozilla-central/rev/23f554ffaca7

Comment 8

[Florin Bilt]

(In reply to Pulsebot from comment #6)

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/adb7c40dad27
pt 1. Move mozjemalloc stats into their own tree r=mccr8
https://hg.mozilla.org/integration/autoland/rev/23f554ffaca7
pt 2. Add about:memory entries for fresh and madvised pages r=mccr8,glandium

== Change summary for alert #42063 (as of Fri, 29 Mar 2024 12:13:11 GMT) ==

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)
31%	Base Content Explicit	linux1804-64-shippable-qr	fission	9,701,733.33 -> 6,664,374.33
31%	Base Content Explicit	linux1804-64-shippable-qr	fission	9,678,949.33 -> 6,661,954.67
29%	Base Content Explicit	macosx1015-64-shippable-qr	fission	10,110,138.67 -> 7,227,219.33
28%	Base Content Explicit	windows11-64-2009-shippable-qr	fission	10,497,917.71 -> 7,527,744.67
19%	Explicit Memory	windows11-64-2009-shippable-qr	fission tp6	590,258,022.19 -> 479,549,445.35
18%	Explicit Memory	macosx1015-64-shippable-qr	fission tp6	650,307,325.78 -> 534,280,579.98

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=42063