1252375 - Fix up jemalloc stats reporting

Status: RESOLVED FIXED

(Core :: XPCOM, defect)

Description

[Nicholas Nethercote [inactive]]

We have some oddities in our jemalloc stats reporting.

Comment 1

[Nicholas Nethercote [inactive]]

Attachment: Fix up jemalloc stats reporting

We have some oddities in our jemalloc stats reporting.

- "heap-overhead-ratio" is a strange measurement: overhead / non-overhead,
  expressed as a percentage. And it omits "bin_unused", which appears to be an
  oversight.

- "heap-committed" also omits "bin_unused".

- There are some minor errors in memory report descriptions.

This patch fixes these and improves the heap reporting. It makes the following
reporting changes:

- "heap-allocated": Duplicated as "heap-committed/allocated". (We keep
  "heap-allocated" because that's a special value used in the computation of
  "heap-unclassified".)

- "heap-committed/overhead": Added; it's the same as the sum of the
  "explicit/heap-overhead/*" values. Together with "heap-committed/allocated"
  it shows clearly what fraction of the heap is overhead and what fraction is
  useful.

- "heap-committed": Removed; now implicit as the "heap-committed/" node.

- "heap-overhead-ratio":
  - Removed from memory reports; now shown as the percentage of the new
    "heap-committed/overhead" node.
  - Still available as a distinguished amount (because it's useful in
    isolation) but renamed to heapOverheadFraction, and the telemetry ID is
    renamed as MEMORY_HEAP_OVERHEAD_FRACTION.

- "heap-chunks": Removed; it's not that interesting, and can be manually
  computed as "heap-mapped" / "heap-chunksize" if necessary.

Comment 2

[Nicholas Nethercote [inactive]]

Here is some example output to make the patch's effects clearer.

OLD

> 156.70 MB (100.0%) -- explicit
> ...
> └────2.54 MB (01.62%) -- heap-overhead
>      ├──1.39 MB (00.89%) ── bookkeeping
>      ├──0.65 MB (00.41%) ── page-cache
>      ├──0.50 MB (00.32%) ── bin-unused
>      └──0.00 MB (00.00%) ── waste
>
> 111.98 MB ── heap-allocated
>       117 ── heap-chunks
>   1.00 MB ── heap-chunksize
> 114.02 MB ── heap-committed
> 117.00 MB ── heap-mapped
>     1.82% ── heap-overhead-ratio

NEW

> 135.77 MB (100.0%) -- explicit
> ...
> ├───11.01 MB (08.11%) -- heap-overhead
> │   ├───6.72 MB (04.95%) ── bin-unused
> │   ├───2.92 MB (02.15%) ── page-cache
> │   ├───1.37 MB (01.01%) ── bookkeeping
> │   └───0.00 MB (00.00%) ── waste
>
> 96.86 MB (100.0%) -- heap-committed
> ├──85.85 MB (88.64%) ── allocated
> └──11.01 MB (11.36%) ── overhead
>
>    85.85 MB ── heap-allocated
>     1.00 MB ── heap-chunksize
>   118.00 MB ── heap-mapped

Comment 3

[Eric Rahm [:erahm]]

Comment on attachment 8725106 [details] [diff] [review]
Fix up jemalloc stats reporting

Review of attachment 8725106 [details] [diff] [review]:
-----------------------------------------------------------------

r=me with an update to explicit/heap-overhead/waste as well.

::: xpcom/base/nsMemoryReporterManager.cpp
@@ +1337,5 @@
> +"allocator is not intentionally keeping alive (i.e., not "
> +"'explicit/heap-overhead/{bookkeeping,page-cache,bin-unused}'). Although the "
> +"allocator will waste some space under any circumstances, a large value here "
> +"may indicate that the heap is highly fragmented, or that allocator is "
> +"performing poorly for some other reason.");

Can we switch to only reporting this if stats.waste != 0? Since adding bin-unused I've never see this non-zero (although that might change if we switch to jemalloc 4). Also I'm not sure the part about being highly fragmented is in any way correct, it's really just stuff we've failed to measure.

Comment 4

[Nicholas Nethercote [inactive]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/90fae59aad6e801ca8c4c7226be35b7116c34d60
Bug 1252375 - Fix up jemalloc stats reporting. r=erahm.

Comment 5

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/90fae59aad6e