Fix up jemalloc stats reporting

RESOLVED FIXED in Firefox 47

Status

()

defect
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: njn, Assigned: njn)

Tracking

unspecified
mozilla47
Points:
---

Firefox Tracking Flags

(firefox47 fixed)

Details

Attachments

(1 attachment)

Assignee

Description

3 years ago
We have some oddities in our jemalloc stats reporting.
Assignee

Comment 1

3 years ago
We have some oddities in our jemalloc stats reporting.

- "heap-overhead-ratio" is a strange measurement: overhead / non-overhead,
  expressed as a percentage. And it omits "bin_unused", which appears to be an
  oversight.

- "heap-committed" also omits "bin_unused".

- There are some minor errors in memory report descriptions.

This patch fixes these and improves the heap reporting. It makes the following
reporting changes:

- "heap-allocated": Duplicated as "heap-committed/allocated". (We keep
  "heap-allocated" because that's a special value used in the computation of
  "heap-unclassified".)

- "heap-committed/overhead": Added; it's the same as the sum of the
  "explicit/heap-overhead/*" values. Together with "heap-committed/allocated"
  it shows clearly what fraction of the heap is overhead and what fraction is
  useful.

- "heap-committed": Removed; now implicit as the "heap-committed/" node.

- "heap-overhead-ratio":
  - Removed from memory reports; now shown as the percentage of the new
    "heap-committed/overhead" node.
  - Still available as a distinguished amount (because it's useful in
    isolation) but renamed to heapOverheadFraction, and the telemetry ID is
    renamed as MEMORY_HEAP_OVERHEAD_FRACTION.

- "heap-chunks": Removed; it's not that interesting, and can be manually
  computed as "heap-mapped" / "heap-chunksize" if necessary.
Attachment #8725106 - Flags: review?(erahm)
Assignee

Comment 2

3 years ago
Here is some example output to make the patch's effects clearer.

OLD

> 156.70 MB (100.0%) -- explicit
> ...
> └────2.54 MB (01.62%) -- heap-overhead
>      ├──1.39 MB (00.89%) ── bookkeeping
>      ├──0.65 MB (00.41%) ── page-cache
>      ├──0.50 MB (00.32%) ── bin-unused
>      └──0.00 MB (00.00%) ── waste
>
> 111.98 MB ── heap-allocated
>       117 ── heap-chunks
>   1.00 MB ── heap-chunksize
> 114.02 MB ── heap-committed
> 117.00 MB ── heap-mapped
>     1.82% ── heap-overhead-ratio

NEW

> 135.77 MB (100.0%) -- explicit
> ...
> ├───11.01 MB (08.11%) -- heap-overhead
> │   ├───6.72 MB (04.95%) ── bin-unused
> │   ├───2.92 MB (02.15%) ── page-cache
> │   ├───1.37 MB (01.01%) ── bookkeeping
> │   └───0.00 MB (00.00%) ── waste
>
> 96.86 MB (100.0%) -- heap-committed
> ├──85.85 MB (88.64%) ── allocated
> └──11.01 MB (11.36%) ── overhead
>
>    85.85 MB ── heap-allocated
>     1.00 MB ── heap-chunksize
>   118.00 MB ── heap-mapped
Comment on attachment 8725106 [details] [diff] [review]
Fix up jemalloc stats reporting

Review of attachment 8725106 [details] [diff] [review]:
-----------------------------------------------------------------

r=me with an update to explicit/heap-overhead/waste as well.

::: xpcom/base/nsMemoryReporterManager.cpp
@@ +1337,5 @@
> +"allocator is not intentionally keeping alive (i.e., not "
> +"'explicit/heap-overhead/{bookkeeping,page-cache,bin-unused}'). Although the "
> +"allocator will waste some space under any circumstances, a large value here "
> +"may indicate that the heap is highly fragmented, or that allocator is "
> +"performing poorly for some other reason.");

Can we switch to only reporting this if stats.waste != 0? Since adding bin-unused I've never see this non-zero (although that might change if we switch to jemalloc 4). Also I'm not sure the part about being highly fragmented is in any way correct, it's really just stuff we've failed to measure.
Attachment #8725106 - Flags: review?(erahm) → review+

Comment 5

3 years ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/90fae59aad6e
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla47
You need to log in before you can comment on or make changes to this bug.