1563755 - Add telemetry to track how effective our GC triggers are

Status: RESOLVED FIXED

(Core :: JavaScript: GC, task, P3)

Description

[Jon Coppeard (:jonco)]

There's a whole bunch of things that can trigger a GC but right now we have no data on how effective these are. GC can be expensive and we don't want to do if it's not going to free memory.

We should add telemetry to measure the proportion of the heap freed by a GC, with the aim of increasing this measure as we tune our triggers.

Comment 1

[Paul Bone [:pbone]]

(In reply to Jon Coppeard (:jonco) from comment #0)

We should add telemetry to measure the proportion of the heap freed by a GC, with the aim of increasing this measure as we tune our triggers.

Bug 1519298 is already filed to work on that. The trick is that due to incremental GC both allocations and deallocations can occur during a GC. To properly understand the effectiveness of a GC you need to measure both. I have some half-done patches for this, feel free to bump the priority on that other bug and I'll finish them.

Comment 2

[Jon Coppeard (:jonco)]

(In reply to Paul Bone [:pbone] from comment #1)

Bug 1519298 is already filed to work on that.

Ah, I didn't realise that was the same thing. I've written some patches for this already.

Comment 3

[Jon Coppeard (:jonco)]

Attachment: Bug 1563755 - Refactor GC telemetry collection r=pbone

Factor out methods to send GC telemetry at the end of a slice and at the end of a collection. Rename heap size fields to indicate that they are for total heap size in bytes as opposed to the size of collected zones.

Comment 4

[Jon Coppeard (:jonco)]

Attachment: Bug 1563755 - Add tenured heap survival rate telemetry r=pbone data-review=chutten?

This records the size of the heap at the start of collection and maintains a count of bytes freed during collection. From this we calcuate a 'survival rate' at the end of collection. The intention is that this is like the nursery's promotion rate so it's the fraction retained rather than the fraction freed.

Depends on D37451

Comment 5

[Paul Bone [:pbone]]

(In reply to Jon Coppeard (:jonco) from comment #2)

(In reply to Paul Bone [:pbone] from comment #1)

Bug 1519298 is already filed to work on that.

Ah, I didn't realise that was the same thing. I've written some patches for this already.

Not exactly, but the same counting code could be used. Oh well.

Comment 6

[Jon Coppeard (:jonco)]

Attachment: data_collection_request.txt

Requesting data collection review.

Comment 7

[Jon Coppeard (:jonco)]

Attachment: Bug 1563755 - Update record of initial heap size when we merge a zone into a zone that's being collected r=pbone?

Just one more change is necessary to make this work. We have to take account of the size of zones merged into collecting zones during an incremental GC.

Depends on D37452

Comment 8

[Chris H-C :chutten]

Comment on attachment 9077087 [details]
data_collection_request.txt

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is Telemetry so is documented in its definitions file [Histograms.json](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/telemetry/Histograms.json) and the [Probe Dictionary](https://telemetry.mozilla.org/probe-dictionary/).

    Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, Jon Coppeard is responsible.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 1, Technical.

    Is the data collection request for default-on or default-off?

Default on for all channels.

    Does the instrumentation include the addition of any new identifiers?

No.

    Is the data collection covered by the existing Firefox privacy notice?

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

No. This collection is permanent.

---
Result: datareview+

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/7c333dee4e3e
Refactor GC telemetry collection r=pbone
https://hg.mozilla.org/integration/mozilla-inbound/rev/55605e3b82ef
Add tenured heap survival rate telemetry r=pbone data-review=chutten

Comment 10

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/7c333dee4e3e
https://hg.mozilla.org/mozilla-central/rev/55605e3b82ef