Closed Bug 1563755 Opened 5 months ago Closed 5 months ago

Add telemetry to track how effective our GC triggers are

Categories

(Core :: JavaScript: GC, task, P3)

task

Tracking

()

RESOLVED FIXED
mozilla70
Tracking Status
firefox69 --- wontfix
firefox70 --- fixed

People

(Reporter: jonco, Assigned: jonco)

References

(Blocks 1 open bug)

Details

Attachments

(3 files, 1 obsolete file)

There's a whole bunch of things that can trigger a GC but right now we have no data on how effective these are. GC can be expensive and we don't want to do if it's not going to free memory.

We should add telemetry to measure the proportion of the heap freed by a GC, with the aim of increasing this measure as we tune our triggers.

(In reply to Jon Coppeard (:jonco) from comment #0)

We should add telemetry to measure the proportion of the heap freed by a GC, with the aim of increasing this measure as we tune our triggers.

Bug 1519298 is already filed to work on that. The trick is that due to incremental GC both allocations and deallocations can occur during a GC. To properly understand the effectiveness of a GC you need to measure both. I have some half-done patches for this, feel free to bump the priority on that other bug and I'll finish them.

Depends on: 1519298

(In reply to Paul Bone [:pbone] from comment #1)

Bug 1519298 is already filed to work on that.

Ah, I didn't realise that was the same thing. I've written some patches for this already.

Factor out methods to send GC telemetry at the end of a slice and at the end of a collection. Rename heap size fields to indicate that they are for total heap size in bytes as opposed to the size of collected zones.

This records the size of the heap at the start of collection and maintains a count of bytes freed during collection. From this we calcuate a 'survival rate' at the end of collection. The intention is that this is like the nursery's promotion rate so it's the fraction retained rather than the fraction freed.

Depends on D37451

(In reply to Jon Coppeard (:jonco) from comment #2)

(In reply to Paul Bone [:pbone] from comment #1)

Bug 1519298 is already filed to work on that.

Ah, I didn't realise that was the same thing. I've written some patches for this already.

Not exactly, but the same counting code could be used. Oh well.

Status: NEW → ASSIGNED

Requesting data collection review.

Attachment #9077087 - Flags: data-review?(chutten)
Blocks: GCScheduling
No longer depends on: 1519298
Attachment #9076864 - Attachment description: Bug 1563755 - Refactor GC telemetry collection r=pbone? → Bug 1563755 - Refactor GC telemetry collection r=pbone
Attachment #9076865 - Attachment description: Bug 1563755 - Add tenured heap survival rate telemetry r=pbone? → Bug 1563755 - Add tenured heap survival rate telemetry r=pbone data-review=chutten?

Just one more change is necessary to make this work. We have to take account of the size of zones merged into collecting zones during an incremental GC.

Depends on D37452

Comment on attachment 9077087 [details]
data_collection_request.txt

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is Telemetry so is documented in its definitions file [Histograms.json](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/telemetry/Histograms.json) and the [Probe Dictionary](https://telemetry.mozilla.org/probe-dictionary/).

    Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, Jon Coppeard is responsible.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 1, Technical.

    Is the data collection request for default-on or default-off?

Default on for all channels.

    Does the instrumentation include the addition of any new identifiers?

No.

    Is the data collection covered by the existing Firefox privacy notice?

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

No. This collection is permanent.

---
Result: datareview+
Attachment #9077087 - Flags: data-review?(chutten) → data-review+
Attachment #9077430 - Attachment is obsolete: true
Pushed by jcoppeard@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/7c333dee4e3e
Refactor GC telemetry collection r=pbone
https://hg.mozilla.org/integration/mozilla-inbound/rev/55605e3b82ef
Add tenured heap survival rate telemetry r=pbone data-review=chutten
Status: ASSIGNED → RESOLVED
Closed: 5 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla70
You need to log in before you can comment on or make changes to this bug.