690458 - Rebucket telemetry memory statistics

Status: RESOLVED FIXED

(Toolkit :: Telemetry, defect)

Description

[Justin Lebar (not reading bugmail)]

Memory statistics reported by telemetry are too coarse-grained.  For instance, RSS is bucketed in the range [32mb, 1gb] with 10 exponential buckets, so one of the buckets covers roughly the range 530mb to 1gb (*).  This is not particularly useful.

If you have an LDAP account, you can, in theory, see the stats for yourself at [1].  Unfortunately the dashboard is down right now (bug 687117).

Anyway, the solution is rebucketing, which is simple enough.  This will invalidate all of the statistics we'd collected up to this point, but they're not really useful anyway.

(*) I'd look up the actual range, but I can't, because the dashboard is down...

[1] https://metrics.mozilla.com/pentaho/content/pentaho-cdf-dd/Render?solution=metrics&path=telemetry/&file=TelemetryHistogram.wcdf

Comment 1

[Justin Lebar (not reading bugmail)]

Attachment: Patch v1

Comment 2

[Justin Lebar (not reading bugmail)]

Comment on attachment 563498 [details] [diff] [review]
Patch v1

Do you want to review this one, Taras, or do you want to pass it off to someone working on MemShrink?

Comment 3

[(dormant account)]

Comment on attachment 563498 [details] [diff] [review]
Patch v1

Memshrinkers should do this one

Comment 4

[Justin Lebar (not reading bugmail)]

Aha, the telemetry dashboard is back.

I'm not sure exactly how exponential histogram bucketing works.  The RSS bucketing is in fact better than I expected; the top bucket is actually ~680mb, not 530mb.  But the top bucket for explicit covers 440mb - 1gb.

The coarseness of these measurements makes it difficult for us to determine whether changes we've made improve real-world memory usage.

Comment 5

[Justin Lebar (not reading bugmail)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/d21d678255d8

Comment 6

[Matt Brubeck (:mbrubeck)]

https://hg.mozilla.org/mozilla-central/rev/d21d678255d8