1477662 - Runtime of collecting data from the memory reporter in Telemetry increased

Status: RESOLVED WONTFIX

(Core :: XPCOM, enhancement)

Description

[Jan-Erik Rediger [:janerik]]

We got an alert that the time it takes to run the memory reporter to collect data when sending a Telemetry ping changed. See [1].
There’s a sudden jump on Linux: https://mzl.la/2JMaVbm.
The relevant changes are in this range[2]. 

This includes bug 1475899, which includes the thread stack memory reporter.

Could this affect the time it takes to collect all memory reporter information?
If so, do we need to worry or are we fine with the increased time as we're also collecting more data?


[1]: http://alerts.telemetry.mozilla.org/index.html#/detectors/1/metrics/354/alerts/?from=2018-07-19&to=2018-07-19
[2]: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=5a8107262015714d2907a85abc24c847ad9b32d2&tochange=690cb3015db6645b335ac4835a50073cb6a3e23c

Comment 1

[Kris Maglione [:kmag]]

Seems likely, yeah. Bug 1476168 should fix (or at least improve) this. But even if it didn't, I'd probably accept 15ms of additional memory reporter overhead for the sake of accurate thread stack information.

Comment 2

[Kris Maglione [:kmag]]

Seems likely, yeah. Bug 1476168 should fix (or at least improve) this. But even if it didn't, I'd probably accept 15ms of additional memory reporter overhead for the sake of accurate thread stack information.

Comment 3

[Andrew McCreight [:mccr8]]

We could also disable it for the telemetry memory report if it becomes a huge problem.

Comment 4

[Kris Maglione [:kmag]]

Looking at this a bit more closely, it looks like this test doesn't even run the memory reporter, it just calls the residentUnique getter.

Half of the time spent in that getter is still just reading the smaps file. The rest is parsing it, and the new parsing is more comprehensive and therefore slightly slower than before.

The whole operation takes about 20ms when the file is about 20,000 lines long, which I think is pretty reasonable for something that never gets called from hot code. We could probably make it faster by writing a special-purpose lexer rather than using getline/sscanf/strtok, but it doesn't seem worth it.