1142357 - Improve memory tracking

Status: RESOLVED WONTFIX

(Testing :: Talos, defect)

Description

[Mike Hommey [:glandium]]

I'm not sure exactly at what point memory (RSS, private bytes, others?) is being measured during tp5, but observations from bug 1141079 comment 18 make me think we're dangerously missing some things.

For example, if for some (buggy) reason image decoding allocates 1GB, uses it, and then frees it after the image has been displayed, if we measure memory usage after the image has been displayed, we never know that our peak memory usage has gone wild.

But peak memory usage may or may not be enough to track. I think ideally we should look at the peak memory usage over some sub intervals of a tp5 run.

Note that this may or may not overlap with AWSY, but AWSY is linux only, sadly.

Comment 1

[Nicholas Nethercote [inactive]]

This bug's title is too general.

Bug 842800 is about measuring peak memory usage. It's hard to do in a cheap way.

But if you just care about process peak RSS on Linux, it's easy: you can read it from the "VmHWM" (for "high-water mark") line in /proc/<pid>/status.

Of course, if we add code to read that and subsequently find that we have unexpected peaks, then we won't have much insight into what's causing them. But I guess we can cross that bridge when the time comes.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

we measure rss/xres/etc. after each page is done loading and recording the time we measure RSS from inside the browser.  The other metrics are collected in a separate thread every 20ms.

should we change the linux stuff to include peakRSS (VmHWM) or replace the existing stuff?

Comment 3

[Nicholas Nethercote [inactive]]

Not sure if it's useful, but peak RSS can now be gotten within the browser via nsIMemoryReporterManager.residentPeak.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

If we pull from the nsIMemoryReporterManager.residentPeak api, we could get this after each page load.

Lately I have been looking at the counters and what we do is:
1) collect a bunch of samples
2) take the average of all the samples and plot it on the graph

so this pulls into question how useful these samples are and if we care about more than just the average.  Ideally we will switch to measuring counters that we find actionable (i.e. peakRSS) and remove the need for the other counters.

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

I think we need to do a few things here, right now we have main_rss concerns (bug 1316575) and private byte collection concerns (bug 1279396).

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

we have switched to relying upon AWSY for memory tracking instead of Talos.

Comment 7

[Mike Hommey [:glandium]]

Are we tracking peak memory in AWSY?

Comment 8

[Eric Rahm [:erahm]]

(In reply to Mike Hommey [:glandium] from comment #7)
> Are we tracking peak memory in AWSY?

No, but we can for OSX and Linux. I could probably add Windows as well.