1253984 - [e10s] 25% – 45% regression on tp5o Private Bytes

Status: RESOLVED INVALID

(Core :: General, defect, P4)

Description

[Chris Peterson [:cpeterson]]

Linux = 45% regression
Windows 7 = 41% regression
Windows XP = 25% regression

https://treeherder.mozilla.org/perf.html#/e10s?filter=Private%20Bytes

Comment 1

[Eric Rahm [:erahm]]

Can we get clarification on what this test measures and how it goes about it? A link to the relevant code would be useful.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

for windows, Private Bytes is collected from the OS:
https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/cmanager_win32.py

this uses the query to find the pdh counter (https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/cmanager_win32.py#35):
\\process(firefox)\\Private Bytes


for linux, Private Bytes is collected from the OS at /proc/<pid>/maps:
https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/cmanager_linux.py#75

Comment 3

[Eric Rahm [:erahm]]

TL;DR - These numbers are misleading and don't measure the thing we want to measure (e10s or not).

(In reply to Joel Maher (:jmaher) from comment #2)
> for windows, Private Bytes is collected from the OS:
> https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/
> cmanager_win32.py
> 
> this uses the query to find the pdh counter
> (https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/
> cmanager_win32.py#35):
> \\process(firefox)\\Private Bytes 

I'm not super familiar with the performance counter interface, but my understanding is that "Private Bytes" is really how much page file backing a process potentially has. This isn't a terribly interesting number (the name is misleading). What we're really looking for is the "Private Working Set" which is not available as a performance counter, we have to calculate it in browser for our resident unique measurement [1].

> for linux, Private Bytes is collected from the OS at /proc/<pid>/maps:
> https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/
> cmanager_linux.py#75

This is measuring private writable virtual memory, which seems incorrect. In browser we use /proc/<pid>/smaps [2].

[1] https://dxr.mozilla.org/mozilla-central/rev/5e14887312d4523ab59c3f6c6c94a679cf42b496/xpcom/base/nsMemoryReporterManager.cpp#606-651
[2] https://dxr.mozilla.org/mozilla-central/rev/5e14887312d4523ab59c3f6c6c94a679cf42b496/xpcom/base/nsMemoryReporterManager.cpp#75-127

Comment 4

[Chris Peterson [:cpeterson]]

Eric is reviewing this test.

Comment 5

[Chris Peterson [:cpeterson]]

Joel and Eric, what is the next action to understand this tp5o Private Bytes test? Should we resolve this bug (and test) as INVALID and file a new bug about creating a test for Private Working Set?

Comment 6

[Eric Rahm [:erahm]]

I'd be okay with splitting out a bug to fix the test. From previous discussions there are two simple solutions:
  1) Within the process: |nsIMemoryReporterManager.residentUnique| [1]
  2) Outside the process: |Process.memory_full_info().uss| value from psutil [2]

For 1) on e10s we'd need to add some sort of messaging to the child process similar to what telemetry does [3].

For 2) we'd need to update the in-tree version of psutil [4]. Unfortunately the API is rather unstable so this might require some integration fixes.

As to whether a 50% increase is expected for private bytes (so this is just sum(USS)), that actually seems okay to me. I can look at some historical data to confirm my expectations.

[1] https://dxr.mozilla.org/mozilla-central/rev/21bf1af375c1fa8565ae3bb2e89bd1a0809363d4/xpcom/base/nsIMemoryReporter.idl#381
[2] http://pythonhosted.org/psutil/#psutil.Process.memory_full_info
[3] https://hg.mozilla.org/releases/mozilla-beta/rev/2a04abb3101e
[4] https://dxr.mozilla.org/mozilla-central/source/python/psutil

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

lets spin off this into a new bug.  I like to measure outside of the process as much as possible, but iirc psutil and nsiMemoryReporterManager use the same technique- so going the in process route would be nice.

I would vote for solving the new more accurate private bytes collection in bug 1250169.

Comment 8

[Chris Peterson [:cpeterson]]

Sounds good. I'll close this bug. Eric says the private bytes test for bug 1250169 does not need to block e10s release as long as MEMORY_TOTAL telemetry data continues to look good.