Closed Bug 1253984 Opened 7 years ago Closed 7 years ago

[e10s] 25% – 45% regression on tp5o Private Bytes


(Core :: General, defect, P4)




Tracking Status
e10s + ---
firefox44 --- wontfix
firefox45 --- wontfix
firefox46 --- ?
firefox47 --- affected
firefox48 --- affected


(Reporter: cpeterson, Assigned: erahm)


(Blocks 1 open bug)


Linux = 45% regression
Windows 7 = 41% regression
Windows XP = 25% regression
Priority: -- → P4
Can we get clarification on what this test measures and how it goes about it? A link to the relevant code would be useful.
Flags: needinfo?(jmaher)
for windows, Private Bytes is collected from the OS:

this uses the query to find the pdh counter (
\\process(firefox)\\Private Bytes

for linux, Private Bytes is collected from the OS at /proc/<pid>/maps:
Flags: needinfo?(jmaher)
TL;DR - These numbers are misleading and don't measure the thing we want to measure (e10s or not).

(In reply to Joel Maher (:jmaher) from comment #2)
> for windows, Private Bytes is collected from the OS:
> this uses the query to find the pdh counter
> (
> \\process(firefox)\\Private Bytes 

I'm not super familiar with the performance counter interface, but my understanding is that "Private Bytes" is really how much page file backing a process potentially has. This isn't a terribly interesting number (the name is misleading). What we're really looking for is the "Private Working Set" which is not available as a performance counter, we have to calculate it in browser for our resident unique measurement [1].

> for linux, Private Bytes is collected from the OS at /proc/<pid>/maps:

This is measuring private writable virtual memory, which seems incorrect. In browser we use /proc/<pid>/smaps [2].

Eric is reviewing this test.
Assignee: nobody → erahm
Joel and Eric, what is the next action to understand this tp5o Private Bytes test? Should we resolve this bug (and test) as INVALID and file a new bug about creating a test for Private Working Set?
Flags: needinfo?(jmaher)
Flags: needinfo?(erahm)
I'd be okay with splitting out a bug to fix the test. From previous discussions there are two simple solutions:
  1) Within the process: |nsIMemoryReporterManager.residentUnique| [1]
  2) Outside the process: |Process.memory_full_info().uss| value from psutil [2]

For 1) on e10s we'd need to add some sort of messaging to the child process similar to what telemetry does [3].

For 2) we'd need to update the in-tree version of psutil [4]. Unfortunately the API is rather unstable so this might require some integration fixes.

As to whether a 50% increase is expected for private bytes (so this is just sum(USS)), that actually seems okay to me. I can look at some historical data to confirm my expectations.

Flags: needinfo?(erahm)
lets spin off this into a new bug.  I like to measure outside of the process as much as possible, but iirc psutil and nsiMemoryReporterManager use the same technique- so going the in process route would be nice.

I would vote for solving the new more accurate private bytes collection in bug 1250169.
Flags: needinfo?(jmaher)
Sounds good. I'll close this bug. Eric says the private bytes test for bug 1250169 does not need to block e10s release as long as MEMORY_TOTAL telemetry data continues to look good.
Closed: 7 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.