1405455 - 21.77 - 23.44% tp5o Private Bytes / tp5o_webext Private Bytes (linux64) regression on push 5ef005eb34d90fe2932e08938a38aa92d83b995b (Mon Oct 2 2017)

Status: RESOLVED WONTFIX

(Core :: DOM: Content Processes, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=5ef005eb34d90fe2932e08938a38aa92d83b995b

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

 23%  tp5o Private Bytes linux64 pgo e10s     987,939,395.59 -> 1,219,475,088.30
 23%  tp5o Private Bytes linux64 opt e10s     988,464,764.88 -> 1,214,519,914.76
 23%  tp5o_webext Private Bytes linux64 opt e10s1,024,177,657.35 -> 1,255,803,045.83
 22%  tp5o_webext Private Bytes linux64 pgo e10s1,047,267,522.54 -> 1,275,261,791.94

Improvements:

 13%  tart summary linux64 opt e10s     5.62 -> 4.91
 13%  tart summary linux64 pgo e10s     5.02 -> 4.39
  7%  tart summary windows10-64 pgo e10s3.76 -> 3.49
  2%  damp summary linux64 pgo e10s     248.93 -> 243.08


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=9787

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

I had originally added a comment to bug 1385249 regarding the improvements- yes there are real improvements and some regressions in the private bytes measured on linux.

:gabor, can you look into this and help come to a resolution?

Comment 2

[Ionuț Goldan [:igoldan]]

Also, these AWSY regressions where linked to bug 1385249

== Change summary for alert #9788 (as of October 02 2017 11:54 UTC) ==

Regressions:

  4%  Heap Unclassified summary windows7-32-stylo-disabled opt stylo-disabled     38,037,211.06 -> 39,367,231.10
  3%  Heap Unclassified summary windows7-32 opt                                   37,485,188.90 -> 38,783,073.38
  3%  JS summary windows7-32 opt                                                  80,205,228.86 -> 82,640,201.82
  3%  Heap Unclassified summary linux64 opt                                       52,834,969.65 -> 54,365,741.66
  3%  Explicit Memory summary windows7-32 opt                                     215,853,062.45 -> 221,901,770.56
  3%  Explicit Memory summary windows7-32-stylo-disabled opt stylo-disabled       211,170,695.57 -> 217,032,125.79
  3%  JS summary windows7-32-stylo-disabled opt stylo-disabled                    79,964,550.16 -> 82,182,996.55
  3%  Resident Memory summary windows7-32-stylo-disabled opt stylo-disabled       286,562,418.36 -> 294,360,359.47
  3%  Heap Unclassified summary linux64-stylo-sequential opt stylo-sequential     52,561,561.39 -> 53,992,915.79
  3%  Heap Unclassified summary windows10-64 opt                                  45,417,724.40 -> 46,613,979.99
  3%  Resident Memory summary windows7-32 pgo                                     285,237,322.11 -> 292,751,661.55
  3%  Explicit Memory summary windows10-64 opt                                    277,409,195.02 -> 284,490,125.39
  2%  Explicit Memory summary windows10-64 opt                                    277,341,225.15 -> 284,044,840.45
  2%  Heap Unclassified summary windows10-64 opt                                  45,315,404.21 -> 46,409,124.09
  2%  JS summary windows10-64 opt                                                 106,449,927.77 -> 109,014,370.23
  2%  Heap Unclassified summary linux64-stylo-disabled opt stylo-disabled         53,279,036.23 -> 54,534,862.92
  2%  Resident Memory summary windows10-64 pgo                                    440,240,977.54 -> 450,309,826.90
  2%  Resident Memory summary windows10-64-stylo-disabled opt stylo-disabled      442,250,003.80 -> 452,100,741.44
  2%  Resident Memory summary windows10-64 opt                                    449,229,302.63 -> 459,187,344.01
  2%  Heap Unclassified summary windows10-64-stylo-disabled opt stylo-disabled    45,720,232.62 -> 46,720,928.02
  2%  JS summary windows10-64 pgo                                                 105,956,232.12 -> 108,188,335.76
  2%  Explicit Memory summary linux32-stylo-disabled opt stylo-disabled           208,717,084.58 -> 213,098,888.54
  2%  Explicit Memory summary windows10-64-stylo-disabled opt stylo-disabled      272,246,989.85 -> 277,919,906.72
  2%  Explicit Memory summary windows7-32 pgo                                     216,179,868.58 -> 220,660,441.58
  2%  Explicit Memory summary linux64-stylo-sequential opt stylo-sequential       274,517,718.80 -> 280,126,328.42
  2%  Explicit Memory summary linux64 opt                                         275,686,554.25 -> 281,318,709.02
  2%  Heap Unclassified summary windows10-64 pgo                                  45,254,504.04 -> 46,173,559.45
  2%  Heap Unclassified summary linux32-stylo-disabled opt stylo-disabled         40,871,682.05 -> 41,697,050.52
  2%  Explicit Memory summary windows10-64 pgo                                    276,983,630.52 -> 282,523,464.41

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=9788

Comment 3

[Gabor Krizsanits (INACTIVE)]

(In reply to Joel Maher ( :jmaher) (UTC-5) from comment #1)
> I had originally added a comment to bug 1385249 regarding the improvements-
> yes there are real improvements and some regressions in the private bytes
> measured on linux.
> 
> :gabor, can you look into this and help come to a resolution?

The original regression in this bug (tp5o Private Bytes on linux) is basically a dupe of bug 1361808. We made the decision back then that we can ignore this and I think, now that we reenable the ppm the same argument still stands.

(In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #2)
> Also, these AWSY regressions where linked to bug 1385249
> 
> == Change summary for alert #9788 (as of October 02 2017 11:54 UTC) ==
> 
> Regressions:
> 
>   4%  Heap Unclassified summary windows7-32-stylo-disabled opt
> stylo-disabled     38,037,211.06 -> 39,367,231.10
>   3%  Heap Unclassified summary windows7-32 opt                             
> 37,485,188.90 -> 38,783,073.38
>   3%  JS summary windows7-32 opt                                            
> 80,205,228.86 -> 82,640,201.82
>   3%  Heap Unclassified summary linux64 opt                                 
> 52,834,969.65 -> 54,365,741.66
>   3%  Explicit Memory summary windows7-32 opt                               
> 215,853,062.45 -> 221,901,770.56
>   3%  Explicit Memory summary windows7-32-stylo-disabled opt stylo-disabled 
> 211,170,695.57 -> 217,032,125.79
>   3%  JS summary windows7-32-stylo-disabled opt stylo-disabled              
> 79,964,550.16 -> 82,182,996.55
>   3%  Resident Memory summary windows7-32-stylo-disabled opt stylo-disabled 
> 286,562,418.36 -> 294,360,359.47
>   3%  Heap Unclassified summary linux64-stylo-sequential opt
> stylo-sequential     52,561,561.39 -> 53,992,915.79
>   3%  Heap Unclassified summary windows10-64 opt                            
> 45,417,724.40 -> 46,613,979.99
>   3%  Resident Memory summary windows7-32 pgo                               
> 285,237,322.11 -> 292,751,661.55
>   3%  Explicit Memory summary windows10-64 opt                              
> 277,409,195.02 -> 284,490,125.39
>   2%  Explicit Memory summary windows10-64 opt                              
> 277,341,225.15 -> 284,044,840.45
>   2%  Heap Unclassified summary windows10-64 opt                            
> 45,315,404.21 -> 46,409,124.09
>   2%  JS summary windows10-64 opt                                           
> 106,449,927.77 -> 109,014,370.23
>   2%  Heap Unclassified summary linux64-stylo-disabled opt stylo-disabled   
> 53,279,036.23 -> 54,534,862.92
>   2%  Resident Memory summary windows10-64 pgo                              
> 440,240,977.54 -> 450,309,826.90
>   2%  Resident Memory summary windows10-64-stylo-disabled opt stylo-disabled
> 442,250,003.80 -> 452,100,741.44
>   2%  Resident Memory summary windows10-64 opt                              
> 449,229,302.63 -> 459,187,344.01
>   2%  Heap Unclassified summary windows10-64-stylo-disabled opt
> stylo-disabled    45,720,232.62 -> 46,720,928.02
>   2%  JS summary windows10-64 pgo                                           
> 105,956,232.12 -> 108,188,335.76
>   2%  Explicit Memory summary linux32-stylo-disabled opt stylo-disabled     
> 208,717,084.58 -> 213,098,888.54
>   2%  Explicit Memory summary windows10-64-stylo-disabled opt stylo-disabled
> 272,246,989.85 -> 277,919,906.72
>   2%  Explicit Memory summary windows7-32 pgo                               
> 216,179,868.58 -> 220,660,441.58
>   2%  Explicit Memory summary linux64-stylo-sequential opt stylo-sequential 
> 274,517,718.80 -> 280,126,328.42
>   2%  Explicit Memory summary linux64 opt                                   
> 275,686,554.25 -> 281,318,709.02
>   2%  Heap Unclassified summary windows10-64 pgo                            
> 45,254,504.04 -> 46,173,559.45
>   2%  Heap Unclassified summary linux32-stylo-disabled opt stylo-disabled   
> 40,871,682.05 -> 41,697,050.52
>   2%  Explicit Memory summary windows10-64 pgo                              
> 276,983,630.52 -> 282,523,464.41
> 
> For up to date results, see:
> https://treeherder.mozilla.org/perf.html#/alerts?id=9788

This is also an expected regression and since the biggest improvements this patch brings to the table is that we don't have to pay the cost of content process startup time in most of the cases (what cpstartup measures now and typically 200-300ms even on fast machines). I would pay this cost any time. Note that we don't see that improvement, since I turned off the ppm for the cpstartup time test, because it would make the test meaningless.

For people using a single tab and have a relatively low memory footprint the cost is relatively high, but that case is not the biggest concern usually. For heavy users where memory consumption really matters even in the single tab case the memory cost of the extra process is relatively small, for multi tab users it's none (beyond 4 tabs), so in general I'm not super worried although I would of course prefer if content process startup were fast and we didn't need the ppm at all.

I think the right person to make the final call is Eric.

Comment 4

[Eric Rahm [:erahm]]

Yeah it's the same deal as bug 1361808. I confirmed it's limited to the startup measurements and is due to an extra content process. While this is unfortunate, it's expected and I'm okay with taking the regression.