1450860 - 11.07 - 13.89% tp5o (windows10-64/windows7-32) regression on push 9d08798cee62 (Fri Mar 30 2018)

Status: RESOLVED WORKSFORME

(Testing :: Talos, defect)

Description

[Ionuț Goldan [:igoldan]]

== Change summary for alert #12427 (as of Fri, 30 Mar 2018 07:08:55 GMT) ==

Regressions:

 14%  tp5o windows7-32 pgo e10s stylo     203.12 -> 231.33
 11%  tp5o windows10-64 pgo e10s stylo    201.24 -> 223.52

Improvements:

 82%  speedometer windows10-64 opt e10s stylo     34.00 -> 61.89
 80%  stylebench windows10-64 opt e10s stylo      35.28 -> 63.40
 79%  tp5o responsiveness windows10-64 pgo e10s stylo1.66 -> 0.34
 79%  dromaeo_css windows7-32 opt e10s stylo      6,939.78 -> 12,408.08
 77%  tp5o responsiveness windows10-64 opt e10s stylo1.75 -> 0.41
 76%  dromaeo_css windows10-64 opt e10s stylo     7,415.49 -> 13,057.31
 75%  speedometer windows7-32 opt e10s stylo      34.08 -> 59.57
 75%  tp5o responsiveness windows7-32 pgo e10s stylo1.21 -> 0.31
 73%  stylebench windows7-32 opt e10s stylo       29.15 -> 50.46
 71%  dromaeo_css windows10-64 pgo e10s stylo     7,779.04 -> 13,274.37
 68%  speedometer windows10-64 pgo e10s stylo     42.35 -> 71.08
 65%  dromaeo_css windows7-32 pgo e10s stylo      7,532.33 -> 12,449.75
 63%  tp5o_webext responsiveness windows10-64 pgo e10s stylo2.59 -> 0.96
 62%  tp5o_webext responsiveness windows10-64 opt e10s stylo2.77 -> 1.05
 60%  tp5o_scroll windows7-32 pgo e10s stylo      2.12 -> 0.84
 60%  speedometer windows7-32 pgo e10s stylo      44.88 -> 71.83
 60%  tp5o_scroll windows7-32 opt e10s stylo      2.17 -> 0.87
 60%  tp5o_webext responsiveness windows7-32 pgo e10s stylo2.23 -> 0.89
 59%  stylebench windows10-64 pgo e10s stylo      45.47 -> 72.15
 57%  tscrollx windows7-32 pgo e10s stylo         2.55 -> 1.09
 55%  tp5o_scroll windows10-64 pgo e10s stylo     1.74 -> 0.79
 54%  a11yr windows10-64 opt e10s stylo           533.80 -> 246.73
 53%  stylebench windows7-32 pgo e10s stylo       43.36 -> 66.54
 53%  tscrollx windows7-32 opt e10s stylo         2.55 -> 1.20
 53%  tscrollx windows10-64 pgo e10s stylo        2.39 -> 1.12
 50%  a11yr windows7-32 opt e10s stylo            635.07 -> 314.90
 50%  tsvgx windows7-32 pgo e10s stylo            373.39 -> 185.20
 49%  tscrollx windows10-64 opt e10s stylo        2.22 -> 1.13
 49%  tp5o_scroll windows10-64 opt e10s stylo     1.75 -> 0.90
 47%  a11yr windows10-64 pgo e10s stylo           355.96 -> 189.47
 45%  tp6_amazon windows10-64 opt e10s stylo      562.02 -> 306.92
 45%  tsvgr_opacity windows10-64 opt e10s stylo   228.89 -> 125.73
 45%  tsvgr_opacity windows7-32 pgo e10s stylo    213.87 -> 117.91
 44%  a11yr windows7-32 pgo e10s stylo            370.38 -> 205.75
 44%  tp6_google windows10-64 opt e10s stylo      790.17 -> 439.00
 44%  tsvgr_opacity windows7-32 opt e10s stylo    253.99 -> 142.50
 43%  tsvgx windows7-32 opt e10s stylo            434.84 -> 246.29
 42%  tp6_amazon windows7-32 pgo e10s stylo       494.25 -> 286.79
 42%  tp6_amazon windows7-32 opt e10s stylo       644.98 -> 374.62
 41%  tsvgr_opacity windows10-64 pgo e10s stylo   201.21 -> 119.43
 41%  about_preferences_basic windows10-64 opt e10s stylo246.18 -> 146.34
 40%  tp6_google windows7-32 opt e10s stylo       800.27 -> 480.67
 40%  tp6_amazon windows10-64 pgo e10s stylo      453.90 -> 273.25
 40%  tp6_google windows10-64 pgo e10s stylo      670.73 -> 404.38
 40%  perf_reftest_singletons windows10-64 opt e10s stylo26.93 -> 16.28
 39%  tp6_facebook windows10-64 opt e10s stylo    311.71 -> 189.83
 39%  kraken windows7-32 opt e10s stylo           1,713.26 -> 1,052.08
 38%  tart windows7-32 opt e10s stylo             4.58 -> 2.84
 38%  ts_paint windows7-32 opt e10s stylo         636.33 -> 395.50
 38%  kraken windows10-64 pgo e10s stylo          1,548.29 -> 964.94
 37%  kraken windows10-64 opt e10s stylo          1,588.16 -> 993.19
 37%  perf_reftest_singletons windows7-32 opt e10s stylo26.87 -> 16.82
 37%  about_preferences_basic windows10-64 pgo e10s stylo207.35 -> 130.53
 37%  sessionrestore windows7-32 opt e10s stylo   460.46 -> 290.42
 37%  sessionrestore_many_windows windows10-64 opt e10s stylo1,472.83 -> 930.83
 37%  sessionrestore windows10-64 opt e10s stylo  415.96 -> 263.42
 37%  tpaint windows10-64 opt e10s stylo          241.61 -> 153.26
 36%  sessionrestore_no_auto_restore windows10-64 opt e10s stylo495.42 -> 315.33
 36%  tart windows7-32 pgo e10s stylo             3.84 -> 2.45
 36%  ts_paint_webext windows7-32 opt e10s stylo  718.88 -> 458.92
 36%  sessionrestore_no_auto_restore windows7-32 opt e10s stylo542.00 -> 346.50
 36%  perf_reftest_singletons windows10-64 pgo e10s stylo23.63 -> 15.11
 36%  kraken windows7-32 pgo e10s stylo           1,663.59 -> 1,063.78
 36%  displaylist_mutate windows10-64 opt e10s stylo6,934.45 -> 4,471.73
 35%  sessionrestore_many_windows windows7-32 opt e10s stylo1,587.04 -> 1,025.92
 35%  cpstartup content-process-startup windows10-64 opt e10s stylo284.29 -> 185.92
 34%  about_preferences_basic windows7-32 opt e10s stylo246.19 -> 162.13
 34%  ts_paint windows7-32 pgo e10s stylo         554.71 -> 365.67
 34%  tp6_google windows7-32 pgo e10s stylo       646.79 -> 428.92
 34%  perf_reftest windows7-32 opt e10s stylo     2.59 -> 1.72
 34%  perf_reftest_singletons windows7-32 pgo e10s stylo22.51 -> 14.96
 33%  sessionrestore windows10-64 pgo e10s stylo  366.38 -> 244.58
 33%  sessionrestore_many_windows windows10-64 pgo e10s stylo1,255.38 -> 840.00
 33%  ts_paint windows10-64 opt e10s stylo        548.21 -> 367.75
 33%  ts_paint_webext windows10-64 opt e10s stylo 621.46 -> 418.75
 32%  tp6_facebook windows7-32 opt e10s stylo     300.27 -> 202.79
 32%  sessionrestore_no_auto_restore windows10-64 pgo e10s stylo435.42 -> 295.00
 32%  tart windows10-64 opt e10s stylo            3.91 -> 2.66
 32%  ts_paint_webext windows7-32 pgo e10s stylo  619.04 -> 422.17
 32%  cpstartup content-process-startup windows10-64 pgo e10s stylo261.71 -> 179.08
 31%  damp windows10-64 opt e10s stylo            109.03 -> 74.82
 31%  cpstartup content-process-startup windows7-32 opt e10s stylo290.25 -> 199.42
 31%  displaylist_mutate windows7-32 opt e10s stylo8,824.82 -> 6,105.05
 30%  sessionrestore windows7-32 pgo e10s stylo   381.12 -> 265.25
 30%  tart windows10-64 pgo e10s stylo            3.45 -> 2.40
 30%  tp6_facebook windows10-64 pgo e10s stylo    254.94 -> 177.79
 30%  rasterflood_svg windows7-32 opt e10s stylo  16,383.11 -> 11,442.58
 30%  ts_paint_webext windows10-64 pgo e10s stylo 548.17 -> 385.00
 30%  tpaint windows10-64 pgo e10s stylo          202.91 -> 142.61
 29%  ts_paint windows10-64 pgo e10s stylo        485.58 -> 343.58
 29%  damp windows10-64 pgo e10s stylo            95.27 -> 67.62
 29%  sessionrestore_many_windows windows7-32 pgo e10s stylo1,264.38 -> 900.17
 29%  damp windows7-32 opt e10s stylo             114.60 -> 81.69
 29%  sessionrestore_no_auto_restore windows7-32 pgo e10s stylo448.25 -> 320.50
 28%  displaylist_mutate windows10-64 pgo e10s stylo5,848.54 -> 4,196.78
 28%  perf_reftest windows7-32 pgo e10s stylo     2.36 -> 1.69
 28%  perf_reftest windows10-64 pgo e10s stylo    2.32 -> 1.66
 28%  perf_reftest windows10-64 opt e10s stylo    2.52 -> 1.81
 27%  basic_compositor_video windows10-64 opt e10s stylo3.71 -> 2.71
 26%  basic_compositor_video windows10-64 pgo e10s stylo3.68 -> 2.71
 26%  basic_compositor_video windows7-32 opt e10s stylo3.68 -> 2.71
 26%  rasterflood_svg windows7-32 pgo e10s stylo  15,352.19 -> 11,325.08
 26%  basic_compositor_video windows7-32 pgo e10s stylo3.65 -> 2.71
 26%  tp6_youtube windows7-32 opt e10s stylo      275.75 -> 204.50
 26%  tp6_facebook windows7-32 pgo e10s stylo     245.81 -> 182.83
 26%  damp windows7-32 pgo e10s stylo             95.26 -> 70.95
 25%  about_preferences_basic windows7-32 pgo e10s stylo189.99 -> 141.79
 25%  displaylist_mutate windows7-32 pgo e10s stylo6,347.07 -> 4,767.63
 24%  tpaint windows7-32 pgo e10s stylo           192.01 -> 145.46
 23%  cpstartup content-process-startup windows7-32 pgo e10s stylo251.67 -> 192.75
 23%  tabpaint windows10-64 pgo e10s stylo        67.19 -> 51.64
 20%  tps windows10-64 opt e10s stylo             23.27 -> 18.67
 19%  tp5o_webext windows10-64 opt e10s stylo     346.66 -> 280.43
 17%  tabpaint windows10-64 opt e10s stylo        69.40 -> 57.38
 17%  tsvg_static windows10-64 opt e10s stylo     52.41 -> 43.49
 15%  tsvg_static windows7-32 opt e10s stylo      52.29 -> 44.34
 15%  tps windows10-64 pgo e10s stylo             21.60 -> 18.34
 15%  tsvg_static windows7-32 pgo e10s stylo      52.56 -> 44.81
 14%  tabpaint windows7-32 pgo e10s stylo         62.52 -> 53.59
  9%  tsvg_static windows10-64 pgo e10s stylo     47.83 -> 43.37
  8%  tp5o_webext windows10-64 pgo e10s stylo     281.94 -> 260.54
  6%  rasterflood_svg windows10-64 opt e10s stylo 11,563.54 -> 10,868.47
  4%  rasterflood_svg windows10-64 pgo e10s stylo 11,265.84 -> 10,778.72

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=12427

Comment 1

[Ionuț Goldan [:igoldan]]

:jmaher Lots and lots and perf improvements on Windows from now on -- perf baseline update. I filed this bug only to highlight the PGO regressions according to tp5o. Their OPT counterparts record quite the opposite: perf improvements.

I guess I can close this bug as WORKSFORME, as it sounds like a temporary glitch?
Also, if I blocked the wrong bug from the patch, please adjust this.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

yes, thanks for filing this- this change is just the new hardware and the PGO only regressions are confusing but not really actionable if OPT is an improvement.