1224677 - Lots of regressions from bug 1210261

Status: RESOLVED WORKSFORME

(Core :: Graphics, defect)

Description

[Mason Chang [Inactive] [:mchang]]

And the regressions come rolling in, which are somewhat expected:

Regression: Mozilla-Inbound-Non-PGO - CanvasMark - WINNT 6.1 (ix) (e10s) - 16.3% decrease
Regression: Mozilla-Inbound-Non-PGO - Talos Page Switch - WINNT 5.1 (ix) - 19.5% increase
Regression: Mozilla-Inbound-Non-PGO - Tab Animation Test - WINNT 6.1 (ix) (e10s) - 3.88% increase
Regression: Mozilla-Inbound-Non-PGO - CanvasMark - WINNT 5.1 (e10s) - 16.6% decrease
Regression: Mozilla-Inbound-Non-PGO - Talos Page Switch - Ubuntu HW 12.04 x64 - 23.9% increase
Regression: Mozilla-Inbound-Non-PGO - CanvasMark - Ubuntu HW 12.04 x64 (e10s) - 18.7% decrease


A very weird improvement (but could be bug 1223907).
Improvement: Mozilla-Inbound-Non-PGO - a11y Row Major MozAfterPaint - Ubuntu HW 12.04 x64 - 4.28% decrease

Comment 1

[Mason Chang [Inactive] [:mchang]]

Regression: Mozilla-Inbound-Non-PGO - CanvasMark - WINNT 6.2 x64 (e10s) - 15.7% decrease

Comment 2

[Mason Chang [Inactive] [:mchang]]

Two more improvements clumped with bug 1223907:

Improvement: Mozilla-Inbound-Non-PGO - SVG, Opacity Row Major - Ubuntu HW 12.04 x64 - 53% decrease
Improvement: Mozilla-Inbound-Non-PGO - a11y Row Major MozAfterPaint - Ubuntu HW 12.04 x64 - 3.52% decrease

Comment 3

[Mason Chang [Inactive] [:mchang]]

Regression: Mozilla-Inbound-Non-PGO - CanvasMark - WINNT 6.1 (ix) - 30.3% decrease
Regression: Mozilla-Inbound-Non-PGO - CanvasMark - WINNT 5.1 (ix) - 31.6% decrease
Regression: Mozilla-Inbound-Non-PGO - CanvasMark - Ubuntu HW 12.04 x64 - 45.2% decrease
Regression: Mozilla-Inbound-Non-PGO - Talos Page Switch - WINNT 6.1 (ix) - 8.5% increase
Regression: Mozilla-Inbound-Non-PGO - Customization Animation Tests - WINNT 5.1 (e10s) - 5.76% increase

Improvements:
Improvement: Mozilla-Inbound-Non-PGO - a11y Row Major MozAfterPaint - WINNT 5.1 (ix) - 4.2% decrease
Improvement: Mozilla-Inbound-Non-PGO - a11y Row Major MozAfterPaint - WINNT 6.1 (ix) - 3.35% decrease

Comment 4

[Mason Chang [Inactive] [:mchang]]

Regression: Mozilla-Inbound-Non-PGO - Tab Animation Test - WINNT 6.2 x64 (e10s) - 2.85% increase
Regression: Mozilla-Inbound-Non-PGO - CanvasMark - WINNT 6.2 x64 - 26.3% decrease

Improvement: Mozilla-Inbound-Non-PGO - SVG, Opacity Row Major - WINNT 6.2 x64 - 64.1% decrease
Improvement: Mozilla-Inbound-Non-PGO - SVG, Opacity Row Major - Ubuntu HW 12.04 x64 - 55.2% decrease
Improvement: Mozilla-Inbound-Non-PGO - a11y Row Major MozAfterPaint - WINNT 6.2 x64 - 4.8% decrease
Improvement: Mozilla-Inbound-Non-PGO - SVG, Opacity Row Major - WINNT 6.1 (ix) - 74.5% decrease
Improvement: Mozilla-Inbound-Non-PGO - Tp5 Optimized - WINNT 6.1 (ix) - 2.86% decrease

Comment 5

[Mason Chang [Inactive] [:mchang]]

Regression: Mozilla-Inbound-Non-PGO - Talos Page Switch - WINNT 6.2 x64 - 9.62% increase

Comment 6

[Mason Chang [Inactive] [:mchang]]

Improvement: Mozilla-Inbound-Non-PGO - SVG, Opacity Row Major - WINNT 5.1 (ix) - 73.8% decrease

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

compare view:
https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-inbound&originalRevision=b0e16f67e098&newProject=mozilla-inbound&newRevision=d1029ba11d5c

keep in mind there are some noisy tests, ones that have >5% difference or >1 platform regressing are most likely legit.  You can see the graphs and subtests easily though.

given the scope of this, do you think these peformance changes are expected, within range, and worth keeping in tree?   any possibly tweaks to make it slightly better?

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

and the view in alert manager:
http://alertmanager.allizom.org:8080/alerts.html?rev=d1029ba11d5c64c455412a58a2a77d14511286d6&showAll=1&testIndex=0&platIndex=0

Comment 9

[Mason Chang [Inactive] [:mchang]]

Regression: Mozilla-Inbound - CanvasMark - MacOSX 10.10 - 13.9% decrease

Improvement: Mozilla-Inbound-Non-PGO - Talos Page Switch - Ubuntu HW 12.04 x64 (e10s) - 5.01% decrease
Improvement: Mozilla-Inbound-Non-PGO - Tp5 Optimized - WINNT 6.2 x64 - 2.94% decrease

> given the scope of this, do you think these peformance changes are expected,
> within range, and worth keeping in tree?   any possibly tweaks to make it
> slightly better?

Backed out for now. We were expecting some performance differences, not sure this bad though.

Comment 10

[Mason Chang [Inactive] [:mchang]]

Bug 1210261 landed without the part that caused the regressions, which was skipping a force refresh. No regressions came out, but some improvements did! Closing this as WFM.