1208048 - 31-33% MacOS* tcanvasmark regression on Mozilla-Inbound (v.44) on September 24, 2015 from push 5ed72fdd632767e03d6862efd10fc15a34256845

Status: RESOLVED WONTFIX

(Core :: Graphics, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

Talos has detected a Firefox performance regression from your commit 5ed72fdd632767e03d6862efd10fc15a34256845 in bug 1206161.  We need you to address this regression.

This is a list of all known regressions and improvements related to your bug:
http://alertmanager.allizom.org:8080/alerts.html?rev=5ed72fdd632767e03d6862efd10fc15a34256845&showAll=1

On the page above you can see Talos alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test, please see: https://wiki.mozilla.org/Buildbot/Talos/Tests#CanvasMark

Reproducing and debugging the regression:
If you would like to re-run this Talos test on a potential fix, use try with the following syntax:
try: -b o -p macosx64 -u none -t chromez  # add "mozharness: --spsProfile" to generate profile data

To run the test locally and do a more in-depth investigation, first set up a local Talos environment:
https://wiki.mozilla.org/Buildbot/Talos/Running#Running_locally_-_Source_Code

Then run the following command from the directory where you set up Talos:
talos --develop -e <path>/firefox -a tcanvasmark

Making a decision:
As the patch author we need your feedback to help us handle this regression.
*** Please let us know your plans by Monday, or the offending patch will be backed out! ***

Our wiki page oulines the common responses and expectations:
https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

Matt, can you take a look at this.  This appears to be the only regression.  Sadly our osx results are very noisy, so we normally get alerts on larger regressions (>8%).  This seems to make sense with Skia causing a canvasmark regression.  This is a large regression- we should understand it.

Comment 2

[Sylvestre Ledru [:Sylvestre]]

[Tracking Requested - why for this release]:
Important perf regression

Comment 3

[Matt Woodrow (:mattwoodrow)]

This is unexpected, I'll investigate this today.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

thanks!  I am in the toronto office if you have questions, in meetings in the common space, but maybe at lunch we can sync up.

Comment 5

[Matt Woodrow (:mattwoodrow)]

I can reproduce this locally.

For the most part it just seems that skia-gl is slower than software for some operations (stroking, get/put pixel data etc), and this benchmark tests a lot of that.

We could probably do better for some things (like caching temp surfaces we allocate for shadows), but there's not too much we can do.

I'll talk to the other gfx people tomorrow and make a decision about what to do with this.

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

Matt, Thanks for jumping on this!  I am excited you can reproduce this locally and that you have some ideas about what to do.

Comment 7

[Matt Woodrow (:mattwoodrow)]

We've decided to just live with this regression. We do better on some sub-tests, and they tend to be the ones we care more about (image blitting).

We also appear to beat chrome still on my machine (most of the time, the results are very noisy).

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

I appreciate you looking into this!  Now we have some docs for this if we have questions in the future.

Comment 9

[Liz Henry (:lizzard)]

Sounds like we don't need to track this since y'all are wontfixing it.