1387247 - 6.15 - 13.69% cart / tart (linux64, osx-10-10) regression on push 9aea49fa17a1 (Thu Aug 3 2017)

Status: RESOLVED WONTFIX

(Firefox :: Theme, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=9aea49fa17a1

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

 14%  cart summary linux64 pgo e10s     7.71 -> 8.77
 13%  cart summary linux64 opt e10s     8.72 -> 9.84
 13%  tart summary linux64 opt e10s     4.95 -> 5.59
 11%  tart summary linux64 pgo e10s     4.30 -> 4.77
  8%  cart summary osx-10-10 opt e10s   10.53 -> 11.35
  6%  tart summary osx-10-10 opt e10s   9.81 -> 10.41


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=8527

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

::dao, I see you are the patch author of one of the patches in bug 1349555- there are some larger regressions here for tart/cart- are these expected?

Comment 2

[Dão Gottwald [:dao]]

We've increased the maximum tab width in bug 1349555, which I think would explain the tart regression. Not sure about cart, but I don't think cart is relevant anymore and should probably be removed since we don't animate entering customize mode anymore.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

I filed bug 1387255 to track disabling cart.  Given that tart is accounted for, should we close this out as wontfix?

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

there is a 6% tresize improvement on win7 :)

Comment 5

[Justin Dolske [:Dolske]]

(In reply to Dão Gottwald [::dao] from comment #2)
> We've increased the maximum tab width in bug 1349555, which I think would
> explain the tart regression.

Can we confirm this by comparing with a push that doesn't increase the tab width? mconley might be able to assist here if needed?

Comment 6

[Dão Gottwald [:dao]]

I can do that.

Comment 7

[Ionuț Goldan [:igoldan]]

Have you got time to look over this issue?

Comment 8

[Dão Gottwald [:dao]]

Not yet.

Comment 9

[Jim Mathies [:jimm]]

Based on comment 2 and we're a week from final 57 merge.

Comment 10

[Dão Gottwald [:dao]]

I did a comparison between the old tab max-width (210px) and the new one (225px):

https://treeherder.mozilla.org/perf.html#/compare?originalProject=try&originalRevision=7baa18b298bc2c5eeabe31565a19fb555c2d0c57&newProject=try&newRevision=76109e523b03f0950e837b030dafb259139fec9d&framework=1&filter=tart

windows10-64: 1.50% improvement -- makes no sense
linux64: 1.36% regression compared to 13% (comment 0)
osx-10-10: 3.78% regression compared to 6% (comment 0)

The bottom line is that Windows is unaffected, the Mac regression can for the most part be explained by the increased tab width, and the Linux regression is mostly unclear. I don't think it's worth digging deeper here. In hindsight it might have been better to land bug 1349555 in smaller pieces -- we didn't do that because in theory the photon tab strip should be cheaper than the Australis one (luckily tresize agreed even though tart didn't).