1367388 - 41.07% tp5o responsiveness (linux64) regression on push 5a0655f29081298bcad111da0a8fd02655a2abc9 (Tue May 23 2017)

Status: RESOLVED WONTFIX

(Core :: JavaScript Engine, defect)

Description

[Ionuț Goldan [:igoldan]]

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=5a0655f29081298bcad111da0a8fd02655a2abc9

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

 41%  tp5o responsiveness linux64 opt e10s     3.81 -> 5.38

Improvements:

  5%  ts_paint osx-10-10 opt e10s     1,235.08 -> 1,178.08
  3%  ts_paint windows8-64 opt e10s   877.79 -> 854.58


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=6791

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Ionuț Goldan [:igoldan]]

:kmag Can you confirm this bug? Looks like the linux platform was affected.

Comment 2

[Kris Maglione [:kmag]]

I don't think this regression is real. My best guess is that, if these patches did have an effect on those numbers, it's because the improvement in ts_paint caused some things to run during the page loader tests rather than before paint.

But these numbers unexpectedly dropped when SETA was disabled, and returned to approximately their previous values at around the time this landed (which was just before another talos-related change).

Comment 3

[Joel Maher ( :jmaher ) (UTC -8) (PTO back normal Nov 17)]

It looks like this is a sustained regression and it is clearly marked as a single revision:
https://treeherder.mozilla.org/perf.html#/graphs?series=%5Bmozilla-inbound,8cba34a5b756cc4c005381526f637533c22f2f31,1,1%5D&zoom=1495553017035.4785,1495559779605.063,3.412599504603504,5.845625637667511&selected=%5Bmozilla-inbound,8cba34a5b756cc4c005381526f637533c22f2f31,207261,276656339,1%5D

I do see that there is an improvement a week earlier during the time period where we accidentally turned of talos for 1.5 days.  Something in there must have improved tp5o responsiveness, this regression is posting slightly higher values that we had before the improvement.

:kmag, are you saying that you fixed something a week earlier and your current patches toggled it back?  a 40% change in value is pretty high.

Comment 4

[Kris Maglione [:kmag]]

No, I'm saying that if this patch is responsible, it's most likely just an artifact of paint happening earlier, and causing slightly higher event queue loads after paint due to queued operations happening after paint rather than before.

But I'm also slightly suspicious that it may be infra-related.

Either way, 40% may be pretty high, but 1.5ms is not.

Comment 5

[Kris Maglione [:kmag]]

I'm going to call this WONTFIX unless someone disagrees. I still think this is just an artifact of exactly when things run during startup, but even if it's not, I think a 1.5ms responsiveness regression on Linux is worth it for a 3-5% startup improvement on Windows and OS-X.