Closed Bug 1297913 Opened 3 years ago Closed 3 years ago

1.34 - 6% cart / kraken (linux64, windows8-64, windowsxp) regression on push 520e4b9d3ed0990d85665e82027c752516a482c2 (Sat Aug 20 2016)

Categories

(Firefox :: Untriaged, defect)

51 Branch
defect
Not set

Tracking

()

RESOLVED WONTFIX
Tracking Status
firefox51 --- affected

People

(Reporter: ashiue, Unassigned)

References

Details

(Keywords: perf, regression, talos-regression)

Talos has detected a Firefox performance regression from push 520e4b9d3ed0990d85665e82027c752516a482c2. As author of one of the patches included in that push, we need your help to address this regression.

Summary of tests that regressed:

  kraken summary windows8-64 opt: 1554.68 -> 1579.07 (1.57% worse)
  kraken summary linux64 opt: 1549.99 -> 1570.82 (1.34% worse)
  cart summary windowsxp opt: 43.57 -> 46.19 (6% worse)
  kraken summary linux64 pgo: 1492.1 -> 1525.28 (2.22% worse)

Summary of tests that improved:

  tabpaint summary windows8-64 opt: 90.29 -> 86.96 (3.69% better)


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=2644

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling
After doing some retriggers, this issue might be caused by :
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=c330ce7c763ced88a84cd36d315984de8dff7b06&tochange=520e4b9d3ed0990d85665e82027c752516a482c2

Hi Jon, as you are the patch author, can you take a look at this and determine what is the root cause? Thanks!
Blocks: 1291351, 1293127
Flags: needinfo?(jcoppeard)
(In reply to Alison Shiue from comment #0)

>   cart summary windowsxp opt: 43.57 -> 46.19 (6% worse)

This is due to the cart 2-customize-exit.half.TART subtest.  This seems to have returned to its previous level with no related changes happening.

>   kraken summary windows8-64 opt: 1554.68 -> 1579.07 (1.57% worse)
>   kraken summary linux64 opt: 1549.99 -> 1570.82 (1.34% worse)
>   kraken summary linux64 pgo: 1492.1 -> 1525.28 (2.22% worse)

These are mainly due to the audio-fft subtest.  I can't imagine how my change would affect that though (the test benchmarks a numeric kernel that does create any objects let alone create any cross compartment wrappers).

Also, these regressions didn't show up on arewefastyet.com which also runs kraken.
Flags: needinfo?(jcoppeard)
talos runs on different hardware than arewefastyet.com.  I suspect that is the primary reason we don't see any issues.

AWFY Kraken Audio FFT shows no change:
https://treeherder.mozilla.org/perf.html#/graphs?series=%5Bmozilla-inbound,da8554860c8fcbd6070262f201c191474c1685a8,1,5%5D

looking into the cart regression, I see it was fixed somewhere in this range:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=1348afa1c6a0f2fd30db419f7099334e965f4ddb&tochange=3b4d6ed9dbec830bb9e5a3d76b93281ad381976c

both the summary and the individual test are fixed in that commit range.  I am not sure if any of those commits would toggle code such to fix it.  Sadly there is no change from that to fiddle with kraken.

Are there possible investigations, or should we mark this as wontfix?
I don't think there's anything we can do here.  Marking wontfix.
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.