1263281 - 3.87 - 4.46% kraken (windows7-32, windows8-64) regression on push d20bb8617b25 (Thu Apr 7 2016)

Status: RESOLVED FIXED

(Core :: IPC, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

Talos has detected a Firefox performance regression from push d20bb8617b25. As author of one of the patches included in that push, we need your help to address this regression.

This is a list of all known regressions and improvements related to the push:
https://treeherder.mozilla.org/perf.html#/alerts?id=802

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see:
https://wiki.mozilla.org/Buildbot/Talos/Tests#kraken

Reproducing and debugging the regression:

If you would like to re-run this Talos test on a potential fix, use try with the following syntax:
try: -b o -p win64,win32 -u none -t dromaeojs-e10s --rebuild 5  # add "mozharness: --spsProfile" to generate profile data

(we suggest --rebuild 5 to be more confident in the results)

To run the test locally and do a more in-depth investigation, first set up a local Talos environment:
https://wiki.mozilla.lorg/Buildbot/Talos/Running#Running_locally_-_Source_Code

Then run the following command from the directory where you set up Talos:
talos --develop -e [path]/firefox -a kraken

(add --e10s to run tests in e10s mode)

Making a decision:
As the patch author we need your feedback to help us handle this regression.
*** Please let us know your plans by Tuesday, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations:
https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

I pushed to try to bisect this:
https://treeherder.mozilla.org/#/jobs?repo=try&author=[e-mail]&fromchange=78cf984976a9&tochange=b3fd813443bf

ideally this will tell us what we want to know.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

the culprit is bug 1235633:
https://hg.mozilla.org/integration/mozilla-inbound/rev/dd3e03fcb06b

:billm, can you help figure out why this is happening?

Comment 4

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

Joel, I pushed a potential fix to try in bug 1264398.
https://treeherder.mozilla.org/#/jobs?repo=try&revision=1b00e056507e
Would you mind doing whatever retriggers are needed to get a valid comparison and see if this fixes the problem?

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

win7 results are in:
https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-central&originalRevision=bc2373295e31&newProject=try&newRevision=1b00e056507e&framework=1&filter=kraken

win8 is scheduled, but backlogged (even after 8+ hours)

Comment 6

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

So it seems like this is fixed?

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

verified these are fixed on the graphs!  Thanks for the fix:)