1248366 - 2.49 - 2.95% damp e10s (linux64, windows7) regression on fx-team (v.47) from push cb04e167e1df (Wed Feb 10 2016)

Status: RESOLVED FIXED

(DevTools :: Framework, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

Talos has detected a Firefox performance regression from push cb04e167e1df. As author of one of the patches included in that push, we need your help to address this regression.

This is a list of all known regressions and improvements related to the push:

https://treeherder.allizom.org/perf.html#/alerts?id=102

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see:
https://wiki.mozilla.org/Buildbot/Talos/Tests#DAMP

Reproducing and debugging the regression:

If you would like to re-run this Talos test on a potential fix, use try with the following syntax:
try: -b o -p linux64,win32 -u none -t g2-e10s[Windows 7] --rebuild 5  # add "mozharness: --spsProfile" to generate profile data
(we suggest --rebuild 5 to be more confident in the results)

To run the test locally and do a more in-depth investigation, first set up a local Talos environment:
https://wiki.mozilla.lorg/Buildbot/Talos/Running#Running_locally_-_Source_Code

Then run the following command from the directory where you set up Talos:
talos --develop -e <path>/firefox -a damp --e10s

Making a decision:
As the patch author we need your feedback to help us handle this regression.
*** Please let us know your plans by Thursday, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations:
https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling</path>

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

I have done a lot of retriggers on the tree to collect more data:
https://treeherder.mozilla.org/#/jobs?repo=fx-team&filter-searchStr=talos%20g2&tochange=6e66d7b4e957&fromchange=912f7aa4d70f

more data is coming in still, but here is a compare view to reference the regressions;
https://treeherder.mozilla.org/perf.html#/compare?originalProject=fx-team&originalRevision=672120149d88&newProject=fx-team&newRevision=cb04e167e1df&framework=1

clicking on the subtests ^ for linux64, the .close methods are taking a big hit.

:ochameau, can you take a look at this bug and help us get this resolved?

Comment 2

[Alexandre Poirot [:ochameau]]

In bug 1188405, I'm introducing a compatibility wrapper in gDevTools.jsm.
This wrapper is most likely slower.
But I landed in bug 1245462, a significant refactoring to bypass this compatibility wrapper.
So that I'm expecting performance to be equivalent between before bug 1188405 and after bug 1245462.
It shouldn't be much better of worse before/after these two patches.

Is there a way to do a Talos run before bug 1188405 and after 1245462?
Or to be very sure of what I'm saying: with 1188405 and a cherry-pick of bug 1245462 on top of it?

Longer story:
Devtools were using gDevTools.jsm. A JSM module.
This module has been split into two common js module, loaded via the devtools loader (Loader.jsm).
In bug 1188405, I just did that. And I kept gDevTools.jsm. In order to keep it working, I added some wrapper that will automatically call one of the two common js modules. This add a layer of function and I can easily understand some performance hit.
In bug 1245462, I refactored all the codebase to load the two common js modules directly, so that there shouldn't be any additional wrapper.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

do we want to push to try with a certain revision and a patch from bug 1245462?

looking at the graphs we don't have any net regressions now that a few days have gone by (i.e. bug 1245462):
https://treeherder.allizom.org/perf.html#/graphs?timerange=1209600&series=[fx-team,e8783606b9fae730f8511d1e1e6f5ad6a5130be2,1]&highlightedRevisions=cb04e167e1df

https://treeherder.allizom.org/perf.html#/graphs?timerange=1209600&series=[fx-team,343d08606a7f3b7341e0dbcdf7d6618db97fe5e0,1]&highlightedRevisions=cb04e167e1df


could we just mark this as fixed?

Comment 4

[Alexandre Poirot [:ochameau]]

Yes. I don't think it is worth investigating it.
We can easily see numbers getting back to normal on Feb 11, when bug 1245462 reached fx-team.
https://treeherder.allizom.org/perf.html#/graphs?timerange=1209600&series=[fx-team,e8783606b9fae730f8511d1e1e6f5ad6a5130be2,1]&highlightedRevisions=cb04e167e1df

But this is great to hear that DAMP is able to track such possible regression!

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

:)