1.81 - 2.52% damp (linux64, osx-10-10, windows10-64, windows7-32) regression on push 48ad9a270454 (Tue Oct 31 2017)

RESOLVED WONTFIX

Status

()

Firefox
Developer Tools: Console
RESOLVED WONTFIX
19 days ago
11 days ago

People

(Reporter: igoldan, Unassigned)

Tracking

(Blocks: 1 bug, {perf, regression, talos-regression})

Firefox Tracking Flags

(Not tracked)

Details

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=68b8b82c458e0dc0bf87934853dea9009387db71&tochange=48ad9a270454f772394bdcb19d666d879f69ac1f

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

  3%  damp summary linux64 pgo e10s     241.77 -> 247.87
  2%  damp summary osx-10-10 opt e10s   303.13 -> 310.57
  2%  damp summary windows7-32 opt e10s 278.39 -> 284.65
  2%  damp summary linux64 opt e10s     278.84 -> 284.64
  2%  damp summary windows10-64 opt e10s263.00 -> 267.75


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=10277

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling
Component: Untriaged → Developer Tools: Console
:nchevobbe These perf regressions were expected, given the new DAMP test you added.

I would have resolved this as WONTFIX, but I noticed the subtests from Linux.
A 4% regression on damp complicated.jsdebugger.open.DAMP test was picked up on the OPT build.
Multiple subtests regressions were noticed on the PGO build, via the damp complicated.* and damp simple.* tests, though some here may be related to other changesets. I will do some extra backfilling, to be more precise.

Please take a look over the subtests and state whehter we can address these perf regressions.

To view the subtests comparison, hover over the "damp summary linux64 opt e10s" alert then press the "subtests" link.
Flags: needinfo?(nchevobbe)
Hello Ionuț,

I checked multiple times, and the changes from that patch are never called when the debugger open.
They are related to the console only and I do think the regression we see here is more a result of 
the new DAMP test.
I applied the patch suggested in https://bugzilla.mozilla.org/show_bug.cgi?id=1411243#c12 (which change how damp run tests), and I can't see any regression in subtests at the moment (not done though).

base: https://treeherder.mozilla.org/#/jobs?repo=try&revision=2773c9789196768b7b040c69dc87116bc2a9053e
new: https://treeherder.mozilla.org/#/jobs?repo=try&revision=f6d02ed951c717ef7b4aa21afe93db89fe1639f6
compare: https://treeherder.mozilla.org/perf.html#/compare?originalProject=try&originalRevision=2773c9789196768b7b040c69dc87116bc2a9053e&newProject=try&newRevision=f6d02ed951c717ef7b4aa21afe93db89fe1639f6&framework=1
Flags: needinfo?(nchevobbe)
(In reply to Nicolas Chevobbe [:nchevobbe] from comment #2)
> I applied the patch suggested in
> https://bugzilla.mozilla.org/show_bug.cgi?id=1411243#c12 (which change how
> damp run tests), and I can't see any regression in subtests at the moment
> (not done though).
Results have arrived. I don't see any regressions either. I guess we can close this now as WONTFIX.
Status: NEW → RESOLVED
Last Resolved: 11 days ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.