1411588 - DAMP tests after moving Net Monitor, Web Console and Inspector over to ES6 classes

Status: RESOLVED WORKSFORME

(DevTools :: Console, defect, P2)

Description

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

In bugs 1409064, 1410119, 1411231 and 1411531 we moved the Network Monitor, Inspector and Web Console over to ES6 classes.

We now need to run these patches through DAMP to check for regressions.

Comment 1

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-central&originalRevision=cfb0f6d21b35&newProject=try&newRevision=a8a5ed746be4ed32923c789a1de0e20cc20f3c3b&framework=1&filter=damp

https://treeherder.mozilla.org/#/jobs?repo=try&revision=a8a5ed746be4ed32923c789a1de0e20cc20f3c3b&group_state=expanded

Comment 2

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

Attachment: Results

Comment 3

[Alexandre Poirot [:ochameau]]

(In reply to Michael Ratcliffe [:miker] [:mratcliffe] from comment #1)
> https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-
> central&originalRevision=cfb0f6d21b35&newProject=try&newRevision=a8a5ed746be4
> ed32923c789a1de0e20cc20f3c3b&framework=1&filter=damp

I don't know how you got that link, but this is using a base changeset with only one talos run.
So that the results are not high confidence.

bug 1399465 should soon document how to use treeherder, perfherder and DAMP.

You should use the link that is sent about your try run, which allow comparing against the last 2 days of mozilla-central talos runs:
https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-central&newProject=try&newRevision=a8a5ed746be4ed32923c789a1de0e20cc20f3c3b&framework=1&filter=damp&selectedTimeRange=172800
The summary results looks good. There is macosx being reported to regress by 3%, but that's not true.
If you hover "315.34 ± 18.24%", you will see a black popup with individual run results.
And there, see that one run was as slow as 588ms, whereas all the others are around 300ms.
Sometimes, machines get busy with background service/task and such thing happen...

Otherwise, you should also and always take a look at subtests, as the summary isn't always trustworthy:
https://treeherder.mozilla.org/perf.html#/comparesubtest?originalProject=mozilla-central&newProject=try&newRevision=a8a5ed746be4ed32923c789a1de0e20cc20f3c3b&originalSignature=edaec66500db21d37602c99daa61ac983f21a6ac&newSignature=edaec66500db21d37602c99daa61ac983f21a6ac&framework=1&selectedTimeRange=172800
Here there is two subpicious results:
  simple.memory.open regressed by 5%
  simple.styleeditor.open also regressed by 5%
This last link was for linux, it is sometime useful to compare to other OSes to see if this is just environmental noise.
Here is the same link but on windows:
https://treeherder.mozilla.org/perf.html#/comparesubtest?originalProject=mozilla-central&newProject=try&newRevision=a8a5ed746be4ed32923c789a1de0e20cc20f3c3b&originalSignature=f687ee347bcbdc55828148af029908049de69b7f&newSignature=f687ee347bcbdc55828148af029908049de69b7f&framework=1&selectedTimeRange=172800
This time, no major regression in subtests.

=> DAMP results look good. I can't state any significative regression, nor improvement.