DAMP still miss performances regressions on inspector.open

NEW
Unassigned

Status

()

Firefox
Developer Tools
P3
normal
4 months ago
4 months ago

People

(Reporter: ochameau, Unassigned)

Tracking

(Blocks: 1 bug)

unspecified
Points:
---

Firefox Tracking Flags

(firefox57 fix-optional)

Details

(Reporter)

Description

4 months ago
Even with bug 1396539 and bug 1394804, we still don't catch significant improvements being made to the inspector.

In the following talos results I ran a first reference build including bug 1396539 and bug 1394804:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=5689dc70e2cc6104b87678e3d609d860cfa531af

Then I pushed another one, also including this two bugs, plus a bunch of recent optimizations made to the inspector: bug 1396600, bug 1397343 and bug 1397330:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=280605daeddeeb64edf33a0bdcb74cd5ee2f493a

This end up reporting this performance numbers:
https://treeherder.mozilla.org/perf.html#/comparesubtest?originalProject=try&originalRevision=5689dc70e2cc&newProject=try&newRevision=280605daeddeeb64edf33a0bdcb74cd5ee2f493a&originalSignature=edaec66500db21d37602c99daa61ac983f21a6ac&newSignature=edaec66500db21d37602c99daa61ac983f21a6ac&filter=inspector.open&framework=1

No alert, confidence is always low.
But the main issue is that it only report regressions instead of improvements!
The timing of the second build are all higher.

I have no clue why the results are wrong. May be we miss something similar to bug 1396539??
(Reporter)

Comment 1

4 months ago
Or could it be related to bug 1394804 comment 30 and the numbers are not as good as before?
If I understand this correctly, we only take 1 run for cold and 5 runs for warm into account.

Updated

4 months ago
Priority: -- → P3

Updated

4 months ago
status-firefox57: --- → fix-optional
You need to log in before you can comment on or make changes to this bug.