1849432 - Fix unexpected baseline changes for the debugger damp tests

Status: RESOLVED FIXED

(DevTools :: Debugger, defect)

Description

[Hubert Boma Manilla (:bomsy)]

The changes from Bug 1844204, unexpectedly changed the baseline for most of the debugger damp tests.

Lets try to figure out and fix the issue.

Comment 1

[BugBot [:suhaib / :marco]]

Set release status flags based on info from the regressing bug 1844204

Comment 2

[Hubert Boma Manilla (:bomsy)]

Attachment: Bug 1849432 - [devtools] Stop adding main.js bundle to the index.html file r=#devtools-reviewers

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/81c3a95411ab
[devtools] Stop adding main.js bundle to the index.html file r=devtools-reviewers,perftest-reviewers,jdescottes,AlexandruIonescu

Comment 4

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/81c3a95411ab

Comment 5

[Julian Descottes [:jdescottes]]

Confirmed we are back to baseline

== Change summary for alert #39374 (as of Wed, 23 Aug 2023 21:15:17 GMT) ==

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
44%	damp custom.jsdebugger.stepOver.DAMP	windows10-64-shippable-qr	e10s fission stylo webrender	201.07 -> 288.89
33%	damp custom.jsdebugger.stepOver.DAMP	windows10-64-shippable-qr	e10s fission stylo webrender-sw	203.00 -> 270.23
27%	damp custom.jsdebugger.reload.DAMP	macosx1015-64-shippable-qr	e10s fission stylo webrender	640.28 -> 811.04
25%	damp custom.jsdebugger.reload.DAMP	macosx1015-64-shippable-qr	e10s fission stylo webrender-sw	653.68 -> 819.57
8%	damp custom.jsdebugger.reload.DAMP	windows10-64-shippable-qr	e10s fission stylo webrender-sw	797.78 -> 859.52
7%	damp custom.jsdebugger.reload.DAMP	windows10-64-shippable-qr	e10s fission stylo webrender	796.41 -> 850.38

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)
31%	damp custom.jsdebugger.pause.DAMP	macosx1015-64-shippable-qr	e10s fission stylo webrender-sw	482.66 -> 335.35
27%	damp custom.jsdebugger.pause.DAMP	macosx1015-64-shippable-qr	e10s fission stylo webrender	463.75 -> 338.20
3%	damp custom.jsdebugger.adding-sources.DAMP	windows10-64-shippable-qr	e10s fission stylo webrender-sw	1,204.85 -> 1,173.53
2%	damp custom.jsdebugger.adding-sources.DAMP	windows10-64-shippable-qr	e10s fission stylo webrender	1,208.22 -> 1,180.60

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=39374