1815168 - webconsole reload DAMP tests are broken (regression on Wed February 1 2023)

Status: RESOLVED FIXED

(DevTools :: Console, defect, P2)

Description

[Julian Descottes [:jdescottes]]

Perfherder has detected a devtools performance regression from push 371e407a353d5c96d2dc553b1bef77c6a6e4df12. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
5%	damp console.bulklog	windows10-64-shippable-qr	e10s fission stylo webrender	1,021.64 -> 1,073.32
5%	damp console.bulklog	windows10-64-shippable-qr	e10s fission stylo webrender-sw	1,018.32 -> 1,068.84
4%	damp console.bulklog	linux1804-64-shippable-qr	e10s fission stylo webrender	1,164.45 -> 1,215.24
3%	damp custom.webconsole.reload.DAMP	windows10-64-shippable-qr	e10s fission stylo webrender	9,215.11 -> 9,513.99
3%	damp console.bulklog	linux1804-64-shippable-qr	e10s fission stylo webrender-sw	1,165.86 -> 1,203.54
3%	damp custom.webconsole.reload.DAMP	windows10-64-shippable-qr	e10s fission stylo webrender-sw	9,182.03 -> 9,470.18

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)
100%	damp complicated.webconsole.reload.DAMP	windows10-64-shippable-qr	e10s fission stylo webrender	1,624.21 -> 0.54
100%	damp complicated.webconsole.reload.DAMP	windows10-64-shippable-qr	e10s fission stylo webrender-sw	1,620.46 -> 0.54
100%	damp complicated.webconsole.reload.DAMP	linux1804-64-shippable-qr	e10s fission stylo webrender	1,461.07 -> 0.56
100%	damp complicated.webconsole.reload.DAMP	linux1804-64-shippable-qr	e10s fission stylo webrender-sw	1,424.75 -> 0.57
100%	damp complicated.webconsole.reload.DAMP	macosx1015-64-shippable-qr	e10s fission stylo webrender	1,412.93 -> 0.50
...	...	...	...	...
13%	damp simple.webconsole.close.DAMP	macosx1015-64-shippable-qr	e10s fission stylo webrender-sw	8.84 -> 7.72

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

For more information on performance sheriffing please see our FAQ.

Comment 1

[Julian Descottes [:jdescottes]]

Hi Hubert, I see Bug 1810992 in the pushlog with DAMP changes, so it's very likely to be related. Most webconsole reload tests are incorrect and measure timings close to 0.

Can you take a look?

Comment 2

[Hubert Boma Manilla (:bomsy)]

Opps! Thanks for the ping. Looking into it now.

Comment 3

[BugBot [:suhaib / :marco]]

The Bugbug bot thinks this bug should belong to the 'DevTools::Console' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 4

[Hubert Boma Manilla (:bomsy)]

Attachment: Bug 1815168 - [devtools] Fix regression on webconsole damp tests r=jdescottes

Wait for the observer to be created before checking the predicate, so the results are accurate.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/887e96e7fa0e
[devtools] Fix regression on webconsole damp tests r=jdescottes,perftest-reviewers,kshampur

Comment 6

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/887e96e7fa0e