1750521 - 4.06 - 3.43% damp custom.jsdebugger.stepIn.DAMP / damp custom.jsdebugger.stepIn.DAMP (Linux) regression on Fri January 14 2022

Status: RESOLVED WONTFIX

(DevTools :: General, defect)

Description

[Julian Descottes [:jdescottes]]

Perfherder has detected a devtools performance regression from push a9d7a52dfa99df81a213caaa7cbc2d723a1d3ec0. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
4%	damp custom.jsdebugger.stepIn.DAMP	linux1804-64-shippable-qr	e10s fission stylo webrender-sw	725.60 -> 755.03
3%	damp custom.jsdebugger.stepIn.DAMP	linux1804-64-shippable-qr	e10s fission stylo webrender	733.45 -> 758.61

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Comment 1

[Julian Descottes [:jdescottes]]

Either from Bug 1569859, Bug 1737985 or Bug 1737972

Comment 2

[Julian Descottes [:jdescottes]]

DAMP try pushes removing the lodash patches:

• baseline https://treeherder.mozilla.org/jobs?repo=try&revision=ecda38778e4dbb30dedbfa0a4c5b270039313efa
• removing 1 patch https://treeherder.mozilla.org/jobs?repo=try&revision=5f71f09f6d4616d1a98aeecff768333104041a99
• removing 2 patches https://treeherder.mozilla.org/jobs?repo=try&revision=413101537046fe280b5a04d8f0a249a6dbfee9fa

If all three have the same performance then, it comes from Bug 1569859

Also note that the regression is Linux only

Comment 3

[Julian Descottes [:jdescottes]]

Regression from Bug 1737985

Comment 4

[Julian Descottes [:jdescottes]]

Regression seems to be from Bug 1737985 - Remove lodash sortBy usage

Given that it's 4% only on one test and linux only, I'm happy to WONTFIX, unless you can think about obvious improvements?

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1737985

Comment 6

[Nicolas Chevobbe [:nchevobbe] [PTO - back January 24]]

(In reply to Julian Descottes [:jdescottes] from comment #4)

Regression seems to be from Bug 1737985 - Remove lodash sortBy usage

Given that it's 4% only on one test and linux only, I'm happy to WONTFIX, unless you can think about obvious improvements?

Looking at the patch it's not obvious what could cause that. Maybe lodash#sortBy have early returns if the passed collection is empty/only has a single element, but I'm not sure this would have such a big impact? One thing that I did in the patch in different place was to clone the passed array to return a new sorted version without mutating the old one. Maybe those created arrays for 0/1-length arrays are causing the regression?
I'll try to have early return in such case, and see if that brings any improvement.

Comment 7

[Nicolas Chevobbe [:nchevobbe] [PTO - back January 24]]

mh, my attempt doesn't seem to be successful https://treeherder.mozilla.org/perfherder/comparesubtest?originalProject=try&newProject=try&newRevision=e7fbd991fde75e35e765824af0717888ed15ded8&originalSignature=3130994&newSignature=3130994&framework=12&originalRevision=cc1f9758664d48c893be53a23d6b940b75a190dd&page=1&filter=stepIn

Comment 8

[Nicolas Chevobbe [:nchevobbe] [PTO - back January 24]]

Since this was not big and linux specific, let's close this