1680773 - 14.08% tp5o responsiveness (linux64-shippable) regression on push 4fc932c5974e567ed1568bd3cba75345fb9c36c3 (Wed December 2 2020)

Status: RESOLVED INVALID

(Core :: JavaScript Engine: JIT, defect)

Description

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Perfherder has detected a talos performance regression from push 4fc932c5974e567ed1568bd3cba75345fb9c36c3. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
14%	tp5o	responsiveness	linux64-shippable	e10s stylo	0.87 -> 0.99

Improvements:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
5%	kraken		windows10-64-shippable	e10s stylo	1,410.10 -> 1,341.84

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Comment 1

[Florin Strugariu [:Bebe]]

== Change summary for alert #27956 (as of Thu, 03 Dec 2020 22:57:56 GMT) ==

Improvements:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
4%	raptor-tp6-office-firefox-cold	loadtime	windows10-64-shippable-qr	webrender	2,261.46 -> 2,172.33

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=27956

Comment 2

[Florin Strugariu [:Bebe]]

== Change summary for alert #27958 (as of Thu, 03 Dec 2020 23:32:50 GMT) ==

Regressions:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
10%	twitch	FirstVisualChange	linux64-shippable-qr	nocondprof warm webrender	695.00 -> 766.67
8%	twitch	SpeedIndex	linux64-shippable-qr	nocondprof warm webrender	754.50 -> 818.00
8%	twitch	PerceptualSpeedIndex	linux64-shippable-qr	nocondprof warm webrender	806.50 -> 870.83
3%	twitch		linux64-shippable-qr	nocondprof warm webrender	194.66 -> 200.18

Improvements:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
55%	google-sheets	LastVisualChange	linux64-shippable-qr	nocondprof warm webrender	5,268.33 -> 2,350.00
4%	speedometer		android-hw-g5-7-0-arm7-api-16-shippable	geckoview nocondprof webrender	10.10 -> 10.48

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=27958

Comment 3

[Jan de Mooij [:jandem]]

I'm going to bisect the patches I pushed, on Try, for the tp5o-responsiveness regression. Hopefully that will narrow it down a bit and tell us more.

Comment 4

[Jan de Mooij [:jandem]]

I didn't see any regressions on Try so I looked at the autoland data. Turns out the alert is wrong, it's based on one data point that was an outlier. Retriggers and backfills point to push e4c8bc7feaa6 instead.

See this graph

Can you move and change this? Thanks.

Comment 5

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Makes sense. Thanks Jan.