1672122 - 4.21 - 4.89% tp5o_webext (windows7-32-shippable) regression on push 735654990df11f4470b68ebe716336a476fa0a12 (Tue October 6 2020)

Status: RESOLVED INVALID

(Firefox :: Security, defect)

Description

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Perfherder has detected a talos performance regression from push 735654990df11f4470b68ebe716336a476fa0a12. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
5%	tp5o_webext		windows7-32-shippable	e10s stylo	355.07 -> 372.43
4%	tp5o_webext		windows7-32-shippable	e10s stylo	353.18 -> 368.04

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Comment 1

[Valentin Gosu [:valentin]]

(In reply to Alexandru Ionescu (needinfo me) [:alexandrui] from comment #0)

Perfherder has detected a talos performance regression from push 735654990df11f4470b68ebe716336a476fa0a12. As author of one of the patches included in that push, we need your help to address this regression.

I think the range is wrong.

https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=19bc84a9ed83efac55c260d5437c3426eeb36b49&tochange=735654990df11f4470b68ebe716336a476fa0a12

We'll discuss our options.

Comment 2

[Florin Strugariu [:Bebe]]

https://treeherder.mozilla.org/perf.html#/graphs?highlightAlerts=1&series=autoland,1927424,1,1&timerange=2592000&zoom=1600676284000,1603277183998,273.55029892043444,414.8276438164585

As this graph proves this is a infrastructure issue
:Alex can you properly update this bug

Comment 3

[(inactive) Nihanth Subramanya [:nhnt11]]

(In reply to Florin Strugariu [:Bebe] (needinfo me) from comment #2)

https://treeherder.mozilla.org/perf.html#/graphs?highlightAlerts=1&series=autoland,1927424,1,1&timerange=2592000&zoom=1600676284000,1603277183998,273.55029892043444,414.8276438164585

As this graph proves this is a infrastructure issue
:Alex can you properly update this bug

:Bebe, could you add a comment describing how exactly an infrastructure issue is proved by this and what the issue is? It's a little unclear to me what this graph shows other than a regression around Oct 5/6.

Comment 4

[(inactive) Nihanth Subramanya [:nhnt11]]

I think I figured it out. On the graph, several of the points around Oct 5 are retriggers - and they show that retriggering a run from a revision prior to the suspect patch, the regression still appears. Also, the spike around Sep 27 are retriggers from today, showing that those old runs will also show the regression if they are run again today.

So, resolving this bug as invalid seems to absorb this regression - whatever the reason - silently. Should we not keep this on file to track the infra issue until it's resolved or at least explained? Is that being tracked elsewhere?

Comment 5

[Alexandru Ionescu (needinfo me) [:alexandrui]]

I am having a conversation with jmaher on releng about this.