1656175 - 3.79 - 4.29% raptor-tp6-wikipedia-firefox-cold loadtime (linux64-shippable-qr) regression on push 2a3440bf9f3d89d839a8da70a0916c888072dd7b (Tue July 21 2020)

Status: RESOLVED INVALID

(Core :: Gecko Profiler, defect, P2)

Description

[Florin Strugariu [:Bebe]]

Perfherder has detected a raptor performance regression from push 2a3440bf9f3d89d839a8da70a0916c888072dd7b. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

4% raptor-tp6-wikipedia-firefox-cold loadtime linux64-shippable-qr opt webrender 2,061.88 -> 2,150.33
4% raptor-tp6-wikipedia-firefox-cold loadtime linux64-shippable-qr opt webrender 2,064.96 -> 2,143.17

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Comment 1

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Bug 1654106 was only a bug fix that removed crashes from bug 1653350 shortly after it landed. (So bug 1654106 might look like the start of the regression, but only because until then bug 1653350 would have crashed and not been counted in the performance numbers.)

Redirecting to Markus.

Comment 2

[Markus Stange [:mstange]]

(In reply to Gerald Squelart [:gerald] (he/him) from comment #1)

(So bug 1654106 might look like the start of the regression, but only because until then bug 1653350 would have crashed and not been counted in the performance numbers.)

It would not have crashed because the check is only performed in debug builds, and only when the profiler is running.

Comment 3

[Markus Stange [:mstange]]

I think it's impossible that this push caused this regression. Can you double-check the regression range?

Comment 4

[Florin Strugariu [:Bebe]]

on it sorry for the missassigment

Comment 5

[Jens Stutte [:jstutte]]

Hi Florin, do you have an update on the regression range here? Thank you!

Comment 6

[Jens Stutte [:jstutte]]

Hi Gerald, is this really worth S2 ? Thanks for checking!

Comment 7

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Thank you Jens for asking.
I can't remember exactly why I picked S2... I can't really justify it now, so I may have mis-clicked. And in fact looking at the severity descriptions, it doesn't even "block non-critical functionality", so I'll drop it to S4.

Comment 8

[Florin Strugariu [:Bebe]]

My mistake here.

reassedthe alert and it looks like I was wong in my assigment

Sorry for the confusion

== Change summary for alert #26571 (as of Wed, 22 Jul 2020 19:38:19 GMT) ==

Regressions:

4% raptor-tp6-wikipedia-firefox-cold loadtime linux64-shippable-qr opt webrender 2,064.96 -> 2,143.17

Improvements:

10% raptor-tp6-wikipedia-firefox-cold loadtime windows10-64-shippable opt 1,893.08 -> 1,704.25
10% raptor-tp6-wikipedia-firefox-cold loadtime windows7-32-shippable opt 1,948.25 -> 1,758.50
8% raptor-tp6-wikipedia-firefox-cold loadtime windows10-64-shippable-qr opt webrender 1,823.67 -> 1,683.00
7% raptor-tp6-wikipedia-firefox-cold windows10-64-shippable opt 1,651.02 -> 1,527.99
7% raptor-tp6-wikipedia-firefox-cold windows7-32-shippable opt 1,681.69 -> 1,564.29
7% raptor-tp6-wikipedia-firefox-cold fcp macosx1014-64-shippable opt 2,131.00 -> 1,986.17
7% raptor-tp6-wikipedia-firefox-cold fcp windows10-64-shippable opt 1,588.83 -> 1,483.00
6% raptor-tp6-wikipedia-firefox-cold fcp windows7-32-shippable opt 1,614.42 -> 1,516.00
6% raptor-tp6-wikipedia-firefox-cold windows10-64-shippable-qr opt webrender 1,639.58 -> 1,542.67
5% raptor-tp6-wikipedia-firefox-cold fcp linux64-shippable-qr opt webrender 2,041.83 -> 1,935.33

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=26571