43.58% raptor-tp6-microsoft-firefox (osx-10-10) regression on push aa43ce6e3442f05ec14243a2f1311012bc6b51cb (Tue Jan 15 2019)
Categories
(Core :: Gecko Profiler, defect)
Tracking
()
People
(Reporter: igoldan, Unassigned)
References
Details
(Keywords: perf, regression)
Raptor has detected a Firefox performance regression from push:
As author of one of the patches included in that push, we need your help to address this regression.
Regressions:
44% raptor-tp6-microsoft-firefox osx-10-10 opt 1,844.07 -> 2,647.70
You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=18750
On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a Treeherder page showing the Raptor jobs in a pushlog format.
To learn more about the regressing test(s) or reproducing them, please see: https://wiki.mozilla.org/Performance_sheriffing/Raptor
*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***
Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Performance_sheriffing/Talos/RegressionBugsHandling
|
Reporter | |
Updated•5 years ago
|
|
|
Reporter | |
Updated•5 years ago
|
|
|
Reporter | |
Comment 1•5 years ago
|
||
The patch contained 5 separate bugs. :rjesup out of the bugs listed bellow, which one is more related to this perf regression?
|
||
Comment 2•5 years ago
|
||
I would suspect bug 1514513 as looking at the subtests, the regression only affected ttfi: https://treeherder.mozilla.org/perf.html#/graphs?series=autoland,1825059,1,10&series=autoland,1825060,1,10&series=autoland,1825061,1,10
|
|
Reporter | |
Comment 3•5 years ago
|
||
(In reply to Dave Hunt [:davehunt] [he/him] ⌚️UTC from comment #2)
I would suspect bug 1514513 as looking at the subtests, the regression only affected ttfi: https://treeherder.mozilla.org/perf.html#/graphs?series=autoland,1825059,1,10&series=autoland,1825060,1,10&series=autoland,1825061,1,10
Good point!
|
||
Comment 4•5 years ago
|
||
TTFI is highly variable due to how it works, and we don't alert on it - but especially in a test without loadtime, TTFI has a major impact on a geomean'd result from all the subtests - as happened here.
We likely should simply ignore this.
|
|
Reporter | |
Comment 5•5 years ago
|
||
(In reply to Randell Jesup [:jesup] from comment #4)
TTFI is highly variable due to how it works, and we don't alert on it - but especially in a test without loadtime, TTFI has a major impact on a geomean'd result from all the subtests - as happened here.
We likely should simply ignore this.
Are you proposing us to avoid outputting TTFI values in absence of loadtime values? This sounds like a harness update for raptor-tp6.
|
|
||
Comment 6•5 years ago
|
||
We've turned off TTFI on raptor for now. I still want to know why TTFI reacted this way
Description
•