Closed Bug 1147379 Opened 7 years ago Closed 5 years ago

2-4% Win7/Linux* kraken regression on Mozilla-Inbound (v.39) on March 19, 2015 from push b79cddbe7de8

Categories

(Core :: JavaScript Engine: JIT, defect)

defect
Not set
normal

Tracking

()

RESOLVED WONTFIX

People

(Reporter: jmaher, Unassigned)

References

Details

(Keywords: perf, regression, Whiteboard: [talos_regression])

Talos has detected a Firefox performance regression from your commit b79cddbe7de8 in bug 1143847.  We need you to address this regression.

This is a list of all known regressions and improvements related to your bug:
http://alertmanager.allizom.org:8080/alerts.html?rev=b79cddbe7de8&showAll=1

On the page above you can see Talos alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test, please see: https://wiki.mozilla.org/Buildbot/Talos/Tests#kraken

Reproducing and debugging the regression:
If you would like to re-run this Talos test on a potential fix, use try with the following syntax:
try: -b o -p win32 -u none -t dromaeojs  # add "mozharness: --spsProfile" to generate profile data

To run the test locally and do a more in-depth investigation, first set up a local Talos environment:
https://wiki.mozilla.org/Buildbot/Talos/Running#Running_locally_-_Source_Code

Then run the following command from the directory where you set up Talos:
talos --develop -e <path>/firefox -a kraken

Making a decision:
As the patch author we need your feedback to help us handle this regression.
*** Please let us know your plans by Monday, or the offending patch will be backed out! ***

Our wiki page oulines the common responses and expectations:
https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling
:jandem, can you take a look at this regression?  I suspect this is more widespread on followup checkins on the 20th, I am awaiting some backfilling and retriggering before making that certain.
Flags: needinfo?(jdemooij)
and more regressions on the next day:
http://alertmanager.allizom.org:8080/alerts.html?rev=847f9159b3e9&showAll=1&testIndex=0&platIndex=0

This is linux32 and linux64 regressions.
Summary: 2% Win7 kraken regression on Mozilla-Inbound (v.39) on March 19, 2015 from push b79cddbe7de8 → 2-4% Win7/Linux* kraken regression on Mozilla-Inbound (v.39) on March 19, 2015 from push b79cddbe7de8
I'm aware of this regression and I just landed a patch (bug 1142669 part 6) that should fix it, at least for the most part. Fingers crossed...
this is also a ~4% win8 regression for kraken.
awesome, I look forward to the improvements in a few hours!
I confirmed today that these are resolved, thanks!
Status: NEW → RESOLVED
Closed: 7 years ago
Flags: needinfo?(jdemooij)
Resolution: --- → FIXED
after the uplift we have some smaller kraken regressions, specifically on win8 (2.18%) and linux64 (2.5%):

win8: http://graphs.mozilla.org/graph.html#tests=[[232,52,31],[232,64,31],[232,63,31]]&sel=1425644878421.2512,1428038489070.1697,1180.3278688524588,1770.4918032786884&displayrange=30&datatype=geo

linux64: http://graphs.mozilla.org/graph.html#tests=[[232,52,35],[232,63,35]]&sel=1424057977172.599,1427816592813.198,1081.967213114754,1672.1311475409834&displayrange=90&datatype=geo

while we had a much larger regression to start with, it appears that we fixed the majority of it, but the graphs show that we never reset back to the old levels for these two platforms.

:jandem, does this make sense?  is there anything you can think of that you can reasonably do to fix this?  should we accept these as is?
Status: RESOLVED → REOPENED
Flags: needinfo?(jdemooij)
Resolution: FIXED → ---
(In reply to Joel Maher (:jmaher) from comment #7)
> after the uplift we have some smaller kraken regressions, specifically on
> win8 (2.18%) and linux64 (2.5%):

This could also be from other patches right? Or do you think the m-c regression turned out to be similar after we fixed the biggest problem?

> :jandem, does this make sense?  is there anything you can think of that you
> can reasonably do to fix this?  should we accept these as is?

Not really, on AWFY the regression turned out to be pretty small and the patch was a big win on benchmarks like Octane (multiple tests improved).
Flags: needinfo?(jdemooij)
(In reply to Jan de Mooij [:jandem] from comment #8)
> Not really

Oops, this was a reply to the second question, sorry for the confusion.
it is possible these are from other patches, maybe a couple really small regressions have a cumulative effect.  Knowing these are small, I am fine accepting these as wontfix.  I like to hear about octane wins :)

I picked this bug because as I looked at the trend, there was a small regression left after the fix here.  It was a lot more obvious with a few more data is data to see the trend.
Seems like this had other improvements and we decided to take this kraken regression. Marking as wontfix.
Status: REOPENED → RESOLVED
Closed: 7 years ago5 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.