Closed Bug 1395949 Opened 7 years ago Closed 7 years ago

another speedometer regression on Aug 31

Categories

(Core :: DOM: Core & HTML, defect, P2)

Product:
Core
Component:
DOM: Core & HTML
Version:
32 Branch
Type:
defect

Tracking

()

Status:
RESOLVED INCOMPLETE

People

(Reporter: bkelly, Unassigned)

References

(Blocks 1 open bug)

Details

(Keywords: regression, regressionwindow-wanted) 

To clarify, I don't have time to do the bisect myself today.  Sorry.
Ryan, do we have someone with cycles to bisect the regression range?
Flags: needinfo?(ryanvm)
See Also: → 1022714
It would be great if we could do bug 1022714 so we didn't have to manually bisect regressions here.
This looks like a really noisy test. Other than one obvious outlier, I'm having a hard time seeing the regression?
https://arewefastyet.com/#machine=36&view=single&suite=speedometer-misc&subtest=React-Redux-TodoMVC-Adding100Items-sync&start=1504043590&end=1504280359
Flags: needinfo?(ryanvm)
Ok, maybe its noise.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → INVALID
Well, there is still something regressed I think.  On August 29/30 we were getting better numbers than we are today.  PGO was getting 86-to-88, but not we are getting 85-to-86.

https://arewefastyet.com/#machine=36&view=single&suite=speedometer-misc&subtest=score
Status: RESOLVED → REOPENED
Resolution: INVALID → ---
That chart is ignoring Aug 31 and Sep 1.  See the chart without the end date restriction:

https://arewefastyet.com/#machine=36&view=single&suite=speedometer-misc&subtest=score
Yeah you're right...  So anecdotally for a while the PGO numbers were stable at 84.  Not sure what to make of the fact that they're not quite back to what they were before bug 1395727 was introduced... :-(
I wonder, could it be  bug 1393597.
Looking at the breakdown graphs, maybe things have more variability since August 29/30.  Its really hard to tell, though, given we had a day of regression introducing variability itself.
Summary: another speedometer regression on Aug 31 on React-Redux-TodoMVC-Adding100Items-sync → another speedometer regression on Aug 31
Now that AWFY is updating again its a bit easier to see if there is a stable regression.

Win 8 64-bit (i7-4700HQ, browser) Ion:
  Aug 30: ~113
  Sep 1 to 4: ~110 to 111

Quantum Reference (Windows, browser, x64) Ion:
  Aug 30: ~78
  Sep 1 to 4: ~78 to 79

This is a bit odd in that the slower machine doesn't show any regression, but the faster machine does.  Perhaps whatever changed is not a CPU perf regression but more firefox using idle time less efficiently.

Anyway, its unclear to me if there is anything we can do here.  Maybe we should just WONTFIX this bug and just keep on trying to improve the benchmark in general.
Priority: -- → P1
CCing Jan to see if he has any thoughts.
(In reply to :Ehsan Akhgari (needinfo please, extremely long backlog, Away until Sep 12) from comment #13)
> CCing Jan to see if he has any thoughts.

Not really, offhand. It might be this:

(In reply to Olli Pettay [:smaug] from comment #10)
> I wonder, could it be  bug 1393597.

But that's just a guess and it's also a fix we really want to keep for responsiveness.
Given the change to the scoring it's hard to see a trend here but I'm tempted to WONTFIX since we're still making progress elsewhere. At a minimum, I'm going to mark as P2.
Priority: P1 → P2
I think let's close it for now, unless if someone finds out evidence that something caused a regression somewhere...
Status: REOPENED → RESOLVED
Closed: 7 years ago → 7 years ago
Resolution: --- → INCOMPLETE
Component: DOM → DOM: Core & HTML
You need to log in before you can comment on or make changes to this bug.