Speedometer AWFY score regressed on Sept 14

RESOLVED DUPLICATE of bug 1392841

Status

()

Core
General
RESOLVED DUPLICATE of bug 1392841
9 months ago
9 months ago

People

(Reporter: bkelly, Unassigned)

Tracking

(Blocks: 1 bug)

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [qf])

Attachments

(1 attachment)

(Reporter)

Description

9 months ago
Take a look at this graph:

https://arewefastyet.com/#machine=36&view=single&suite=speedometer-misc&subtest=score&start=1505204633&end=1505440147

While there is some variation it seems starting Sep 14 we are hovering around a new regressed mean.  Seems anywhere from a 3% to 5% regreesion.
(Reporter)

Comment 1

9 months ago
Disable all data sets except "Firefox (Ion)" to get a better view of the issue.

Comment 2

9 months ago
pgo is the one we should be looking at. I see 40 as the most recent result and Sept 12 had 40 too.
But indeed, non-pgo looks worrisome, however the changelog doesn't seem to have any interesting changes.
(Reporter)

Comment 3

9 months ago
(In reply to Olli Pettay [:smaug] (pto Sep 14-15) from comment #2)
> pgo is the one we should be looking at. I see 40 as the most recent result
> and Sept 12 had 40 too.
> But indeed, non-pgo looks worrisome, however the changelog doesn't seem to
> have any interesting changes.

We don't get enough samples for PGO to really tell much of anything.  Also, I believe its results lag non-PGO in the graph.  (The x-axis is the date the test ran, not the date of the commit its testing AFAIK.)
(Reporter)

Comment 4

9 months ago
Created attachment 8908681 [details]
Screenshot 2017-09-15 12.23.45.png

Also, look at the PGO samples in this chart.  It certainly look like an upward trend to me.

The new AWFY geometric mean scoring really makes it hard to see this stuff in our default AWFY view.  The scale is all wrong now.
(Reporter)

Comment 5

9 months ago
Seems like increased spikes across a lot of test cases.  On machine 36 it shows up the best in:

  speedometer-misc-Vanilla-ES2015-TodoMVC-Adding100Items-sync

https://arewefastyet.com/#machine=36&view=single&suite=speedometer-misc&subtest=Vanilla-ES2015-TodoMVC-Adding100Items-sync

Updated

9 months ago
Blocks: 1339557

Comment 6

9 months ago
This is the regression caused by bug 1392841 which was backed out.
Status: NEW → RESOLVED
Last Resolved: 9 months ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1392841
You need to log in before you can comment on or make changes to this bug.