1779868 - Regression on AWFY around the July13 mark on speedometer-Elm-TodoMVC* benchmarks

Status: RESOLVED WONTFIX

(Core :: JavaScript Engine, defect, P3)

Description

[Mayank Bansal]

Go to AWFY : https://arewefastyet.com/win10/benchmarks/raptor-desktop-speedometer?numDays=60

Look at the following benchmarks :
Elm-TodoMVC/DeletingItems
Elm-TodoMVC/DeletingItems/Sync
Elm-TodoMVC
Elm-TodoMVC/Adding100Items/Sync
Elm-TodoMVC/CompletingAllItems/Sync
React-Redux-TodoMVC/CompletingAllItems/Sync
React-Redux-TodoMVC/DeletingItems

Unable to find a regression range.

Comment 1

[Nicolas B. Pierron [:nbp]]

The regression range is:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=445cd004a4228df0a33898514edb3be98038b2fb&tochange=b363ae1f1a6d62e907818d48d616c53b494b40f0

Except that there is a merge of mozilla-central in the middle that I do not know how to interpret as it exhibit bi-modal behaviour and does not seems to show the slow down on the fast part.

If the regression does not exists before the merge, this would suggest that the regression is either coming from the merge commit or in the range:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=55ce1e6ba55cbb1436a3b7ecc5b5f8c9d629aa39&tochange=b363ae1f1a6d62e907818d48d616c53b494b40f0

Tom, Emilio, Jan, any idea what changes might have caused this regression?

Comment 2

[Emilio Cobos Álvarez (:emilio)]

My patch is an extension-API only patch, so unlikely to cause it I'd say.

Comment 3

[Jan de Mooij [:jandem]]

Bug 1779378 is a fix for a GC issue, but it's possible it changed GC timing, for example moving a GC to this particular test.

Looking at the Elm graph, however, it seems there was an improvement around the time bug 1529337 was backed out?

Comment 4

[Tom Schuster]

Very interesting. Could we check if the tests use a Content-Security-Policy?

Comment 5

[Nicolas B. Pierron [:nbp]]

(In reply to Jan de Mooij [:jandem] from comment #3)

Looking at the Elm graph, however, it seems there was an improvement around the time bug 1529337 was backed out?

This does not seems to be the case with all Elm benchmarks.

Comment 6

[Alex Finder|PTO, back on Nov 25th]

There was an update to Speedometer 2.1 that landed at the end of May. I think it's unlikely that it caused the regressions, since those started appearing around July 13th. There is a similar issue reported on this topic that was closed as won't fix. There is also an upcoming ticket that will change how the Speedometer benchmark scores are extracted and computed, so that might change the results when it lands.

Comment 7

[Mayank Bansal]

Should we close this bug?

Comment 8

[Iain Ireland [:iain]]

Note that Elm was removed from Speedometer 3.

Comment 9

[Alex Finder|PTO, back on Nov 25th]

(In reply to Mayank Bansal from comment #7)

Should we close this bug?

Checked on the Performance Tests channel, closing it as WONTFIX.