1575546 - 12.95 - 30.16% raptor-tp6-imdb-firefox / raptor-tp6-instagram-firefox (macosx1014-64-shippable) regression on push abe0d15df80fa37934832fb3ed3cdaa9f9687b3c (Sat August 3 2019)

Status: RESOLVED INVALID

(Core :: CSS Parsing and Computation, defect)

Description

[Marian Raiciof [:marauder]]

Raptor has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=abe0d15df80fa37934832fb3ed3cdaa9f9687b3c

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

30% raptor-tp6-instagram-firefox fcp macosx1014-64-shippable opt 225.29 -> 293.25
22% raptor-tp6-instagram-firefox macosx1014-64-shippable opt 336.70 -> 411.50
14% raptor-tp6-imdb-firefox macosx1014-64-shippable opt 594.33 -> 680.24
13% raptor-tp6-imdb-firefox fcp macosx1014-64-shippable opt 503.14 -> 568.29

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=22483

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a Treeherder page showing the Raptor jobs in a pushlog format.

To learn more about the regressing test(s) or reproducing them, please see: https://wiki.mozilla.org/TestEngineering/Performance/Raptor

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/TestEngineering/Performance/Talos/RegressionBugsHandling

Comment 1

[Emilio Cobos Álvarez (:emilio)]

That is actually... surprising, since the patch is an optimization in other benchmarks which I measured locally. I'll investigate.

Comment 2

[Emilio Cobos Álvarez (:emilio)]

Hmm, are the regressions macos only? That's extremely surprising given the patch doesn't touch any mac-specific code-path.

Comment 3

[Emilio Cobos Álvarez (:emilio)]

Hmm, so am I misinterpreting the graph?

I don't see it changing meaningfully before or after my patch, what am I missing, can you explain what the regression numbers are supposed to mean?

This test seems bi-modal, btw, we should probably figure out what's causing that to make the test more reliable...

Comment 4

[Emilio Cobos Álvarez (:emilio)]

Waiting for comment 2 / comment 3. Please ni? back if there's a reply and I forget.

I need to try run this on a local machine next.

Comment 5

[Marian Raiciof [:marauder]]

Hi Emilio,
I asked for Ionut Goldan's opinion and i will mark the ticket and the alert as invalid because of that bi-modal pattern.
Thanks!

Comment 6

[Emilio Cobos Álvarez (:emilio)]

Great, thank you.