1571380 - 20.06% raptor-tp6-apple-firefox loadtime (macosx1014-64-shippable) regression on push 76a5046906d74ade6b8476709867a74e9c80e539 (Tue July 30 2019)

Status: RESOLVED INVALID

(DevTools :: Debugger, defect)

Description

[Marian Raiciof [:marauder]]

Raptor has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=76a5046906d74ade6b8476709867a74e9c80e539

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

20% raptor-tp6-apple-firefox loadtime macosx1014-64-shippable opt 569.87 -> 684.21

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=22195

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a Treeherder page showing the Raptor jobs in a pushlog format.

To learn more about the regressing test(s) or reproducing them, please see: https://wiki.mozilla.org/Performance_sheriffing/Raptor

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Performance_sheriffing/Talos/RegressionBugsHandling

Comment 1

[Jason Laster [:jlast]]

I'm not sure how to investigate this. The patch changes one line of code, which is not run unless the debugger is open. I am not sure where the test is enabled, but the only thing I can think of is that this is taking into account the test harness because in general, we should not be testing firefox performance with devtools open.

Would you be able to measure the change after backing out this patch?
Is there an easy way for me to see the test and run it locally ?

Comment 2

[Marian Raiciof [:marauder]]

Hi Jason,
For now, i did a few more backfills on newer data points to see what's going on.
I'll get back with updates.
Thanks!

Comment 3

[Jason Laster [:jlast]]

Hi Mirian, just following up. Any updates?

Comment 4

[Marian Raiciof [:marauder]]

Hi Jason,

I spoke with Ionut Goldan and we decided to mark this alert as invalid because it is not a clear regression nor an improvement.
The graph being very noisy.

Regards,
Marian