Closed Bug 1570893 Opened 2 years ago Closed 2 years ago

15.63% tp5n nonmain_startup_fileio (windows10-64-shippable) regression

Categories

(Testing :: General, defect, P3)

x86_64
Windows 10
defect

Tracking

(firefox69 wontfix, firefox70 wontfix, firefox71 wontfix)

RESOLVED WONTFIX
Tracking Status
firefox69 --- wontfix
firefox70 --- wontfix
firefox71 --- wontfix

People

(Reporter: igoldan, Unassigned)

References

Details

(4 keywords)

Talos has detected a Firefox performance regression around this push:

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=b1d382dd18da4460ac88849c12049fa6d1a4cee2

Regressions:

16% tp5n nonmain_startup_fileio windows10-64-shippable opt e10s stylo 1,238,800.65 -> 1,432,454.33

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=21373

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Performance_sheriffing/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Performance_sheriffing/Talos/Running

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Performance_sheriffing/Talos/RegressionBugsHandling

This is a really tricky situation. Had 2 explanations for this so far, but none held.

It's possible this is some sort of performance intermittent regression.
My new explanation now: between May 29 and June 1 a Windows infra change happened. Now, why am I not able to reproduce this on older commits? It's possible the infra change problem kicks in because of a Bugzilla bug that's also related to Windows.

:aklotz could you look into this weird situation? I'm pretty much clueless at this point. The test outputs bimodal data at this moment, which is not ideal.

Flags: needinfo?(aklotz)
Priority: -- → P3

(In reply to Ionuț Goldan [:igoldan], Performance Sheriff from comment #1)

This is a really tricky situation. Had 2 explanations for this so far, but none held.

It's possible this is some sort of performance intermittent regression.
My new explanation now: between May 29 and June 1 a Windows infra change happened. Now, why am I not able to reproduce this on older commits? It's possible the infra change problem kicks in because of a Bugzilla bug that's also related to Windows.

:aklotz could you look into this weird situation? I'm pretty much clueless at this point. The test outputs bimodal data at this moment, which is not ideal.

Unfortunately I don't really have any ideas as to what is going on. The good news is that, even though that alert shows a spike in startup I/O, it is not on the main thread. I'd be more concerned if the I/O was for the main thread, but since it isn't, and in the absence of any other corroborating evidence of negative impact, I'm inclined to say that we should just eat this regression.

Flags: needinfo?(aklotz)
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.