Open Bug 1970893 Opened 6 months ago Updated 4 months ago

Very large increase in variance on browsertime cold pageload tests on macos 1470

Categories

(Testing :: Raptor, defect, P2)

Product:
Testing
Component:
Raptor
Type:
defect

Tracking

(Not tracked)

People

(Reporter: sparky, Unassigned)

References

Details

(Whiteboard: [fxp])

See the graphs here: https://arewefastyet.com/mac1470/cold-page-load/overview?numDays=365

Over the last few months the variance in our pageload tests has increased a lot. Interestingly, our warm pageloads don't seem to be affected as much by this issue: https://arewefastyet.com/mac1470/warm-page-load/overview?numDays=365

This is likely to be related to bug 1967525, but in case the fixes that occur there don't resolve the issue, we'll use this bug to dig into it further.

Summary: Very large increase in variance on browsertime pageload tests on macos 1470 cold pageload tests → Very large increase in variance on browsertime cold pageload tests on macos 1470
See Also: → 1970956

The changes in bug 1967525 did not completely fix this issue. We'll need to look into it further. The variance has decreased but there are still a lot of outliers when comparing against pre-February data.

You need to log in before you can comment on or make changes to this bug.