Closed Bug 1725384 Opened 2 months ago Closed 20 days ago

16.89% wikia SpeedIndex (Linux) regression on Mon August 2 2021

Categories

(Core :: DOM: Navigation, defect, P1)

Firefox 93
defect

Tracking

()

RESOLVED FIXED
94 Branch
Fission Milestone MVP
Tracking Status
firefox-esr78 --- unaffected
firefox-esr91 --- unaffected
firefox91 --- unaffected
firefox92 --- wontfix
firefox93 --- wontfix
firefox94 --- fixed

People

(Reporter: davehunt, Assigned: emilio)

References

(Regression)

Details

(Keywords: perf, perf-alert, regression, Whiteboard: [fission-perf])

Attachments

(2 files)

Perfherder has detected a browsertime performance regression from push afcdd8e3c67656f49be753e0cf04f18838639543. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio Test Platform Options Absolute values (old vs new)
17% wikia SpeedIndex linux1804-64-shippable-qr fission warm webrender 418.50 -> 489.17

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

:emilio could one of your patches have caused this regression?

Flags: needinfo?(emilio)

There were other regressions as well:
warm: 25% for youtube, also bing, fandom, gsearch, gslides, imdb, linkedin, microsoft. A few possible cold-load regressions as well - buzzfeed, microsoft, netflix, wikia, youtube

Fission Milestone: --- → MVP
Whiteboard: [fission-perf]
Regressed by: 1677324
Component: Performance → DOM: Navigation
Product: Testing → Core
Version: unspecified → Firefox 93

Netflix cold definitely regressed, and the apparent regression point strongly implies Emelio's patch

Tentatively assigning this bug to Emilio, since this bug looks like a regression from his grey flash fix (bug 1677324).

On AWFY, I see a warm load regression blip on Aug 3 for some sites (e.g. Amazon and Instagram), but the only persistent regressions I see since Aug 3 are Google Slides, Wikia, and YouTube:

https://arewefastyet.com/linux64/warm-page-load/overview?numDays=30&series=Firefox,Firefox-Fission

Assignee: nobody → emilio
Priority: -- → P1

Yeah, I'm actively looking into this.

So here's a try run as a baseline ("bad"):

And here's one with my patch effectively reverted ("good"):

Here's the perherder comparison ("good" vs. "bad"):

Which does show at least a huge regression in imgur FirstVisualChange warm, and the smaller Wikia SpeedIndex warm regression in comment 0.

I'm waiting for profiles for the wikia thing (because I accidentally requested them on windows, but the regression seems Linux-specific for some reason), so I decided to look at the imgur regression. But I'm confused, because if I download the Linux imgur results, I expect the "good" results to have a FirstVisualChange of ~80, but I don't see any video with that, all the videos under both results here have a "First Visual Change" of ~440. I've been running this:

for video in imgur-good/browsertime-results/imgur/pages/imgur_com/gallery/m5tYJL6/data/video/*.mp4; do echo $video; ~/src/moz/gecko-4/tools/browsertime/node_modules/browsertime/browsertime/visualmetrics.py -i $video 2>/dev/null; echo ""; done

Are those videos somehow only for cold loads? Is there an error in that comparison I linked above somehow?

Flags: needinfo?(rjesup)

This shouldn't change behavior, but cleans up the code (some of the conditions
were from the original patch from Nika).

Answered in matrix; we really need dave hunt or sparky to answer that. You can look at some site that shows a major difference in warmload from cold to verify

Flags: needinfo?(rjesup)
Flags: needinfo?(emilio)
Status: NEW → RESOLVED
Closed: 2 months ago
Resolution: --- → FIXED
Target Milestone: --- → 93 Branch

Too late for Fx92. Given that bug 1677324 is a Fission M8 bug, I guess we're going to have to live with it for a cycle.

Status: REOPENED → ASSIGNED
Target Milestone: 93 Branch → ---
Depends on: 1731132

Bug 1731132 should've helped here. Other than this kind of tweak I'm out of ideas.

Flags: needinfo?(emilio)

Wikia numbers look now rather good
https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&series=mozilla-central,3422675,1,13&timerange=7776000

I think we could close this now. We'll keep looking at the performance numbers and optimized whatever we can.

Status: ASSIGNED → RESOLVED
Closed: 2 months ago20 days ago
Resolution: --- → FIXED
Target Milestone: --- → 94 Branch
You need to log in before you can comment on or make changes to this bug.