1909836 - 11.17 - 2.96% reddit SpeedIndex / reddit loadtime + 2 more (Linux, OSX, Windows) regression on Sat July 20 2024

Status: NEW

(Core :: Layout, defect)

Description

[Andra Esanu (needinfo me)]

Perfherder has detected a browsertime performance regression from push 6904d078df0be1cd17df43b3ed219a17708386b8. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)	Performance Profiles
11%	reddit SpeedIndex	linux1804-64-shippable-qr	fission warm webrender	397.70 -> 442.12	Before/After
7%	reddit SpeedIndex	windows11-64-shippable-qr	fission warm webrender	267.84 -> 287.28	Before/After
4%	reddit loadtime	macosx1015-64-shippable-qr	fission warm webrender	505.36 -> 523.33	Before/After
3%	reddit loadtime	macosx1015-64-shippable-qr	fission warm webrender	509.25 -> 524.33	Before/After

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

You can run these tests on try with ./mach try perf --alert 1481

For more information on performance sheriffing please see our FAQ.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1789166

Comment 2

[Emilio Cobos Álvarez [:emilio]]

So the bad profile in the first link shows that most (all?) of the extra time is idle (waiting for a network load?).

The change could make us prioritize loads differently (or trigger more loads like background images), so it's not totally unexpected, but still I want to understand what are we loading and why are we waiting for it. Maybe the image code could be tweaked to prioritize things differently.

Julien, I'm trying to do something a bit unorthodox, but I'm not succeeding. I'm looking at a good profile and a bad profile.

I want to see what items (if any) are different in the network track.

If there are none, then this is purely a network prioritization issue, and I have an idea of how to fix it.

If there are some, we might need to trigger some of those loads, but still we could try to prioritize differently.

I was trying to do this on devtools:

let bad = $$(".networkChartRowItem").map(i => i.innerText)

But I think the tree is virtualized so it doesn't catch everything I want. Do you know of a better way of doing it?

Comment 3

[Julien Wajsberg [:julienw]]

I think this does what you want if I understand properly:

let allNetworkMarkers = filteredMarkers.filter(({data}) => data?.type === "Network")
let allURIs = allNetworkMarkers.map(({data}) => data.URI)

Then you can look what's inside allNetworkMarkers and filter even more, for example with the content type, or the "pri" property for priority (I'm not sure what values it holds TBH :) ).

Please ask if you need anything more :-)

Comment 4

[Emilio Cobos Álvarez [:emilio]]

Ok, that's exactly what I needed, thanks Julien!

Interestingly, we load the same resources, and the load event fires later, but visually the page loads roughly the same way (i.e., before my patch the sidebar loads after the load event, after my patch it loads before, but it still loads roughly the same way).

Andra, is there any way to see a side-by-side of these? I suspect this is not really actionable / a meaningful regression (specially given PerceptualSpeedIndex and co didn't move...)

Comment 5

[Emilio Cobos Álvarez [:emilio]]

I see a side-by-side button in https://treeherder.mozilla.org/perfherder/alerts?id=1481 but they don't point to any job?

Comment 6

[Andra Esanu (needinfo me)]

Hi, here is the side-by-side link for 6904d078df0b

Comment 7

[Emilio Cobos Álvarez [:emilio]]

I don't see anything visually worse, but I guess looking at the page, they are using <details> for the sub-menus, so it's expected that we can end up doing a bit more work... However it's expected, and it's intended to allow doing things like find-in-page inside <details> elements...

Comment 8

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1789166

Comment 9

[Daniel Holbert [:dholbert]]

(In reply to Emilio Cobos Álvarez (:emilio) from comment #7)

looking at the page, they are using <details> for the sub-menus, so it's expected that we can end up doing a bit more work... However it's expected, and it's intended to allow doing things like find-in-page inside <details> elements...

emilio: It sounds like this is WONTFIX then, probably?

(I think we can consider it triaged & no-worse-than-S3 given your investigation so far, at least.)

Comment 10

[Daniel Holbert [:dholbert]]

side note, I spammed some retriggers (so now we've got 10+ samples per platform before and after, instead of 2-5 samples) and the regression does seem to remain:

https://treeherder.mozilla.org/perfherder/compare?originalProject=autoland&originalRevision=e2635b8bf747fe8542ea4442db5aca85bd94eb1b&newProject=autoland&newRevision=6904d078df0be1cd17df43b3ed219a17708386b8&framework=13&page=1&filter=reddit+speedindex&showOnlyImportant=1

That shows a 5-10% regression on each platform, under "reddit SpeedIndex opt fission warm webrender", with "Confidence: High". Though per comment 7 I'm not sure it's worrisome.