Closed Bug 2060853 Opened 29 days ago Closed 21 days ago

294.67 - 2.06% google-slides largestContentfulPaint + 113 more (Android, Linux, OSX, Windows) regression on Mon July 27 2026

Categories

(Firefox :: General, defect)

defect

Tracking

()

RESOLVED WONTFIX
Tracking Status
firefox-esr140 --- unaffected
firefox-esr153 --- unaffected
firefox153 --- unaffected
firefox154 --- unaffected
firefox155 --- fix-optional

People

(Reporter: performance-sheriff-bot, Unassigned)

References

(Blocks 1 open bug, Regression)

Details

(Keywords: perf, perf-alert, regression)

Attachments

(2 files)

Perfherder has detected a browsertime performance regression from push c81c89c15dc7d2ce83e475bb4ca191a9dd577e82. As author of one of the patches included in that push, we need your help to address this regression.

Please acknowledge, and begin investigating this alert within 3 business days, or the patch(es) may be backed out in accordance with our regression policy.

Regression Test Platform Options Absolute values [old vs new] Performance Profiles
295% google-slides largestContentfulPaint (doc) windows11-64-24h2-shippable fission warm webrender 231.43 ms -> 913.41 ms Before/After
234% google-slides largestContentfulPaint (doc) windows11-64-24h2-shippable bytecode-cached cold fission webrender 422.40 ms -> 1,409.39 ms Before/After
208% google-slides largestContentfulPaint (doc) windows11-64-24h2-shippable cold fission webrender 449.60 ms -> 1,385.40 ms Before/After
175% imdb PerceptualSpeedIndex (doc) macosx1470-64-shippable fission warm webrender 142.12 ms -> 390.47 ms Before/After
154% imdb ContentfulSpeedIndex (doc) macosx1470-64-shippable fission warm webrender 123.68 ms -> 314.58 ms Before/After
147% imdb PerceptualSpeedIndex (doc) linux2404-64-shippable fission warm webrender 135.83 ms -> 335.96 ms Before/After
127% imdb ContentfulSpeedIndex (doc) linux2404-64-shippable fission warm webrender 124.60 ms -> 283.08 ms Before/After
121% nytimes ContentfulSpeedIndex (doc) macosx1470-64-shippable fission warm webrender 108.59 -> 240.44 Before/After
98% imdb loadtime (doc) macosx1470-64-shippable cold fission webrender 872.96 ms -> 1,732.34 ms Before/After
74% imdb loadtime (doc) linux2404-64-shippable cold fission webrender 940.39 ms -> 1,634.25 ms Before/After
... ... ... ... ... ...
2% instagram ContentfulSpeedIndex (doc) linux2404-64-shippable cold fission webrender 464.97 -> 474.83 Before/After
2% instagram ContentfulSpeedIndex (doc) linux2404-64-shippable cold fission webrender 463.58 ms -> 473.36 ms Before/After
2% google-mail ContentfulSpeedIndex (doc) linux2404-64-shippable cold fission webrender 887.19 ms -> 905.82 ms Before/After
2% google-mail FirstVisualChange (doc) linux2404-64-shippable cold fission webrender 885.20 ms -> 903.77 ms Before/After
2% google-slides PerceptualSpeedIndex (doc) linux2404-64-shippable bytecode-cached cold fission webrender 846.18 ms -> 863.60 ms Before/After
Improvements Test Platform Options Absolute values [old vs new] Performance Profiles
98% facebook-nav.friends LastVisualChange (doc) macosx1470-64-shippable cold fission webrender 10,741.95 ms -> 161.97 ms
98% facebook-nav.groups LastVisualChange (doc) macosx1470-64-shippable cold fission webrender 10,746.76 ms -> 211.66 ms
95% facebook-nav.friends ContentfulSpeedIndex (doc) macosx1470-64-shippable cold fission webrender 1,499.01 ms -> 78.39 ms
94% facebook-nav.groups ContentfulSpeedIndex (doc) macosx1470-64-shippable cold fission webrender 1,750.10 ms -> 98.13 ms
90% facebook-nav.friends SpeedIndex (doc) macosx1470-64-shippable cold fission webrender 955.83 ms -> 93.73 ms
... ... ... ... ... ...
2% expedia ContentfulSpeedIndex (doc) linux2404-64-shippable cold fission webrender 428.00 ms -> 418.47 ms Before/After

Support & Actionable Steps

For questions, please contact fbilt@mozilla.com. You can also find help on Slack (#perf-help) or Matrix (#perftest).

Action guide details:

Flags: needinfo?(sclements)

Set release status flags based on info from the regressing bug 2056188

See Also: → 2058879
Flags: needinfo?(sclements)

:sclements, since you are the author of the regressor, bug 2056188, could you take a look? Also, could you set the severity field?

For more information, please visit BugBot documentation.

Flags: needinfo?(sclements)

Redirecting to mstange, sparky since they're helping with these sets of issues.

Flags: needinfo?(sclements)
Flags: needinfo?(mstange.moz)
Flags: needinfo?(gmierzwinski)

Starting to examine the before and after profiles.
There are some early oddities, for instance imdb before, https://share.firefox.dev/3UhLyrO, doesn't display the large amazon add, while with Nova a process is made for that content, ssl-images-amazon.com, and the ad is show, https://share.firefox.dev/4hSxS0i

Flags: needinfo?(acreskey)

Ah - Nova makes the content area a little bit smaller.

Retrieved from the browsertime files
.[0].browserScripts[0].pageinfo.visualElements.viewport

                viewport
before          1265x939
after           1255x924
delta           -10, -15

This has some exaggerated impact on the recorded site tests.

e.g.
• on imdb it changes the Amazon ad request to one that actually succeeds, thus slowing creating a new content process, loading the ad and slowing down the navigation
• on google-slides a different LCP element is chosen. Otherwise visual metrics like SpeedIndex are within the noise.

In terms of the improvements:
• As far as I can tell, the facebook navigations tests appears to be broken prior to the nova enablement (does not navigate to "friends" or "groups" as a login pop-up appears). I'll follow up.

I'll go through the rest, but so far this doesn't look like a real problem.

The google-slides largestContentfulPaint regression is a "mode" change - in the past there was a low mode, and now all datapoints are in the high mode. But the high mode existed before Nova.

Flags: needinfo?(mstange.moz)

Issues with the facebook nav tests filed under bug 2061481

See Also: → 2061481
Attached image nytimes_nova.png

The triggered regression on NYTimes isn't a direct comparison because with the small viewport the Nova run doesn't include the main image from the early content.

And the visual metrics of this test can also be dominated by a timer-based paywall pop-up that comes later (logged as a possible issue in bug 2061483).

I don't see any real regression here.

Flags: needinfo?(acreskey)

(In reply to Andrew Creskey [:acreskey] from comment #8)

I don't see any real regression here.

Can we mark this as INVALID then?

Flags: needinfo?(acreskey)

(In reply to Dave Townsend [:mossop] from comment #9)

(In reply to Andrew Creskey [:acreskey] from comment #8)

I don't see any real regression here.

Can we mark this as INVALID then?

I believe I've ruled out all the big shifts as invalid, but let me just take a look at some of the small changes e.g. 2% google-mail FirstVisualChange just to make sure we're not missing anything.

I believe that some of the regressions picked up from this alert are valid.

I haven't sorted them all, but here are a couple:

Buzzfeed cold on MacOs

buzzfeed FirstVisualChange coldfission
179.0ms -> 272.6ms
52.3% regression

FirstVisualChange and SpeedIndex, both visual metrics determined by screen recording, increase by ~100ms
Time series: https://perfherder2.netlify.app/?series=autoland,5151282,13&series=autoland,5151285,13&range=1783527900000,1786119900000&reps=0

One thing I see in profiles is a very long, 73.2ms WR OS Compositor frame with Nova
Before: https://share.firefox.dev/45LCfD1
With Nova: https://share.firefox.dev/4wRJWno

I don't see this on the warm navigations.

Yahoo mail cold on Macos

yahoo-mail FirstVisualChange coldfission
200.7ms -> 296.0ms
47.49% regression

Similar issue, with FirstVisualChange and SpeedIndex about 80ms slower
Time series: https://perfherder2.netlify.app/?series=autoland,5151166,13&series=autoland,5151169,13&range=1783527900000,1786119900000&reps=0

Also seeing a very long, 86.8ms WR OS Compositor frame with Nova
Before: https://share.firefox.dev/3Skv1Tt
With Nova: https://share.firefox.dev/3RZctrQ

This may only happen once per session (e.g. shader compilation?) but I'm not sure yet.

Oh interesting. Yes I believe those are shader compiles and they only affect the cold loads. It would be interesting to see the list of shaders (including the shader variant settings) that is used with/without nova; I could imagine that the rounded corner clip makes a difference but I think that would be a bug - we use OS compositor clipping for the content area rounded corners so the shaders in the content area should be the same. (And the shaders for the browser chrome would affect an earlier composite, not the one where the page content starts coming in.)

Another issue that looks valid:

Instagram on linux, cold

Instagram FirstVisualChange coldfission
87.0ms -> 108.3ms
24.53% regression

Time series: https://perfherder2.netlify.app/?series=autoland,5691215,13&range=1780939980000,1786123980000&sel=autoland,5691215,2625299254,-1&cmp=autoland,5691215,2648380606,-1&reps=0

I'll need someone familiar with our render pipeline to diagnose, but one thing I see in the Nova profiles is the presence of a second longer composite, 26ms in this case:
Before: https://share.firefox.dev/4fZGZtu
With Nova: https://share.firefox.dev/4z5HQl5

Quick note on the reported Android regressions:

Test                             Options       Before      After    Change
-------------------------------  ---------  ---------  ---------  --------
amazon loadtime                  cold         1124.76    1629.56   +44.88%
youtube-watch loadtime           cold          850.75    1166.98   +37.17%
amazon LastVisualChange          cold         1468.32    1962.81   +33.68%
youtube-watch loadtime           cold+fis      848.67    1093.26   +28.82%
amazon-search loadtime           cold         1061.21    1324.79   +24.84%
amazon-search LastVisualChange   cold         1305.63    1549.82   +18.70%

I'm quite sure these are erroneous for a couple of reasons:
• To my understanding browser.nova.enabled does not affect Fenix
• The (noisy) test scores appear to have returned to their baseline ranges all by themselves. time series

Depends on: 2061850

(In reply to Markus Stange [:mstange] from comment #12)

Oh interesting. Yes I believe those are shader compiles and they only affect the cold loads. It would be interesting to see the list of shaders (including the shader variant settings) that is used with/without nova; I could imagine that the rounded corner clip makes a difference but I think that would be a bug - we use OS compositor clipping for the content area rounded corners so the shaders in the content area should be the same. (And the shaders for the browser chrome would affect an earlier composite, not the one where the page content starts coming in.)

During navigation we do in fact build three shaders with Nova that are not otherwise built, although I only see about 11ms of build time: bug 2061850

Continuing to go through the alerts.

The google docs warm SpeedIndex regression is not a real problem, it's simply a side effect of how the document is assembled in the smaller viewport.

You can see in the ci time Series that no other visual metrics are affected.

MacOs
google-docs SpeedIndex warm
~1100ms to ~1300ms
13.55% regression

The MacOs reddit LastVisualChange looks to be just the vertical scroll bar continues to move for a longer period.
Not a problem.

MacOs reddit-billgates-post-2.hot LastVisualChange
cold
310.5ms -> 463.5ms
49.25% regression
Attached image bing-search-warm.png

The bing-search regression (Windows, bing-search ContentfulSpeedIndex. 448.6ms -> 548.5ms) looks to be the byproduct of how the different layouts animated in the top right of the page (with and without Nova).

Also, the images are missing from the later frames, both before and after Nova.
Does not look like a real problem for users to me.

I went through these alerts and I didn't see any issues that I would classify as real performance regressions for users.

Shifting when some shaders are compiled slows down the first navigation a little bit, but otherwise it would have slowed down the first chrome render, bug 2061850.

Otherwise the reported alerts are generally that a given submetric moved as a result of the slightly different viewport used with Nova.

Flags: needinfo?(acreskey)

(In reply to Andrew Creskey [:acreskey] from comment #19)

I went through these alerts and I didn't see any issues that I would classify as real performance regressions for users.

Shifting when some shaders are compiled slows down the first navigation a little bit, but otherwise it would have slowed down the first chrome render, bug 2061850.

Otherwise the reported alerts are generally that a given submetric moved as a result of the slightly different viewport used with Nova.

Closing based on that summary

Status: NEW → RESOLVED
Closed: 21 days ago
Resolution: --- → WONTFIX
Flags: needinfo?(gmierzwinski)
See Also: → 2065197
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: