Closed Bug 1930651 Opened 1 year ago Closed 1 year ago

382.78 - 8.2% google-search-restaurants LastVisualChange / bing SpeedIndex + 16 more (Android) regression on Tue November 5 2024

Categories

(Firefox for Android :: Toolbar, defect, P1)

Product:
Firefox for Android
Component:
Toolbar
Platform:
All
Android
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
134 Branch
Tracking Flags:
Tracking Status
firefox132 --- unaffected
firefox133 --- unaffected
firefox134 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: petru)

References

(Blocks 1 open bug, Regression)

Details

(Keywords: perf, perf-alert, regression, Whiteboard: [fxdroid][group3][toolbar-redesign-beta-blocker])

Attachments

(3 files, 1 external file, 1 obsolete file)

good results first test.mp4
295.60 KB, video/mp4
Details
bad results first test.mp4
296.07 KB, video/mp4
Details
GoogleRestaurantTestResultsComparison
386.55 KB, image/jpeg
Details

Perfherder has detected a browsertime performance regression from push f49863193c13c1def4db2dd3ea9c5d6bd9d517a7. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio Test Platform Options Absolute values (old vs new)
383% google-search-restaurants LastVisualChange android-hw-a55-14-0-aarch64-shippable warm webrender 113.78 -> 549.28
222% google-maps LastVisualChange android-hw-a55-14-0-aarch64-shippable warm webrender 168.32 -> 542.72
203% google-search-restaurants LastVisualChange android-hw-a55-14-0-aarch64-shippable cold webrender 211.51 -> 640.66
112% bing-search-restaurants LastVisualChange android-hw-a55-14-0-aarch64-shippable warm webrender 501.76 -> 1,062.84
94% stackoverflow LastVisualChange android-hw-a55-14-0-aarch64-shippable cold webrender 1,596.55 -> 3,103.44
80% google-maps LastVisualChange android-hw-a55-14-0-aarch64-shippable cold webrender 405.38 -> 729.02
71% instagram LastVisualChange android-hw-a55-14-0-aarch64-shippable cold webrender 1,540.37 -> 2,633.72
68% ebay-kleinanzeigen LastVisualChange android-hw-a55-14-0-aarch64-shippable warm webrender 729.29 -> 1,225.77
67% bing-search-restaurants LastVisualChange android-hw-a55-14-0-aarch64-shippable cold webrender 704.93 -> 1,178.26
67% ebay-kleinanzeigen-search LastVisualChange android-hw-a55-14-0-aarch64-shippable warm webrender 744.07 -> 1,241.93
... ... ... ... ...
28% facebook-cristiano LastVisualChange android-hw-a55-14-0-aarch64-shippable cold webrender 991.49 -> 1,267.42
12% ebay-kleinanzeigen ContentfulSpeedIndex android-hw-a55-14-0-aarch64-shippable warm webrender 331.59 -> 369.78
11% google-maps ContentfulSpeedIndex android-hw-a55-14-0-aarch64-shippable cold webrender 321.66 -> 356.58
10% ebay-kleinanzeigen-search ContentfulSpeedIndex android-hw-a55-14-0-aarch64-shippable warm webrender 374.02 -> 410.58
8% bing SpeedIndex android-hw-a55-14-0-aarch64-shippable warm webrender 263.62 -> 285.24

Improvements:

Ratio Test Platform Options Absolute values (old vs new)
41% cnn-ampstories FirstVisualChange android-hw-a55-14-0-aarch64-shippable warm webrender 364.02 -> 213.75
21% cnn-ampstories FirstVisualChange android-hw-a55-14-0-aarch64-shippable cold webrender 737.18 -> 582.97
17% cnn-ampstories ContentfulSpeedIndex android-hw-a55-14-0-aarch64-shippable warm webrender 364.45 -> 302.90
13% cnn-ampstories PerceptualSpeedIndex android-hw-a55-14-0-aarch64-shippable warm webrender 364.27 -> 316.68
11% youtube LastVisualChange android-hw-a55-14-0-aarch64-shippable warm webrender 866.78 -> 772.83
7% instagram ContentfulSpeedIndex android-hw-a55-14-0-aarch64-shippable warm webrender 2,068.08 -> 1,913.81

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

You can run all of these tests on try with ./mach try perf --alert 42465

The following documentation link provides more information about this command.

For more information on performance sheriffing please see our FAQ.

If you have any questions, please do not hesitate to reach out to bacasandrei@mozilla.com.

Flags: needinfo?(petru)

Some jobs are failing to run, but Bug 1927677 seemed the most likely to have caused this regression since all alerts are on the android platform. Other suspects are Bug 1921814 (push 80eb89ff8e3d) and Bug 1927546 (push 4472f9cb7110).

Set release status flags based on info from the regressing bug 1927677

status-firefox132: --- → unaffected
status-firefox133: --- → unaffected

Folks, could one of your patches in the regression range have caused this regression? Thanks

Flags: needinfo?(sthompson)
Flags: needinfo?(rjesup)

I think the potential for perf changes from bug 1921814 are confined to scenarios where the user has lots of tabs and the user is moving focus around the tab strip. I wouldn't expect any effects on visual changes in the content area.

Does Fenix use browser components (e.g. browser/components/tabbrowser/content/tabs.js) in addition to the toolkit components (e.g. toolkit/content/widgets/tabbox.js)? Most of the impact from bug 1921814, if there is any, would be in the browser component; my changes to the toolkit side should have been minimal.

Blocks: android-toolbar-phase-1-beta
Severity: -- → S3
Priority: -- → P1
Whiteboard: [fxdroid][group3][toolbar-redesign-beta-blocker]

Notes from toolbar triage meeting: Petru, we're going to pull this into this current sprint just to make sure we investigate this to the degree that we identify the nature of the regression

  • The result could be that the performance tests may just need to be looked at more closely (e.g. if it's measuring something about the page layout that has changed) - if it's not a legitimate regression, we should work with the Perf team to make sure the new result is clarified as the intentional new baseline performance measurement.
  • The result could also be that it's a legitimate regression, but that the Toolbar bug didn't actually cause this regression, and we can bounce this to whoever actually owns the regression.
  • Of course, it could also be a legitimate regression AND related to the Toolbar work, in which case we should resolve it before release.

Recommendation: work with Kaya (and sthompson and rjesup) to review this regression to figure out the nature of the actual regression.

Note: all of the above is true for you, as well, Harrison, regarding Bug 1929579.

cc'ing Harrison and Kaya as well.

Thought ./mach try perf --alert xxxx would run the exact same tests for the exact same commits but seems like it will run the exact same test for the current HEAD.
Looking at the commits log from https://hg.mozilla.org/mozilla-central/shortlog/9efa2d7e9e4c562953e77462912b1c9d6ca6fded I ran this perf alert try for each commit to hopefully see where the spike originates:

./mach try perf --alert 42465 for `Bug 1928016 - Add information to provide compatible Python version using virtual environments

remote: View your change here:
remote: https://hg.mozilla.org/try/rev/d9fa376952aac683f1a32109c929a19b2860c7af
remote:
remote: Follow the progress of your build on Treeherder:
remote: https://treeherder.mozilla.org/jobs?repo=try&revision=d9fa376952aac683f1a32109c929a19b2860c7af
remote: recorded changegroup in replication log in 0.012s
To hg::ssh://hg.mozilla.org/try

  • [new branch] 8b1c962e75feea9a3517bf8a0c64cc5b8a169fed -> branches/default/tip

!!!NOTE!!!
You'll be able to find a performance comparison here once the tests are complete (ensure you select the right framework):
https://perf.compare/compare-results?baseRev=32dd3704389892c4ae8df2fd91d7b148f1504043&newRev=d9fa376952aac683f1a32109c929a19b2860c7af&baseRepo=try&newRepo=try&framework=13

The old comparison tool is still available at this URL:
https://treeherder.mozilla.org/perfherder/compare?originalProject=try&originalRevision=32dd3704389892c4ae8df2fd91d7b148f1504043&newProject=try&newRevision=d9fa376952aac683f1a32109c929a19b2860c7af&framework=13

  •      2 commits/try-runs are created...          *

Base revision's try run: https://treeherder.mozilla.org/jobs?repo=try&revision=32dd3704389892c4ae8df2fd91d7b148f1504043
Local revision's try run: https://treeherder.mozilla.org/jobs?repo=try&revision=d9fa376952aac683f1a32109c929a19b2860c7af

 

 

./mach try perf --alert 42465 for `Bug 1927546: Fix sieve port number

remote: View your change here:
remote: https://hg.mozilla.org/try/rev/ab0549dff0a6ef315c81d823f44d7af3ccf6aff7
remote:
remote: Follow the progress of your build on Treeherder:
remote: https://treeherder.mozilla.org/jobs?repo=try&revision=ab0549dff0a6ef315c81d823f44d7af3ccf6aff7
remote: recorded changegroup in replication log in 0.020s
To hg::ssh://hg.mozilla.org/try

  • [new branch] e0cb6a57c6217549dcbc9a4c13f3795f04392a0c -> branches/default/tip

!!!NOTE!!!
You'll be able to find a performance comparison here once the tests are complete (ensure you select the right framework):
https://perf.compare/compare-results?baseRev=cef91ba40afdf74f3191d5af2bad0bf6afb44db2&newRev=ab0549dff0a6ef315c81d823f44d7af3ccf6aff7&baseRepo=try&newRepo=try&framework=13

The old comparison tool is still available at this URL:
https://treeherder.mozilla.org/perfherder/compare?originalProject=try&originalRevision=cef91ba40afdf74f3191d5af2bad0bf6afb44db2&newProject=try&newRevision=ab0549dff0a6ef315c81d823f44d7af3ccf6aff7&framework=13

  •      2 commits/try-runs are created...          *

Base revision's try run: https://treeherder.mozilla.org/jobs?repo=try&revision=cef91ba40afdf74f3191d5af2bad0bf6afb44db2
Local revision's try run: https://treeherder.mozilla.org/jobs?repo=try&revision=ab0549dff0a6ef315c81d823f44d7af3ccf6aff7

 

 

./mach try perf --alert 42465 for `Bug 1921814 - add keyboard focus for tab group labels

remote: View your change here:
remote: https://hg.mozilla.org/try/rev/f607e63316a2836b0f481153829aecf1250bf465
remote:
remote: Follow the progress of your build on Treeherder:
remote: https://treeherder.mozilla.org/jobs?repo=try&revision=f607e63316a2836b0f481153829aecf1250bf465
remote: recorded changegroup in replication log in 0.012s
To hg::ssh://hg.mozilla.org/try

  • [new branch] 7b14f4f6ca5419745466ec90556a56e3d71501b1 -> branches/default/tip

!!!NOTE!!!
You'll be able to find a performance comparison here once the tests are complete (ensure you select the right framework):
https://perf.compare/compare-results?baseRev=420261a7211babce2d1811d937bc748f6d2e2462&newRev=f607e63316a2836b0f481153829aecf1250bf465&baseRepo=try&newRepo=try&framework=13

The old comparison tool is still available at this URL:
https://treeherder.mozilla.org/perfherder/compare?originalProject=try&originalRevision=420261a7211babce2d1811d937bc748f6d2e2462&newProject=try&newRevision=f607e63316a2836b0f481153829aecf1250bf465&framework=13

  •      2 commits/try-runs are created...          *

Base revision's try run: https://treeherder.mozilla.org/jobs?repo=try&revision=420261a7211babce2d1811d937bc748f6d2e2462
Local revision's try run: https://treeherder.mozilla.org/jobs?repo=try&revision=f607e63316a2836b0f481153829aecf1250bf465

 

 

./mach try perf --alert 42465 for `Bug 1927677 - part 1 - Remove divider from on top of the OS navigation bar

remote: View your change here:
remote: https://hg.mozilla.org/try/rev/bf4c7726eb5d67af5da74e680da688d2e95a358b
remote:
remote: Follow the progress of your build on Treeherder:
remote: https://treeherder.mozilla.org/jobs?repo=try&revision=bf4c7726eb5d67af5da74e680da688d2e95a358b
remote: recorded changegroup in replication log in 0.012s
To hg::ssh://hg.mozilla.org/try

  • [new branch] d725227b0f7e13c9e769a10cf19e52c2b8658937 -> branches/default/tip

!!!NOTE!!!
You'll be able to find a performance comparison here once the tests are complete (ensure you select the right framework):
https://perf.compare/compare-results?baseRev=8f3847008f2984faf827bf910cd8d9f7b1abb6ee&newRev=bf4c7726eb5d67af5da74e680da688d2e95a358b&baseRepo=try&newRepo=try&framework=13

The old comparison tool is still available at this URL:
https://treeherder.mozilla.org/perfherder/compare?originalProject=try&originalRevision=8f3847008f2984faf827bf910cd8d9f7b1abb6ee&newProject=try&newRevision=bf4c7726eb5d67af5da74e680da688d2e95a358b&framework=13

  •      2 commits/try-runs are created...          *

Base revision's try run: https://treeherder.mozilla.org/jobs?repo=try&revision=8f3847008f2984faf827bf910cd8d9f7b1abb6ee
Local revision's try run: https://treeherder.mozilla.org/jobs?repo=try&revision=bf4c7726eb5d67af5da74e680da688d2e95a358b

 

 

./mach try perf --alert 42465 for `Bug 1927677 - part 2 - Don't add top toolbar divider after microsurvey ends

remote: View your change here:
remote: https://hg.mozilla.org/try/rev/41ed74d4384572c2cfd3bd14f8eed1b1102887b5
remote:
remote: Follow the progress of your build on Treeherder:
remote: https://treeherder.mozilla.org/jobs?repo=try&revision=41ed74d4384572c2cfd3bd14f8eed1b1102887b5
remote: recorded changegroup in replication log in 0.012s
To hg::ssh://hg.mozilla.org/try

  • [new branch] 399537dabdb8562db21c0da15d685c86e9232b59 -> branches/default/tip

!!!NOTE!!!
You'll be able to find a performance comparison here once the tests are complete (ensure you select the right framework):
https://perf.compare/compare-results?baseRev=9e8b6412e9523a4f65a4ff4da6db29ff2180eac1&newRev=41ed74d4384572c2cfd3bd14f8eed1b1102887b5&baseRepo=try&newRepo=try&framework=13

The old comparison tool is still available at this URL:
https://treeherder.mozilla.org/perfherder/compare?originalProject=try&originalRevision=9e8b6412e9523a4f65a4ff4da6db29ff2180eac1&newProject=try&newRevision=41ed74d4384572c2cfd3bd14f8eed1b1102887b5&framework=13

  •      2 commits/try-runs are created...          *

Base revision's try run: https://treeherder.mozilla.org/jobs?repo=try&revision=9e8b6412e9523a4f65a4ff4da6db29ff2180eac1
Local revision's try run: https://treeherder.mozilla.org/jobs?repo=try&revision=41ed74d4384572c2cfd3bd14f8eed1b1102887b5

 

 

./mach try perf --alert 42465 for `Bug 1927677 - part 3 - Show the navbar divider in the same layout as the navbar

remote: View your change here:
remote: https://hg.mozilla.org/try/rev/a345df1e76b15c6ddaea517613a21ec9c6729649
remote:
remote: Follow the progress of your build on Treeherder:
remote: https://treeherder.mozilla.org/jobs?repo=try&revision=a345df1e76b15c6ddaea517613a21ec9c6729649
remote: recorded changegroup in replication log in 0.014s
To hg::ssh://hg.mozilla.org/try

  • [new branch] e503a0ebaa9642199b8f057e29f189df8e6ab8cd -> branches/default/tip

!!!NOTE!!!
You'll be able to find a performance comparison here once the tests are complete (ensure you select the right framework):
https://perf.compare/compare-results?baseRev=b42da2a27ecc2452ae62aa99fe6fb284da29b799&newRev=a345df1e76b15c6ddaea517613a21ec9c6729649&baseRepo=try&newRepo=try&framework=13

The old comparison tool is still available at this URL:
https://treeherder.mozilla.org/perfherder/compare?originalProject=try&originalRevision=b42da2a27ecc2452ae62aa99fe6fb284da29b799&newProject=try&newRevision=a345df1e76b15c6ddaea517613a21ec9c6729649&framework=13

  •      2 commits/try-runs are created...          *

Base revision's try run: https://treeherder.mozilla.org/jobs?repo=try&revision=b42da2a27ecc2452ae62aa99fe6fb284da29b799
Local revision's try run: https://treeherder.mozilla.org/jobs?repo=try&revision=a345df1e76b15c6ddaea517613a21ec9c6729649

 

 

./mach try perf --alert 42465 for Bug 1929272 - Move transformRulesToMap` to shared file.

remote: View your change here:
remote: https://hg.mozilla.org/try/rev/c2278dc2565fc8e05247a31af39cf3b57394db2d
remote:
remote: Follow the progress of your build on Treeherder:
remote: https://treeherder.mozilla.org/jobs?repo=try&revision=c2278dc2565fc8e05247a31af39cf3b57394db2d
remote: recorded changegroup in replication log in 0.012s
To hg::ssh://hg.mozilla.org/try

  • [new branch] 26a101bb879592a152e952dde45c6e4306eb1fd9 -> branches/default/tip

!!!NOTE!!!
You'll be able to find a performance comparison here once the tests are complete (ensure you select the right framework):
https://perf.compare/compare-results?baseRev=03291a16a00f5b6b27035f888af10d8700915e15&newRev=c2278dc2565fc8e05247a31af39cf3b57394db2d&baseRepo=try&newRepo=try&framework=13

The old comparison tool is still available at this URL:
https://treeherder.mozilla.org/perfherder/compare?originalProject=try&originalRevision=03291a16a00f5b6b27035f888af10d8700915e15&newProject=try&newRevision=c2278dc2565fc8e05247a31af39cf3b57394db2d&framework=13

  •      2 commits/try-runs are created...          *

Base revision's try run: https://treeherder.mozilla.org/jobs?repo=try&revision=03291a16a00f5b6b27035f888af10d8700915e15
Local revision's try run: https://treeherder.mozilla.org/jobs?repo=try&revision=c2278dc2565fc8e05247a31af39cf3b57394db2d

Attachment #9437065 - Attachment is obsolete: true
Flags: needinfo?(petru)

Based on the above result seems like indeed this patch for bug 1927677 introduced this regression.
Not sure what exactly is tested and what exactly is the cause of the regression.
Checked with @zmckenney and will investigate a bit more what could have impacted this and an alternative approach about which I'll leave an update here tomorrow.

Flags: needinfo?(sthompson)
Flags: needinfo?(rjesup)
Assignee: nobody → petru
Status: NEW → ASSIGNED

Decided in the team to backout the original patch pending more investigation.

With the above patch we can see that the regression is fixed - https://perf.compare/compare-results?baseRev=32dd3704389892c4ae8df2fd91d7b148f1504043&baseRepo=try&newRev=f1d1c0adbff592505928d4ad31cf3f0767d8e332&newRepo=try&framework=13&search=google-search-restaurants

Pushed by plingurar@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/a8873c7d35a2 Revert "Bug 1927677 - part 3 - Show the navbar divider in the same layout as the navbar" r=android-reviewers,mcarare

https://hg.mozilla.org/mozilla-central/rev/a8873c7d35a2

Status: ASSIGNED → RESOLVED
Closed: 1 year ago
status-firefox134: affected → fixed
Resolution: --- → FIXED
Target Milestone: --- → 134 Branch
Regressions: 1932159

With a bit more time and understanding of these tests I investigated more using the results from this try as a good base where I see the attached video and the following data for the google-search-restaurants test:

"visualMetrics":[
   {
      "FirstVisualChange":112,
      "LastVisualChange":123,
      "SpeedIndex":118,
      "PerceptualSpeedIndex":116,
      "PerceptualSpeedIndexProgress":[
         { "timestamp":0, "percent":0 },
         { "timestamp":112, "percent":96 },
         { "timestamp":123, "percent":100 }
      ],
      "ContentfulSpeedIndex":112,
      "ContentfulSpeedIndexProgress":[
         { "timestamp":0, "percent":0 },
         { "timestamp":112, "percent":100 },
         { "timestamp":123, "percent":100 }
      ],
      "VisualProgress":[
         { "timestamp":0, "percent":0 },
         { "timestamp":112, "percent":45 },
         { "timestamp":123, "percent":100 }
      ],
      "videoRecordingStart":1888,
      "VisualReadiness":11,
      "VisualComplete85":123,
      "VisualComplete95":123,
      "VisualComplete99":123
   }
]

Also used the results from this try as showcasing the regression from where I get the attached video and the following data:

"visualMetrics":[
   {
      "FirstVisualChange":102,
      "LastVisualChange":655,
      "SpeedIndex":124,
      "PerceptualSpeedIndex":113,
      "PerceptualSpeedIndexProgress":[
         { "timestamp":0, "percent":0 },
         { "timestamp":102,  "percent":90 },
         { "timestamp":133,  "percent":100 },
         { "timestamp":139, "percent":100 },
         { "timestamp":201, "percent":100 },
         { "timestamp":218, "percent":100 },
         { "timestamp":276, "percent":100 },
         { "timestamp":655, "percent":100 }
      ],
      "ContentfulSpeedIndex":135,
      "ContentfulSpeedIndexProgress":[
         { "timestamp":0, "percent":0 },
         { "timestamp":102, "percent":75 },
         { "timestamp":133, "percent":76 },
         { "timestamp":139, "percent":77 },
         { "timestamp":201, "percent":80 },
         { "timestamp":218, "percent":80 },
         { "timestamp":276, "percent":82 },
         { "timestamp":655, "percent":100 }
      ],
      "VisualProgress":[
         { "timestamp":0, "percent":0 },
         { "timestamp":102, "percent":46 },
         { "timestamp":133, "percent":97 },
         { "timestamp":139, "percent":98 },
         { "timestamp":201, "percent":99 },
         { "timestamp":218, "percent":99 },
         { "timestamp":276, "percent":99 },
         { "timestamp":655, "percent":100 }
      ],
      "videoRecordingStart":1904,
      "VisualReadiness":553,
      "VisualComplete85":133,
      "VisualComplete95":133,
      "VisualComplete99":201
   }

Using this command (tweaked with a different path for the "bad results"

ffmpeg -v debug -i "/Users/petru/Downloads/Good results/browsertime-videos-annotated/google-search-restaurants/pages/www_google_com/search/warm/query-783930bf/data/video/1.mp4" -vsync 0 -vf "crop=1080:1818:0:291,scale=iwmin(400/iw,400/ih):ihmin(400/iw,400/ih),mpdecimate=0:64:640:0.001" "/Users/petru/Downloads/Good results/browsertime-videos-annotated/google-search-restaurants/pages/img-%d.png"

I get 65 very similar images for both of the above attached good/bad results videos.

Used also https://www.diffchecker.com/image-compare/ to compare the images and the only difference that I see
(when comparing the 9th good image and the 8th bad one (in the 9th bad one the google map was already loaded))
is that as expected the toolbar is now a bit smaller with the browser content a bit larger - since all the offending patch did was to remove the separate top divider.
The progress bar which is showed for the top toolbar is not included in the cropped images so I'm not sure what could have caused the difference in inferring the page visual updates.

If looking at the PerceptualSpeedIndexProgress data for both good/bad results we see very similar times to get to percent":100 but similar to bug 1929579? we see more updates in the bad results data.

@Greg Do you think this is another variant of what was seen in bug 1929579?
Any idea on what other things to investigate?
Should we also ignore this as a false positive?

Flags: needinfo?(gmierz2)

I checked locally, and when I crop out the scroll bar manually, the last visual change is the same in the before, and after cases. Also, if I run visualmetrics with full resolution the last visual change is the same in the before and after case. So it's the exact same issue as I found, and described here: https://bugzilla.mozilla.org/show_bug.cgi?id=1929579#c5.

However, I'm not sure what's going on with the bing speedindex metric. It doesn't have a regression in LastVisualChange like dailymail had in the other bug. I tested it locally, and the speedindex value doesn't change if I use full resolution, or if I crop the scroll bar out. I've triggered a side-by-side here to see what the difference might be: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&selectedTaskRun=fQsFj6ySRziwDe99mDYF4A.0&revision=565fd80298f9ff9d405ab90130b307a62a556c6f&searchStr=bing%2Cfenix

So for bing, the side-by-side videos look identical: https://firefoxci.taskcluster-artifacts.net/daRSiErlS3W_m8wTeuHzAw/0/public/build/side-by-side/browsertime-tp6m-fenix-bing-nofis/warm-side-by-side.mp4

Locally, I tested cropping out the caret and that didn't change anything, but cropping out the top area with the cookie notice made a bit of a difference. The numbers were closer to each after that (still seems like the regressing video had some regression but it's hard to say). It's possible that the picture/camera icon is causing the regression here as well. So it's either being caused by a delay in the cookie notice, or the picture/camera icon.

It could be useful to get a profile for the bing test so I've triggered one in the try run I linked in comment #17.

Flags: needinfo?(gmierz2)

I see in your comment you mentioned that a divider was removed which resulted in more pixels for the browser viewport. It's possible that having more pixels is causing the regression in bing too.

Thank you!
I'll try to investigate more but it looks to me like we should either

  • accept this as a false positive and do the divider change that would result in a slightly bigger browser viewport
  • try avoiding this regression (which might mask other things) by not updating the browser viewport height

(In reply to Pulsebot from comment #12)

Pushed by plingurar@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a8873c7d35a2
Revert "Bug 1927677 - part 3 - Show the navbar divider in the same layout as
the navbar" r=android-reviewers,mcarare

Perfherder has detected the following browsertime performance change. Some jobs are failing to run, making it challenging to identify the specific revision responsible for the changes in performance. Could you please confirm whether a8873c7d35a2a7db807918558d65611f9481e8d5 may have caused it?

Regressions:

Ratio Test Platform Options Absolute values (old vs new)
4% speedometer3 NewsSite-Nuxt/NavigateToWorld/Sync android-hw-a55-14-0-aarch64-shippable webrender 43.51 -> 45.17
4% speedometer3 NewsSite-Nuxt/NavigateToWorld/Sync android-hw-a55-14-0-aarch64-shippable webrender 43.20 -> 44.77

Improvements:

Ratio Test Platform Options Absolute values (old vs new)
20% facebook-cristiano LastVisualChange android-hw-a55-14-0-aarch64-shippable cold webrender 1,130.89 -> 909.37

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests.
If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.
You can run these tests on try with ./mach try perf --alert 42804
For more information on performance sheriffing please see our FAQ.

Flags: needinfo?(petru)

The above speedometer3 regressions were already reported in bug 1932159.
Giving that they're the result of reverting this patch which showed bigger regressions for other tests it looks like there's nothing more to do.

Flags: needinfo?(petru)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: