44.99% google-docs-canvas LastVisualChange (OSX) regression on Sat October 16 2021
Categories
(Toolkit Graveyard :: OS.File, defect)
Tracking
(firefox96 affected)
Tracking | Status | |
---|---|---|
firefox96 | --- | affected |
People
(Reporter: alexandrui, Unassigned)
References
(Regression)
Details
(Keywords: perf, perf-alert, regression)
Attachments
(2 files)
Perfherder has detected a browsertime performance regression from push b0ea805e58e7c711b0494d644e936932dccdd4d7. As author of one of the patches included in that push, we need your help to address this regression.
Regressions:
Ratio | Test | Platform | Options | Absolute values (old vs new) |
---|---|---|---|---|
45% | google-docs-canvas LastVisualChange | macosx1014-64-shippable-qr | warm webrender | 1,163.33 -> 1,686.67 |
7% | google-docs-canvas PerceptualSpeedIndex | macosx1014-64-shippable-qr | warm webrender | 828.46 -> 888.25 |
Improvements:
Ratio | Test | Platform | Options | Absolute values (old vs new) |
---|---|---|---|---|
5% | wikipedia fcp | linux1804-64-shippable-qr | warm webrender | 184.83 -> 174.92 |
4% | wikipedia fnbpaint | linux1804-64-shippable-qr | warm webrender | 195.00 -> 188.00 |
Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.
For more information on performance sheriffing please see our FAQ.
Comment 1•2 years ago
|
||
I think the above alert is also missing:
- improvements in session restore from https://treeherder.mozilla.org/perfherder/alerts?id=31982 (bug 1649599 comment 21).
- improvements in JS + explicit memory on Windows from https://treeherder.mozilla.org/perfherder/alerts?id=31952 (bug 1735500 comment 7).
Note that bug 1649599 and bug 1735500 both landed at the same time.
To be honest, I'm not quite sure how either of these would have affected the google-docs-canvas test.
Bug 1649599 only affected the session restore code.
Whilst bug 1735500 did change the initial browser type, I'm wondering why it might have affected something that I understand is a page load test - unless the test is somehow including the browser switching time. However if that was the case, I'd have expected it to show up on more of the tests.
Gijs, Dave, is there anyone with more experience of the google-docs-canvas tests that could make an assessment here?
Updated•2 years ago
|
Comment 2•2 years ago
|
||
These are alerts for macOS 10.14, but 10.15 was flat over the same time period:
so I'm going to suggest we wontfix/ignore this, given the other improvements which seem more sensible, unless someone has spare time to profile the difference and see what's happening? It seems very unlikely to be real, and anyway we're out of time to do something about it...
Speaking of which, why are we getting an alert now for a change from October that we're practically shipping? (last beta is tomorrow)
Comment 3•2 years ago
•
|
||
(In reply to :Gijs (he/him) from comment #2)
unless someone has spare time to profile the difference and see what's happening?
I've rerun the jobs with the profiler enabled:
Comment 4•2 years ago
|
||
:aionescu could you provide a side-by-side video comparison of the regression?
Comment 5•2 years ago
|
||
I can't really see much in those profiles, but also, it looks to me as if the profile jobs have:
[task 2021-11-26T15:50:22.246Z] 15:50:22 INFO - 'browsertime_visualmetrics': False,
and they don't seem to include markers for the tests involved here, so it's hard to know which parts of the profile are relevant... Florian, am I missing something here?
(I still think investigating this further might not be the best use of our time given the regression was limited to 10.14 and happened quite a while ago.)
Reporter | ||
Comment 6•2 years ago
|
||
Reporter | ||
Comment 7•2 years ago
|
||
Comment 8•2 years ago
|
||
(In reply to :Gijs (he/him) from comment #2)
so I'm going to suggest we wontfix/ignore this, given the other improvements which seem more sensible, unless someone has spare time to profile the difference and see what's happening? It seems very unlikely to be real, and anyway we're out of time to do something about it...
Agreed, based on limited insight provided by profiles, platform specificity, and minimal noticable difference in side-by-side videos.
Speaking of which, why are we getting an alert now for a change from October that we're practically shipping? (last beta is tomorrow)
I believe this alert was originally associated with another alert summary, which went unnoticed.
Updated•11 months ago
|
Updated•11 months ago
|
Description
•