3.12 - 3.09% speedometer EmberJS-Debug-TodoMVC/CompletingAllItems/Sync / speedometer EmberJS-Debug-TodoMVC/CompletingAllItems (OSX) regression on Thu May 16 2024
Categories
(Firefox Build System :: Toolchains, defect)
Tracking
(firefox-esr115 unaffected, firefox126 unaffected, firefox127 unaffected, firefox128 fixed)
| Tracking | Status | |
|---|---|---|
| firefox-esr115 | --- | unaffected |
| firefox126 | --- | unaffected |
| firefox127 | --- | unaffected |
| firefox128 | --- | fixed |
People
(Reporter: afinder, Assigned: glandium)
References
(Regression)
Details
(Keywords: perf, perf-alert, regression)
Attachments
(3 files)
2025-05-15 build, assembly view of array_join
Perfherder has detected a browsertime performance regression from push c6dd0210975b42f6a4d9e08b8047cf1e4f13ffe5. As author of one of the patches included in that push, we need your help to address this regression.
Regressions:
| Ratio | Test | Platform | Options | Absolute values (old vs new) | Performance Profiles |
|---|---|---|---|---|---|
| 3% | speedometer EmberJS-Debug-TodoMVC/CompletingAllItems/Sync | macosx1400-64-shippable-qr | fission webrender | 64.15 -> 66.15 | Before/After |
| 3% | speedometer EmberJS-Debug-TodoMVC/CompletingAllItems | macosx1400-64-shippable-qr | fission webrender | 65.26 -> 67.28 | Before/After |
Improvements:
| Ratio | Test | Platform | Options | Absolute values (old vs new) | Performance Profiles |
|---|---|---|---|---|---|
| 4% | speedometer Vanilla-ES2015-TodoMVC/CompletingAllItems/Sync | macosx1015-64-shippable-qr | fission webrender | 14.62 -> 14.00 | Before/After |
| 4% | speedometer Vanilla-ES2015-Babel-Webpack-TodoMVC/CompletingAllItems/Sync | macosx1015-64-shippable-qr | fission webrender | 14.59 -> 13.99 | Before/After |
| 4% | speedometer Vanilla-ES2015-TodoMVC/CompletingAllItems | macosx1015-64-shippable-qr | fission webrender | 16.52 -> 15.85 | Before/After |
| 3% | webaudio | windows10-64-shippable-qr | fission webrender | 81.21 -> 78.42 | Before/After |
Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the patch(es) may be backed out in accordance with our regression policy.
If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.
You can run these tests on try with ./mach try perf --alert 402
For more information on performance sheriffing please see our FAQ.
|
||
Comment 1•26 days ago
|
||
Set release status flags based on info from the regressing bug 1878764
|
||
Comment 2•21 days ago
|
||
I profiled Nightly builds from 2024-05-15 and 2024-05-17 locally on an M1Max Macbook Pro.
2024-05-15: https://share.firefox.dev/456incU
2024-05-17: https://share.firefox.dev/4aKVj4I
I don't see anything explaining a 3% difference. But I do see a 1.3% difference in array_join.
The profiles were captured on https://extra--speedometer-preview.netlify.app/?suite=EmberJS-Debug-TodoMVC&iterationCount=100
See the Profile Info dropdown for profiler settings - I used an interval of 0.4ms and generally chose profiler settings with low overhead.
|
|
||
Comment 3•21 days ago
|
||
|
|
||
Comment 4•21 days ago
|
||
|
|
||
Comment 5•21 days ago
|
||
The assembly looks quite similar and yet the instruction sample counts are quite different. I don't have an explanation at the moment.
|
Assignee | |
Updated•14 days ago
|
|
|
Assignee | |
Comment 6•14 days ago
|
||
This has the potential to regress other things, and if that happens,
we'll balance the pros and cons. But first, let's attempt this.
|
||
Updated•14 days ago
|
Pushed by mh@glandium.org: https://hg.mozilla.org/integration/autoland/rev/2d836066995d Revert clang 18 change that causes regressions in some speedometer tests. r=firefox-build-system-reviewers,sergesanspaille
|
||
Comment 8•14 days ago
|
||
| bugherder | ||
|
Reporter | |
Comment 9•8 days ago
|
||
(In reply to Pulsebot from comment #7)
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/2d836066995d
Revert clang 18 change that causes regressions in some speedometer tests.
r=firefox-build-system-reviewers,sergesanspaille
Perfherder has detected a browsertime performance change from push 2d836066995d9c7cdb5c191401fa8bd393bb0eab.
Improvements:
| Ratio | Test | Platform | Options | Absolute values (old vs new) | Performance Profiles |
|---|---|---|---|---|---|
| 3% | speedometer EmberJS-Debug-TodoMVC/CompletingAllItems | macosx1400-64-shippable-qr | fission webrender | 67.05 -> 64.83 | Before/After |
| 3% | speedometer EmberJS-Debug-TodoMVC/CompletingAllItems/Sync | macosx1400-64-shippable-qr | fission webrender | 65.90 -> 63.71 | Before/After |
| 2% | speedometer EmberJS-Debug-TodoMVC | macosx1400-64-shippable-qr | fission webrender | 213.45 -> 209.09 | Before/After |
Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests.
If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.
You can run these tests on try with ./mach try perf --alert 647
For more information on performance sheriffing please see our FAQ.
Description
•