Nightly-as-release builds are slower than regular Nightly builds (not PGO'ing properly)
Categories
(Firefox Build System :: Task Configuration, defect)
Tracking
(firefox126 fixed)
| Tracking | Status | |
|---|---|---|
| firefox126 | --- | fixed |
People
(Reporter: mstange, Assigned: jrmuizel)
References
Details
(Keywords: perf-alert, Whiteboard: [fxp])
Attachments
(1 file)
We have Nightly-as-release builds on autoland. These builds let us answer the question: What would the performance Firefox Release be if we shipped Nightly as Release today?
The idea is that Nightly builds have extra diagnostic checks enabled which slow things down, so Nightly-as-release builds should be faster than regular Nightly builds.
However, this graph of Speedometer 3 scores (higher is better) shows that the opposite is the case: Nightly-as-release is slower than Nightly.
I think that's because PGO isn't working properly in Nightly-as-release.
Nightly-as-release builds reuse the PGO profile from the regular Nightly run. This appears to cause trouble:
Regular Nightly build log: 245 instances of "function control flow change detected"
Nightly-as-release build log: 10036 instances of "function control flow change detected"
This means that Nar builds are currently not useful because they don't have realistic performance.
|
Assignee | |
Comment 1•4 months ago
|
||
|
Reporter | |
Comment 2•4 months ago
|
||
This issue affects all platforms but we care about Windows the most, so it's fine to just fix Windows in this bug and leave the other platforms to follow-ups.
|
||
Updated•4 months ago
|
|
||
Updated•4 months ago
|
|
||
Updated•4 months ago
|
|
|
Assignee | |
Comment 3•4 months ago
|
||
|
|
Reporter | |
Updated•3 months ago
|
|
|
Assignee | |
Comment 4•3 months ago
|
||
Without this we have a bunch of CFG hash mismatches and the performance is bad.
Pushed by jmuizelaar@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/03cadc907692 Use a nightly-as-release profile when building nightly-as-release r=firefox-build-system-reviewers,glandium
|
||
Comment 6•2 months ago
|
||
| bugherder | ||
|
||
Updated•2 months ago
|
|
||
Comment 7•2 months ago
|
||
(In reply to Pulsebot from comment #5)
Pushed by jmuizelaar@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/03cadc907692
Use a nightly-as-release profile when building nightly-as-release
r=firefox-build-system-reviewers,glandium
== Change summary for alert #42001 (as of Mon, 25 Mar 2024 08:10:49 GMT) ==
Improvements:
| Ratio | Test | Platform | Options | Absolute values (old vs new) | Performance Profiles |
|---|---|---|---|---|---|
| 5% | speedometer3 | windows10-64-nightlyasrelease-qr | fission webrender | 11.45 -> 11.97 | Before/After |
| 3% | speedometer3 | windows10-64-nightlyasrelease-qr | fission webrender | 11.55 -> 11.89 | Before/After |
For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=42001
|
|
||
Updated•2 months ago
|
|
|
Assignee | |
Comment 8•2 months ago
|
||
(In reply to Andra Esanu (needinfo me) from comment #7)
For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=42001
Why does the alert give two different ratios?
|
|
||
Comment 9•2 months ago
|
||
(In reply to Jeff Muizelaar [:jrmuizel] from comment #8)
Why does the alert give two different ratios?
We had two alerts and we grouped them together. Our alerting system can cause multiple alerts to be created for the same revision range (even though the main culprit revision is the same for both). This causes differing performance ratios to be generated, since we're comparing the same metric for different revisions.
Description
•