Closed Bug 1631778 Opened 4 months ago Closed 3 months ago

0.11 - 51.82% build times / installer size (android-4-0-armv7-api16, android-4-2-x86, android-5-0-aarch64, android-5-0-x86_64, osx-shippable) regression on push b4ebfaf0e6f3ed856786b6e145741e273066b225 (Tue April 21 2020)


(Core :: Graphics: WebRender, defect)




Tracking Status
firefox-esr68 --- unaffected
firefox75 --- unaffected
firefox76 --- unaffected
firefox77 --- fixed


(Reporter: marauder, Assigned: jnicol)


(Blocks 1 open bug, Regression)


(Keywords: perf-alert, regression)


(2 files)

Perfherder has detected a 2 performance regression from push b4ebfaf0e6f3ed856786b6e145741e273066b225. As author of one of the patches included in that push, we need your help to address this regression.


52% build times android-4-0-armv7-api16 opt gcp taskcluster-projects/970387039909/machineTypes/custom-32-73728 901.42 -> 1,368.53
50% build times android-4-2-x86 opt gcp taskcluster-projects/970387039909/machineTypes/custom-32-73728 939.93 -> 1,414.02
50% build times android-5-0-aarch64 opt gcp taskcluster-projects/970387039909/machineTypes/custom-32-73728 928.57 -> 1,395.18
43% build times android-5-0-x86_64 opt gcp taskcluster-projects/970387039909/machineTypes/custom-32-73728 991.34 -> 1,414.16
0.11% installer size osx-shippable opt instrumented 114,670,240.33 -> 114,798,422.83

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Blocks: 1628613
Component: Performance → Graphics: WebRender
Flags: needinfo?(jnicol)
OS: Unspecified → macOS
Product: Testing → Core
Regressed by: 1604615
Hardware: Unspecified → All
Version: Version 3 → unspecified

Sorry about this!

I believe the build time regressions are due to some indeterminism in the build-time generated file Sorting the list of shaders before writing them to that file seems to fix it from my testing. Will have a patch up shortly.

For the installer size regression, some increase is I believe unavoidable, because we now need to include a fair bit more text for the shader source code. Marian, do you happen to know what units those numbers are in? Is that an increase of around 130kB? In any case, I should be able to roughly halve it, by only including desktop GL shaders on Mac, and excluding the GLES ones.

Flags: needinfo?(jnicol) → needinfo?(marian.raiciof)

The list of pre-optimized shaders was being generated in a
non-deterministic order, causing large build time regressions. This
sort the list of shaders before writing them to the file.

Assignee: nobody → jnicol

The full set of optimized shaders can be quite large, so only optimize
for the GL version we expect to be used on the target platform. That
is GLES on Android and Windows (where we presume ANGLE is being used),
and GL elsewhere.

If a different GL version is used we will simply fall back to the
unoptimized shaders.

Depends on D71905

Pushed by
Ensure webrender's is filled deterministically. r=jrmuizel

Hey Jamie, i don't know what units are used by those numbers.

Flags: needinfo?(marian.raiciof)
Pushed by
Only optimize shaders for expected GL version on each platform r=jrmuizel
See Also: → 1632316
Closed: 4 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla77

The second patch here unfortunately undoes the windows perf wins from bug 1604615 :(

I used cfg!(target_os = windows) to decide whether to optimize GLES shaders. But since we cross compile windows builds from Linux and this is in a build script, it thinks the target_os is "linux" so optimizes the GL shaders instead. This means that when we run on windows, we can't find the optimized shaders, so use the unoptimized ones, so don't get the perf win. Android builds will have the same problem.

Regressions: 1632444

This generated a big regression on Raptor:
== Change summary for alert #25736 (as of Mon, 27 Apr 2020 06:38:34 GMT) ==


19% raptor-tp6-twitch-firefox-cold fcp windows10-64-shippable-qr opt 149.79 -> 177.50
18% raptor-tp6-twitch-firefox-cold fcp windows10-64-shippable-qr opt 150.88 -> 178.33
9% raptor-tp6-yahoo-mail-firefox-cold fcp windows10-64-shippable-qr opt 324.17 -> 353.75
7% raptor-tp6-yandex-firefox-cold fcp windows10-64-shippable-qr opt 276.33 -> 295.17
4% raptor-tp6-wikipedia-firefox-cold fcp windows10-64-shippable opt 1,275.58 -> 1,320.67
3% raptor-tp6-google-mail-firefox-cold loadtime windows10-64-shippable opt 621.29 -> 642.33
3% raptor-tp6-wikipedia-firefox-cold loadtime windows10-64-shippable opt 1,423.88 -> 1,469.92
3% raptor-tp6-microsoft-firefox-cold fcp windows10-64-shippable-qr opt 505.17 -> 521.17
3% raptor-tp6-sheets-firefox-cold loadtime windows10-64-shippable opt 1,613.00 -> 1,662.92
2% raptor-tp6-outlook-firefox-cold loadtime windows10-64-shippable opt 611.12 -> 626.08
2% raptor-tp6-sheets-firefox-cold windows10-64-shippable opt 888.84 -> 907.64
2% raptor-tp6-yahoo-mail-firefox-cold windows10-64-shippable-qr opt 502.08 -> 512.54

For up to date results, see:

Flags: needinfo?(jnicol)
Resolution: FIXED → ---

The regression should already be fixed by bug 1632444.

Flags: needinfo?(jnicol)

Just to confirm before we close this out as fixed by 1632444., Marian - are you still able to repro this?

Flags: needinfo?(marian.raiciof)
Closed: 4 months ago3 months ago
Flags: needinfo?(marian.raiciof)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.