Closed Bug 1908744 Opened 2 years ago Closed 5 days ago

Update ANGLE to 9eadd12e33d65b8c9101ddef1d77d1c9a07e07bf (Chrome Beta 148.0.7778)

Categories

(Core :: Graphics, task, P1)

task

Tracking

()

RESOLVED FIXED
154 Branch
Tracking Status
firefox154 --- fixed

People

(Reporter: jgilbert, Assigned: ErichDonGubler)

References

(Blocks 14 open bugs)

Details

(Keywords: webcompat:platform-bug)

User Story

user-impact-score:60

Attachments

(8 files, 1 obsolete file)

48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
No description provided.
Blocks: 1854003
Blocks: 1908749
Blocks: 1885033
Blocks: 1885034
Blocks: 1899421
Blocks: 1885035
Blocks: 1899422
Blocks: 1899423
Blocks: 1854001
Blocks: 1908753
Assignee: nobody → ahale

I'm going to attempt to update ANGLE to 7599 (very latest version) as we normally update to chromium beta's stable angle but this will land a bit late and there are some fixes after current stable angle that we want.

This is a first time for me updating ANGLE, and our documentation is hard to find and a little bit-rotted so I am likely to write new docs to put on firefox-source-docs based on my findings.

Note to self: https://bugzilla.mozilla.org/show_bug.cgi?id=2008997 has been applied to gecko but not mozilla/angle, will need to create the commit (with proper author).

See Also: → 2008997
Depends on: 2030848

I ended up doing a lot of the work for this to unblock myself for bug 2035532. It's not ready to land but should be a good starting point for someone to take over. Or I can resume when I get blocked again.

Firefox branch: https://github.com/jamienicol/firefox/commits/angle-update/
Angle branch: https://github.com/jamienicol/angle/commits/firefox-update/

The angle branch is branch firefox-129 of https://github.com/mozilla/angle/ rebased on to ANGLE branch chromium/7778. Though it's both missing a few of the most recent fixes from firefox-129, and the base is now few commits behind chromium/7778. (And 7778 is no longer the current beta branch). So it should be rebased again onto a newer upstream base, and the missing commits from firefox-129 should be cherry picked.

The patch stack is much shorter than before: the upstream cherry picks we carried can now be dropped, I squashed the "add missing ..." commits into the commits they were missing from to tidy it up, and the support for compiling with pre-C++20 can be dropped.

I got claude to do some of the conflict resolution where code had significantly changed, so these should be looked at carefully.

On the firefox side I think there are a few things to figure out:

  • ShCompileOptions::enforcePackingRestrictions was removed upstream. See the angle bug for details, but essentially they don't believe this check is worthwhile at compile time (or translation time). I think they still enforce it at link time, but we'll only use ANGLE to link shaders on windows.
    • Chrome therefore skips these tests in "no passthrough" mode (which doesn't use ANGLE), but runs them in "passthrough" mode (which does use angle).
    • We need to decide whether to accept these tests as failing on non-Windows (i.e. on platforms that only use angle for shader translation only rather than for rendering), revert the patch and accept the maintainance burden of such, or add equivalent validation to our WebGL code.
  • The immutable-tex-render-feedback CTS tests now fail due to this angle change. Angle thinks the CTS is wrong, and the tests are marked as failing in chromium too. So I think it's an easy call for us to do the same, but thought it was worth calling out
  • ANGLE dropped support for GLSL version < 150 as a shader translator output target. Beacuse ANGLE no longer support GL versions < 3.2. This will mean we can no longer translate shaders for GL context versions < 3.2, and therefore cannot support webgl. This affects the translator, i.e. all platforms not just windows.
    • So we either need to
      a) block webgl versions which don't support that GLSL/GL version
      b) revert this patch and accept the maintainence burden
      c) Or see if we can get angle to revert the patch and continue supporting these targets
    • For what it's worth, I checked telemetry to see what percentage of our users have OpenGL < 3.2. We don't expose the version directly but using the adapter id as a proxy claude estimated about 1.2%. Which I think makes a) unviable.
Depends on: 2039887
Blocks: 2039887
No longer depends on: 2039887
User Story: (updated)

ANGLE dropped support for GLSL version < 150 as a shader translator output target

To summarise some offline discussions: Ashley spoke to ANGLE folks and they dropped support because their number of affected users was negligible. Jeff found that there exists some hardware that does not support desktop GL 3.2, but probably does support GLES 2.0. We should therefore attempt to give these users WebGL 1 on top of GLES 2.

So the plan was to accept dropping support for GLSL versions < 150, but ensure we attempt to create a GLES 2 context rather than a desktop GL < 3.2

No longer blocks: 2039887
Assignee: ahale → egubler
Status: NEW → ASSIGNED
Depends on: 2045022
Depends on: 2045041
Summary: Update ANGLE to at least May2024 → Update ANGLE to latest (at least May2024)
Priority: P3 → P1
Summary: Update ANGLE to latest (at least May2024) → Update ANGLE to Chrome Beta 148.0.7778

I've created the firefox-153 branch in mozilla/angle to track the patches being accumulated from the current base.

Depends on: 2046052
Blocks: 2035541
See Also: → 2046052

I moved to the Netherlands recently from Florida. To my utter non-surprise, the system locale changed how things were rendered! This is a problem for re-vendoring ANGLE specifically, because the artifacts produced in moz.yaml and cherry_picks.txt are supposed to be consistent between the users who run them. Ugh!

Squash some system-dependent behavior by passing a consistent --date=format:… to all invocations of git log in the vendor_from_git Python module, and revendor ANGLE from the same commit. The only change this applies is a leading zero to some commit timestamps; otherwise, there are no changes to imported ANGLE code. This was necessary because there's no locale-independent way of asking Git to print dates without a leading space (i.e., %e) or leading zero (i.e., %d). So, we opt for the latter.

I doubt we've squashed everything system-dependent here, but this was enough to get things back to "normal" on my machine.

ANGLE commit aaeeea0c42a877535428b146c7fd342ec19f0529 started adding the Device ID to the renderer string. We must update the KNOWN_ON_CI renderer list to reflect the new format. We've only observed the last item in the list, so only add it; if we need to add more later, we can do so in a similar manner.

It's been years in the making! 😅 I return to revendoring ANGLE with
tons of support from @jnicol, who has done much (most?) of the work
associated with this patch.

There are some important call-outs in this patch; this is mostly content written by @jnicol, which I've glued together into a single commit.

Overall scope

It's been years since we revendored ANGLE. There are a huge number of changes, and that makes reviewing this…well, prohibitively difficult, to say the least! My suggestion is to not try to review the entire set of changes, much like we don't force a reviewers to deal with all source for code in, say, third_party/ (which we eventually hope to move our checkout to).

update-angle.py changes

The new library angle_common_shader_state is now used by both the translator and libGLESv2. We therefore add it to gfx/angle/moz.build's DIRS like we do for angle_common, and use already_added_dirs to prevent it being added multiple times.

Desktop GL front-end support has been removed upstream (in commit b16d105fc63a7365d6809d5033cb1b2172d8d311) so no longer needs to be explicitly disabled.

Conversely, there now exists a wgpu/Dawn backend which must be explicitly disabled.

The angle_enable_apple_translator_workarounds arg has been removed (in commit 20b6624a88a599fec92cceaf7d80b0e1141449cf). We must instead manually ensure the required sources are vendored and added to the translator's sources for macOS builds. In the future the need for hacks like this will be obviated by switching to gn_processor.py to generate our builds scripts in bug 2035532.

Due to update-angle.py flattening GN source_sets' sources into the libraries which depend on them, some sources are compiled multiple times in different libraries which are then linked together, causing linker errors. We manually remove problematic sources from one of their libraries to avoid this. Again, this will be fixed in a more robust manner by bug 2035532 (as gn_processor.py emits source_sets as static libraries).

Lastly, the library //third_party/llvm-build/Release+Asserts/lib/clang/23/lib/windows/clang_rt.builtins-x86_64.lib is being pulled in as a dependency for some reason, so we filter out any dependencies beginning with //. And synchronization is now brought in as a genuine dependency on Windows, but we must avoid adding this to OS_LIBS on other platforms. For now, we only ever add it to OS_LIBS on Windows, as we know that other platforms have no OS lib dependencies. Once again, these issues will be fixed properly in bug 2035532.

Test expectations

immutable-tex-render-feedback tests now fail due to an upstream change where they believe the CTS is incorrect.

uniform-packing-restrictions tests now fail on platforms where ANGLE is only used for shader translation, due to ANGLE no longer enforcing these rules at compile time.

EXT_depth_clamp tests now pass on Windows, and vector-matrix-constructor-scalarization tests now pass on macOS, due to upstream fixes.

:jnicol and I exhaustively went over every downstream commit we had in firefox-129, and determined what we need to do for each of them to ensure we don't lose anything important. Things generally fell into these buckets:

  1. A fix is already in the new ANGLE base, no action item needed (beyond maybe dropping the commit because it conflicts instead of empties out).
  2. The fix needs to be carried forward because it's not yet in ANGLE upstream (at the point we're revendoring from, anyway).

I have updated the firefox-153 branch to accommodate all of the known action items except a tricky forward-porting reversion of jamienicol/firefox:092bd3b41b224d823eb601a3e47e2ea3999fb0e4 (ShCompileOptions being removed) and at least some work to correspondingly revert it in ANGLE, which I plan to work on today. I think this is close enough that we can start testing in CI, and maybe even a review process for the stack.

Here's a roadmap of what's left:

  1. Figure out the forward-port reversion of ShCompileOptions and consume it, adjusting test expectations accordingly.
  2. Do some testing to figure out what regressions might be present.
  3. Complete a review process.

I think (1) and (2) can happen independently, and (3) is soft-dependent on (2) so a reviewed patch doesn't get completely invalidated.

Pushed by egubler@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/f67420a758a2 https://hg.mozilla.org/integration/autoland/rev/f2a891d77669 chore(angle): update `KNOWN_ON_CI` renderer list with new ANGLE value r=gfx-reviewers,aosmond

options stopped having any effect in upstream angle commit 02e7f967597a2d706517c456739f6aecd4828a52, when its behaviour was turned on by default. The flag was later removed altogether in commit 37698dcf13dd61a3baac496eebc6bfd151df352f.

Pushed by egubler@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/261d2a25bfbd https://hg.mozilla.org/integration/autoland/rev/2cd63d2bf914 chore(angle): remove unused `ShCompileOptions.variables` config r=gfx-reviewers,jnicol
Pushed by agoloman@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/d70684b03aa4 https://hg.mozilla.org/integration/autoland/rev/2b246d425a48 Revert "Bug 1908744 - chore(angle): remove unused `ShCompileOptions.variables` config r=gfx-reviewers,jnicol" for causing failures @test_conformance__attribs__gl-bindAttribLocation-aliasing.html.

Backed out for causing failures @test_conformance__attribs__gl-bindAttribLocation-aliasing.html.

Flags: needinfo?(egubler)

Erich, I think you have to land that one along with the update - the variable still has an effect in the current version of ANGLE we have vendored

Flags: needinfo?(egubler)
Attachment #9597901 - Attachment description: Bug 1908744 - chore(angle): remove unused `ShCompileOptions.variables` config r=#gfx-reviewers → Bug 1908744, part 1 - chore(angle): remove unused `ShCompileOptions.variables` config r=#gfx-reviewers

(NOTE: Most of this patch's development was done by @jnicol. I stand behind it as a patch I understand, but want to recognize his efforts as most of its material.)

The new library angle_common_shader_state is now used by both the translator and libGLESv2. We therefore add it to gfx/angle/moz.build's DIRS like we do for angle_common, and use already_added_dirs to prevent it being added multiple times.

Desktop GL front-end support has been removed upstream (in commit b16d105fc63a7365d6809d5033cb1b2172d8d311) so no longer needs to be explicitly disabled.

Conversely, there now exists a wgpu/Dawn backend which must be explicitly disabled.

The angle_enable_apple_translator_workarounds arg has been removed (in commit 20b6624a88a599fec92cceaf7d80b0e1141449cf). We must instead manually ensure the required sources are vendored and added to the translator's sources for macOS builds. In the future the need for hacks like this will be obviated by switching to gn_processor.py to generate our builds scripts in bug 2035532.

Due to update-angle.py flattening GN source_sets' sources into the libraries which depend on them, some sources are compiled multiple times in different libraries which are then linked together, causing linker errors. We manually remove problematic sources from one of their libraries to avoid this. Again, this will be fixed in a more robust manner by bug 2035532 (as gn_processor.py emits source_sets as static libraries).

Lastly, the library //third_party/llvm-build/Release+Asserts/lib/clang/23/lib/windows/clang_rt.builtins-x86_64.lib is being pulled in as a dependency for some reason, so we filter out any dependencies beginning with //. And synchronization is now brought in as a genuine dependency on Windows, but we must avoid adding this to OS_LIBS on other platforms. For now, we only ever add it to OS_LIBS on Windows, as we know that other platforms have no OS lib dependencies. Once again, these issues will be fixed properly in bug 2035532.

(NOTE: Most of this patch's development was done by @jnicol. I stand behind it as a patch I understand, but want to recognize his efforts as most of its material.)

  • immutable-tex-render-feedback tests now fail due to an upstream change where they believe the CTS is incorrect.
  • EXT_depth_clamp tests now pass on Windows, and vector-matrix-constructor-scalarization tests now pass on macOS, due to upstream fixes.
Attachment #9596895 - Attachment description: WIP: Bug 1908744 - chore(angle): update ANGLE to `chromium/7778` r=#gfx-reviewers → WIP: Bug 1908744, part 4 - chore(angle): update ANGLE to `chromium/7778` r=#gfx-reviewers
Attachment #9597901 - Attachment description: Bug 1908744, part 1 - chore(angle): remove unused `ShCompileOptions.variables` config r=#gfx-reviewers → Bug 1908744, part 1 - chore(angle): stop using removed `ShCompileOptions.variables` config r=#gfx-reviewers
Attachment #9599147 - Attachment description: Bug 1908744, part 2 - chore(angle): update `update-angle.py` for upcoming ANGLE update r=#gfx-reviewers → Bug 1908744, part 2 - chore(angle): update `update-angle.py` r=#gfx-reviewers
Attachment #9599148 - Attachment description: Bug 1908744, part 3 - test(angle): update expectations for ANGLE update r=#gfx-reviewers → Bug 1908744, part 3 - test(angle): update expectations r=#gfx-reviewers
Attachment #9596895 - Attachment description: WIP: Bug 1908744, part 4 - chore(angle): update ANGLE to `chromium/7778` r=#gfx-reviewers → Bug 1908744, part 4 - chore(angle): update ANGLE to `chromium/7778` r=#gfx-reviewers

:jkratzer: Since ANGLE is a sec-sensitive component, I wanted to ping you about this patch stack being ready enough to maybe point some auditing resources at early. Did you want to set something up before landing? Otherwise, I think this is basically ready to land after a typical review process.

Flags: needinfo?(jkratzer)
Attachment #9596895 - Attachment description: Bug 1908744, part 4 - chore(angle): update ANGLE to `chromium/7778` r=#gfx-reviewers → Bug 1908744, part 4 - chore(angle): update base to 9eadd12e33d65b8c9101ddef1d77d1c9a07e07bf r=#gfx-reviewers
Summary: Update ANGLE to Chrome Beta 148.0.7778 → Update ANGLE to 9eadd12e33d65b8c9101ddef1d77d1c9a07e07bf (Chrome Beta 148.0.7778)

One of the Mozilla-specific patches we had prior to this patch sequence exposed private variable size limits as user-configurable. At that time, the single limit member MaxPrivateVariableSizeInBytes enforced both individual and total private variable sizes. This wasn't necessarily the most intelligent check, but it did its job for us.

Upstream, two changes happened that we needed to account for with a new revision of the patch. The MaxPrivateVariableSizeInBytes limit was split into Max{,Total}PrivateVariableSizeInBytes, splitting out the individual and total size checks into two separate knobs. The former also requires opt-in with ShCompileOptions.rejectWebglShadersWithLargeVariables. Begin using both.

Attachment #9599147 - Attachment description: Bug 1908744, part 2 - chore(angle): update `update-angle.py` r=#gfx-reviewers → Bug 1908744, part 3 - chore(angle): update `update-angle.py` r=#gfx-reviewers
Attachment #9599148 - Attachment description: Bug 1908744, part 3 - test(angle): update expectations r=#gfx-reviewers → Bug 1908744, part 4 - test(angle): update expectations r=#gfx-reviewers
Attachment #9596895 - Attachment description: Bug 1908744, part 4 - chore(angle): update base to 9eadd12e33d65b8c9101ddef1d77d1c9a07e07bf r=#gfx-reviewers → Bug 1908744, part 5 - chore(angle): update base to 9eadd12e33d65b8c9101ddef1d77d1c9a07e07bf r=#gfx-reviewers

Okay, this patch stack is ready to go. Jamie and I have gone to some pretty significant to make sure we're not breaking anything important with the new rebase. So, I'm planning on landing this ASAP.

🫡 Thanks for the patience, everyone!

Flags: needinfo?(jkratzer)
Pushed by egubler@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/4f95d15c6332 https://hg.mozilla.org/integration/autoland/rev/cefaacd164b1 part 1 - chore(angle): stop using removed `ShCompileOptions.variables` config r=gfx-reviewers,jnicol https://github.com/mozilla-firefox/firefox/commit/f45cc36d8576 https://hg.mozilla.org/integration/autoland/rev/f9501d3298de part 2 - chore(angle): migrate to reworked `Max{,Total}PrivateVariableSizeInBytes` limits r=gfx-reviewers,jnicol https://github.com/mozilla-firefox/firefox/commit/b7050d38b8ed https://hg.mozilla.org/integration/autoland/rev/f57f4eb32ba3 part 3 - chore(angle): update `update-angle.py` r=gfx-reviewers,nical https://github.com/mozilla-firefox/firefox/commit/828f6e7efe56 https://hg.mozilla.org/integration/autoland/rev/7019c4c5b85d part 4 - test(angle): update expectations r=gfx-reviewers,nical https://github.com/mozilla-firefox/firefox/commit/6e035dc7d652 https://hg.mozilla.org/integration/autoland/rev/4c67671fda1b part 5 - chore(angle): update base to 9eadd12e33d65b8c9101ddef1d77d1c9a07e07bf r=gfx-reviewers,nical
Pushed by csabou@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/1d312a19cc2e https://hg.mozilla.org/integration/autoland/rev/75dc405e97bb Revert "Bug 1908744, part 5 - chore(angle): update base to 9eadd12e33d65b8c9101ddef1d77d1c9a07e07bf r=gfx-reviewers,nical" for causing hazard failures.

Backed out for causing hazard failures.

Push with failures

Failure log

Backout link

Flags: needinfo?(egubler)

I…hadn't heard of hazard testing before! Time to learn. /me rolls up sleeves

Flags: needinfo?(egubler)

I think only partially backing this out might itself cause other failures. See comment 16 where attempting to land just the first patch by itself resulted in a backout. Should we back the whole thing out?

Flags: needinfo?(csabou)

It backed out the whole stack: https://hg-edge.mozilla.org/integration/autoland/rev/75dc405e97bb9f032dc94dabcd1358288e827047
Please ignore comment 27 as Bugherder should have done.

Flags: needinfo?(csabou)
Depends on: 2052331
Blocks: 2039887

With the current state of the root analysis in hazards testing, two cases in patches for bug 1908744 fail:

  • std::__future_base::_State_baseV2, whose destructor is defined out-of-line in libstdc++.
  • angle::Closure, a polymorphic abstract base. std::make_shared is only used for its subclasses; the deleter runs the derived destructor, and the base destructor is never used directly.

Instead of asserting, only perform the rewrite when the synthesized destructor actually exists in the body database. When it does not, return nothing so the ~shared_ptr() call is analyzed normally.

Comment on attachment 9604183 [details]
Bug 1908744 - chore(hazards): only rewrite shared_ptr<T> dtors when T::~T() exists r=sfink

Revision D310296 was moved to bug 2052331. Setting attachment 9604183 [details] to obsolete.

Attachment #9604183 - Attachment is obsolete: true

Okay, so, I've filed bug 2052331 to handle what seems like the only real CI failure with autoland checks. That's a neat bit of progress! Hopefully :sfink can help us get unblocked here soon. 🤞🏻

Pushed by egubler@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/3670c00a7607 https://hg.mozilla.org/integration/autoland/rev/d3316cb73818 part 1 - chore(angle): stop using removed `ShCompileOptions.variables` config r=gfx-reviewers,jnicol https://github.com/mozilla-firefox/firefox/commit/fd1892c99921 https://hg.mozilla.org/integration/autoland/rev/0a80bc7cfeb2 part 2 - chore(angle): migrate to reworked `Max{,Total}PrivateVariableSizeInBytes` limits r=gfx-reviewers,jnicol https://github.com/mozilla-firefox/firefox/commit/33dd45304318 https://hg.mozilla.org/integration/autoland/rev/e88cd835b4bb part 3 - chore(angle): update `update-angle.py` r=gfx-reviewers,nical https://github.com/mozilla-firefox/firefox/commit/b11cc9a596f4 https://hg.mozilla.org/integration/autoland/rev/7b03fdad9cf3 part 4 - test(angle): update expectations r=gfx-reviewers,nical https://github.com/mozilla-firefox/firefox/commit/2fe9f8877008 https://hg.mozilla.org/integration/autoland/rev/22c80afa7078 part 5 - chore(angle): update base to 9eadd12e33d65b8c9101ddef1d77d1c9a07e07bf r=gfx-reviewers,nical
Pushed by abutkovits@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/a3d2605e3c70 https://hg.mozilla.org/integration/autoland/rev/0b736d13634f Revert "Bug 1908744, Bug 2052331, part 5 - chore(angle): update base to 9eadd12e33d65b8c9101ddef1d77d1c9a07e07bf r=gfx-reviewers,nical" for causing Hazard Bustages.
Flags: needinfo?(egubler)
Pushed by egubler@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/c929fb16090d https://hg.mozilla.org/integration/autoland/rev/b7131ffd2c21 part 1 - chore(angle): stop using removed `ShCompileOptions.variables` config r=gfx-reviewers,jnicol https://github.com/mozilla-firefox/firefox/commit/fc8270fb0217 https://hg.mozilla.org/integration/autoland/rev/ab8aceebce73 part 2 - chore(angle): migrate to reworked `Max{,Total}PrivateVariableSizeInBytes` limits r=gfx-reviewers,jnicol https://github.com/mozilla-firefox/firefox/commit/b0c7462fb774 https://hg.mozilla.org/integration/autoland/rev/e39ffef9d882 part 3 - chore(angle): update `update-angle.py` r=gfx-reviewers,nical https://github.com/mozilla-firefox/firefox/commit/f2dfc2255c9f https://hg.mozilla.org/integration/autoland/rev/53af7abe73ed part 4 - test(angle): update expectations r=gfx-reviewers,nical https://github.com/mozilla-firefox/firefox/commit/6a2def449e8b https://hg.mozilla.org/integration/autoland/rev/16134e4ac77c part 5 - chore(angle): update base to 9eadd12e33d65b8c9101ddef1d77d1c9a07e07bf r=gfx-reviewers,nical
Regressions: 2054355
Attachment #9607076 - Attachment description: WIP: Bug 1908744 - chore: touch `CLOBBER` for ANGLE update → Bug 1908744 - chore: touch `CLOBBER` for ANGLE update r=#gfx-reviewers
Status: ASSIGNED → RESOLVED
Closed: 5 days ago
Resolution: --- → FIXED
Target Milestone: --- → 154 Branch
Regressions: 2055074
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: