1943149 - Update WGPU to upstream (week of 2025-01-20)

Status: RESOLVED FIXED

(Core :: Graphics: WebGPU, task, P1)

Comment 1

[Erich Gubler [:ErichDonGubler] (he/him)]

Attachment: Bug 1943149 - chore: audit latest releases of `{naga,wgpu-{core,hal,types}}` (23.*, 24.0.0) r=#supply-chain-reviewers

Comment 2

[Erich Gubler [:ErichDonGubler] (he/him)]

Attachment: Bug 1943149 - refactor(swgl): use env vars. instead of `cc::Build::target` to avoid `*C{,XX}FLAGS*` r=#gfx-reviewers

Newer versions of cc reject an unknown target, so we won't be able
to use this approach any more.

Comment 3

[Erich Gubler [:ErichDonGubler] (he/him)]

Attachment: Bug 1943149 - build(rust): upgrade `jobserver` 0.1.25 → 0.1.32 r=#supply-chain-reviewers

Comment 4

[Erich Gubler [:ErichDonGubler] (he/him)]

Attachment: Bug 1943149 - chore: audit `cc` 1.0.89 → 1.2.10 r=#supply-chain-reviewers

Comment 5

[Erich Gubler [:ErichDonGubler] (he/him)]

Attachment: Bug 1943149 - chore: audit `hashlink` 0.9.1 → 0.10.0 r=#supply-chain-reviewers

Comment 6

[Erich Gubler [:ErichDonGubler] (he/him)]

Attachment: Bug 1943149 - chore: audit `indexmap` 2.2.6 → 2.7.1 r=#supply-chain-reviewers

Comment 7

[Erich Gubler [:ErichDonGubler] (he/him)]

Attachment: Bug 1943149 - chore: audit `libsqlite3-sys` 0.28.0 → 0.31.0, `rusqlite` 0.31.0 → 0.33.0 r=#supply-chain-reviewers

Comment 8

[Erich Gubler [:ErichDonGubler] (he/him)]

Attachment: Bug 1943149 - chore: audit `metal` 0.30.0 → 0.31.0 r=#supply-chain-reviewers

Comment 9

[Erich Gubler [:ErichDonGubler] (he/him)]

Attachment: Bug 1943149 - chore(swgl): `cargo +1.84 fmt` r=#gfx-reviewers

Comment 10

[Erich Gubler [:ErichDonGubler] (he/him)]

Attachment: Bug 1943149 - chore(swgl): `cargo +1.84 clippy --fix` r=#gfx-reviewers

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8d7fc1a4bd16
chore: audit `hashlink` 0.9.1 → 0.10.0 r=supply-chain-reviewers
https://hg.mozilla.org/integration/autoland/rev/aca35d0586bc
chore: audit `indexmap` 2.2.6 → 2.7.1 r=supply-chain-reviewers
https://hg.mozilla.org/integration/autoland/rev/883fd681a67d
chore: audit `libsqlite3-sys` 0.28.0 → 0.31.0, `rusqlite` 0.31.0 → 0.33.0 r=supply-chain-reviewers

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6a816a37e239
refactor(swgl): use env vars. instead of `cc::Build::target` to avoid `*C{,XX}FLAGS*` r=gfx-reviewers,lsalzman

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/404b2c7266c8
chore: audit `cc` 1.0.89 → 1.2.10 r=supply-chain-reviewers

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4567ae997352
build(rust): upgrade `jobserver` 0.1.25 → 0.1.32 r=supply-chain-reviewers

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/21828af6c7d8
chore(swgl): `cargo +1.84 fmt` r=gfx-reviewers,lsalzman
https://hg.mozilla.org/integration/autoland/rev/50026351d1c0
chore(swgl): `cargo +1.84 clippy --fix` r=gfx-reviewers,lsalzman

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d7b13e506173
chore: audit `metal` 0.30.0 → 0.31.0 r=supply-chain-reviewers

Comment 17

[Erich Gubler [:ErichDonGubler] (he/him)]

Okay, so I have an update working locally after a couple of days of effort. It took so long because there were a lot of dependencies to update, and it took me a while to work through everything that needed to be updated.

I haven't posted a WIP patch for the re-vendor of WGPU yet, but I will do that after the spread of current patches in this bug lands. There are still some outstanding things to resolve before I can get those changes landed.

1. Resolve whether we will be bumping the minimum Rust version for mozilla-central to 1.82 or newer (CC :glandium). If not, we're blocked here, and I need to explore downgrading hashbrown from 15.2 to 14.5.
2. If (1) pans out, inform folks that work on application-services, update the PR to respect the new minimum Rust version, and get it landed.
3. Adjust patches to consume the landed PR from (2).

After these, it'll be an otherwise typical review flow.

Comment 18

[Erich Gubler [:ErichDonGubler] (he/him)]

Try push 1 to test my current changes, squashed into a single revision: https://treeherder.mozilla.org/jobs?repo=try&revision=28b3aecdf756b26d1da825602c92c76ba0f8edd5

Comment 19

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/8d7fc1a4bd16
https://hg.mozilla.org/mozilla-central/rev/aca35d0586bc
https://hg.mozilla.org/mozilla-central/rev/883fd681a67d
https://hg.mozilla.org/mozilla-central/rev/6a816a37e239
https://hg.mozilla.org/mozilla-central/rev/404b2c7266c8
https://hg.mozilla.org/mozilla-central/rev/4567ae997352
https://hg.mozilla.org/mozilla-central/rev/21828af6c7d8
https://hg.mozilla.org/mozilla-central/rev/50026351d1c0
https://hg.mozilla.org/mozilla-central/rev/d7b13e506173

Comment 20

[Erich Gubler [:ErichDonGubler] (he/him)]

Attachment: Bug 1943149 - build(rust): bump minimum Rust version 1.76 → 1.82 r=glandium

Comment 21

[Erich Gubler [:ErichDonGubler] (he/him)]

Attachment: Bug 1943149 - build(rust): upgrade `cc` 1.0.89 → 1.2.10 r=#supply-chain-reviewers

N.B. that this requires Rust 1.82. This is changed in a previous patch
in this patch's stack, and so should be satisfied.

Comment 22

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8c0c09c42a2b
chore: audit latest releases of `{naga,wgpu-{core,hal,types}}` (23.*, 24.0.0) r=supply-chain-reviewers

Comment 23

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/8c0c09c42a2b

Comment 24

[Phabricator Automation]

Comment on attachment 9462602 [details]
Bug 1943149 - build(rust): bump minimum Rust version 1.76 → 1.82 r=glandium

Revision D236028 was moved to bug 1945020. Setting attachment 9462602 [details] to obsolete.

Comment 25

[Erich Gubler [:ErichDonGubler] (he/him)]

Attachment: Bug 1943149 - build(swgl): work around broken (upstream) `cc::Tool` detection of `clang --driver-mode=cl` r=#gfx-reviewers

In upstream, a new heuristic for detecting compiler families in
cc::Tool was introduced. The heuristic we originally patched upstream
is now the fallback heuristic, and does not get used for our clang
compiler. This causes our interaction with default flags and
cc::Tool::is_like_msvc to be incorrect in swgl, since clang's CLI
to use cl-like arguments, and rejects clang-like arguments.

Work around this by detecting checking Tools' base command and
"wrapper arguments" to see if we're (1) using clang and (2) we have
a wrapper argument matching --driver-mode=cl. If so, provide cl-like
arguments in swgl, rather than clang-like arguments.

Comment 26

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d1cb92f43de1
build(swgl): work around broken (upstream) `cc::Tool` detection of `clang --driver-mode=cl` r=gfx-reviewers,nical

Comment 27

[Sandor Molnar[:smolnar]]

Backed out for causing wr bustages

Backout link: https://hg.mozilla.org/integration/autoland/rev/ef93a3ee2425bcbf790f3c897a485ec900e80c58

Push with failures

Failure log -> ./swgl/build.rs:259: operators should go at the end of the first line

Comment 28

[Erich Gubler [:ErichDonGubler] (he/him)]

Attachment: Bug 1943149 - build(swgl): work around broken (upstream) `cc::Tool` detection of `clang --driver-mode=cl` r=#gfx-reviewers

In upstream, a new heuristic for detecting compiler families in
cc::Tool was introduced. The heuristic we originally patched upstream
is now the fallback heuristic, and does not get used for our clang
compiler. This causes our interaction with default flags and
cc::Tool::is_like_msvc to be incorrect in swgl, since clang's CLI
to use cl-like arguments, and rejects clang-like arguments.

Work around this by detecting checking Tools' base command and
"wrapper arguments" to see if we're (1) using clang and (2) we have
a wrapper argument matching --driver-mode=cl. If so, provide cl-like
arguments in swgl, rather than clang-like arguments.

Comment 29

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/18a4b3819fd8
build(swgl): work around broken (upstream) `cc::Tool` detection of `clang --driver-mode=cl` r=gfx-reviewers,nical

Comment 30

[Erich Gubler [:ErichDonGubler] (he/him)]

Attachment: Bug 1943149 - docs(rust): note Rust version as 1.76 for Firefox 136 r=glandium

Comment 31

[Phabricator Automation]

Comment on attachment 9463665 [details]
Bug 1943149 - docs(rust): note Rust version as 1.76 for Firefox 136 r=glandium

Revision D236655 was moved to bug 1945020. Setting attachment 9463665 [details] to obsolete.

Comment 32

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/18a4b3819fd8

Comment 33

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/68af412c331d
build(rust): upgrade `cc` 1.0.89 → 1.2.10

Comment 34

[agoloman]

Backed out for causing android build bustages.

• Backout link
• Push with failures
• Failure Log

Comment 35

[Erich Gubler [:ErichDonGubler] (he/him)]

Investigating. 😭 Might need help from :jnicol here.

Comment 36

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0993363f6e3c
build(rust): upgrade `cc` 1.0.89 → 1.2.10

Comment 37

[agoloman]

Backed out for causing android build bustages.

• Backout link
• Push with failures
• Failure Log

Comment 38

[Erich Gubler [:ErichDonGubler] (he/him)]

Rats, a newer cc (bug 1945694) didn't work, because it breaks PGO builds on Android (courtesy of bug 1946006). Welp, time to roll up my sleeves and figure this out!

Comment 39

[Phabricator Automation]

Comment on attachment 9462603 [details]
Bug 1943149 - build(rust): upgrade cc 1.0.89 → 1.2.10 r=#supply-chain-reviewers

Revision D236029 was moved to bug 1945694. Setting attachment 9462603 [details] to obsolete.

Comment 40

[Erich Gubler [:ErichDonGubler] (he/him)]

Attachment: Bug 1943149 - build(webgpu): update WGPU to db896a86d0db0637149d7012bd865059f8783ec3 r=#webgpu-reviewers!

Comment 41

[Erich Gubler [:ErichDonGubler] (he/him)]

Attachment: Bug 1943149 - test(webgpu): demote newly failing tests to tier 3 r=#webgpu-reviewers!

Comment 42

[Erich Gubler [:ErichDonGubler] (he/him)]

Attachment: Bug 1943149 - test(webgpu): promote newly `PASS`ing tests to tier 2 r=#webgpu-reviewers!

Comment 43

[Erich Gubler [:ErichDonGubler] (he/him)]

Attachment: Bug 1943149 - test(webgpu): adjust for new perma outcomes in tier 3 r=#webgpu-reviewers!

Comment 44

[Erich Gubler [:ErichDonGubler] (he/him)]

Attachment: Bug 1943149 - test(webgpu): accept observed intermittents in tier 3 r=#webgpu-reviewers!

Comment 45

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f83ff837ae3e
build(webgpu): update WGPU to db896a86d0db0637149d7012bd865059f8783ec3 r=supply-chain-reviewers,webgpu-reviewers,nical
https://hg.mozilla.org/integration/autoland/rev/33a153e890e7
test(webgpu): demote newly failing tests to tier 3 r=webgpu-reviewers,nical
https://hg.mozilla.org/integration/autoland/rev/d1567c298d34
test(webgpu): promote newly `PASS`ing tests to tier 2 r=webgpu-reviewers,nical
https://hg.mozilla.org/integration/autoland/rev/9570b1380c92
test(webgpu): adjust for new perma outcomes in tier 3 r=webgpu-reviewers,nical
https://hg.mozilla.org/integration/autoland/rev/c35a98735cfd
test(webgpu): accept observed intermittents in tier 3 r=webgpu-reviewers,nical

Comment 46

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2a76600f28d9
Fix wpt lint failure. CLOSED TREE

Comment 47

[Erich Gubler [:ErichDonGubler] (he/him)]

:smolnar, the lint failure suppression you've added with 2a76600f28d9 shouldn't be necessary, unless jobs intended to be tier 3 are being run in non-tier-3 contexts. What job did you discover failures in? 🤔 I ask because that change is going to get reverted with the automation we have with moz-webgpu-cts, and I'd like to fix the root cause quickly.

Comment 48

[Sandor Molnar[:smolnar]]

(In reply to Erich Gubler [:ErichDonGubler] from comment #47)

:smolnar, the lint failure suppression you've added with 2a76600f28d9 shouldn't be necessary, unless jobs intended to be tier 3 are being run in non-tier-3 contexts. What job did you discover failures in? 🤔 I ask because that change is going to get reverted with the automation we have with moz-webgpu-cts, and I'd like to fix the root cause quickly.

unfortunately, because of some build bustages, don't have any more insight about any other failures, beside the lint one. maybe in a couple of hours I will have full code coverage.

Comment 49

[Erich Gubler [:ErichDonGubler] (he/him)]

OH, I'd checked in bad WPT metadata. I'm so sorry! I needed to fix that before queueing to land. 😓 I'm so sorry about that.

Comment 50

[Sandor Molnar[:smolnar]]

(In reply to Erich Gubler [:ErichDonGubler] from comment #49)

OH, I'd checked in bad WPT metadata. I'm so sorry! I needed to fix that before queueing to land. 😓 I'm so sorry about that.

it seems my fix hasn't really "fixed" it :(

wpt-lint is still complaining

L.E it seems it's this line seems to be the problem.

want me to apply the same fix as before or how do you want to proceed ?

Comment 51

[Erich Gubler [:ErichDonGubler] (he/him)]

Looks like there's at least one other instance of fix that needs to happen; here's the patch I made to fix things up:

Commit ID: de6506dc6aeade7acb9a39b4a7620b62811756a8
Change ID: kotpwpxkkvltmnqqowrskykyysvxtpkr
Author   : Erich Gubler <[e-mail]> (3 minutes ago)
Committer: Erich Gubler <[e-mail]> (1 minute ago)

    fixup! Bug 1943149 - test(webgpu): accept observed intermittents in tier 3 r=#webgpu-reviewers!

diff --git a/testing/web-platform/mozilla/meta/webgpu/cts/webgpu/api/validation/capability_checks/limits/maxComputeWorkgroupStorageSize/cts.https.html.ini b/testing/web-platform/mozilla/meta/webgpu/cts/webgpu/api/validation/capability_checks/limits/maxComputeWorkgroupStorageSize/cts.https.html.ini
index 6708caf040..95f4d67589 100644
--- a/testing/web-platform/mozilla/meta/webgpu/cts/webgpu/api/validation/capability_checks/limits/maxComputeWorkgroupStorageSize/cts.https.html.ini
+++ b/testing/web-platform/mozilla/meta/webgpu/cts/webgpu/api/validation/capability_checks/limits/maxComputeWorkgroupStorageSize/cts.https.html.ini
@@ -3,10 +3,10 @@
   implementation-status: backlog
   expected:
     if os == "win" and debug: [OK, TIMEOUT]
-    if os == "mac": [OK, TIMEOUT]
     if os == "win" and not debug: CRASH
     if os == "linux" and debug: [OK, TIMEOUT]
     if os == "linux" and not debug: [OK, CRASH]
+    if os == "mac": [OK, TIMEOUT]
   [:limitTest="atDefault";testValueName="atLimit";async=false;wgslType="S1"]
     expected:
       if os == "win": [PASS, TIMEOUT, NOTRUN]
diff --git a/testing/web-platform/mozilla/meta/webgpu/cts/webgpu/api/validation/capability_checks/limits/maxInterStageShaderVariables/cts.https.html.ini b/testing/web-platform/mozilla/meta/webgpu/cts/webgpu/api/validation/capability_checks/limits/maxInterStageShaderVariables/cts.https.html.ini
index ec37f2d922..4b80ac6077 100644
--- a/testing/web-platform/mozilla/meta/webgpu/cts/webgpu/api/validation/capability_checks/limits/maxInterStageShaderVariables/cts.https.html.ini
+++ b/testing/web-platform/mozilla/meta/webgpu/cts/webgpu/api/validation/capability_checks/limits/maxInterStageShaderVariables/cts.https.html.ini
@@ -3,8 +3,8 @@
   implementation-status: backlog
   expected:
     if os == "win" and debug: [OK, TIMEOUT]
+    if os == "linux": [OK, CRASH]
     if os == "mac": [OK, TIMEOUT]
-    if os == "linux": [OK, CRASH]
   [:limitTest="atDefault";testValueName="atLimit";async=false;pointList=false;frontFacing=false;sampleIndex=false;sampleMaskIn=false;sampleMaskOut=false]
     expected:
       if os == "win": FAIL
diff --git a/testing/web-platform/mozilla/meta/webgpu/cts/webgpu/api/validation/capability_checks/limits/maxSampledTexturesPerShaderStage/cts.https.html.ini b/testing/web-platform/mozilla/meta/webgpu/cts/webgpu/api/validation/capability_checks/limits/maxSampledTexturesPerShaderStage/cts.https.html.ini
index f75516b019..72bcefd53a 100644
--- a/testing/web-platform/mozilla/meta/webgpu/cts/webgpu/api/validation/capability_checks/limits/maxSampledTexturesPerShaderStage/cts.https.html.ini
+++ b/testing/web-platform/mozilla/meta/webgpu/cts/webgpu/api/validation/capability_checks/limits/maxSampledTexturesPerShaderStage/cts.https.html.ini
@@ -426,13 +426,12 @@
 [cts.https.html?q=webgpu:api,validation,capability_checks,limits,maxSampledTexturesPerShaderStage:createPipeline,at_over:*]
   tags: [webgpu, webgpu-long]
   implementation-status: backlog
-  expected: [OK, TIMEOUT]
   expected:
     if os == "win" and debug: [OK, CRASH]
     if os == "win" and not debug: [OK, TIMEOUT]
     if os == "linux" and debug: [OK, TIMEOUT]
     if os == "linux" and not debug: [OK, CRASH]
-    if os == "mac" and not debug: [OK, TIMEOUT]
+    if os == "mac": [OK, TIMEOUT]
   [:limitTest="atDefault";testValueName="atLimit";async=false;bindingCombination="compute";order="backward";bindGroupTest="differentGroups"]
     expected:
       if os == "win" and debug: [PASS, TIMEOUT, NOTRUN]
@@ -3423,12 +3422,11 @@
 [cts.https.html?q=webgpu:api,validation,capability_checks,limits,maxSampledTexturesPerShaderStage:createPipelineLayout,at_over:*]
   tags: [webgpu, webgpu-long]
   implementation-status: backlog
-  expected: [OK, TIMEOUT]
   expected:
     if os == "win": [OK, TIMEOUT]
     if os == "linux" and debug: [OK, TIMEOUT]
     if os == "linux" and not debug: [OK, CRASH]
-    if os == "mac" and not debug: [OK, TIMEOUT]
+    if os == "mac": [OK, TIMEOUT]
   [:limitTest="atDefault";testValueName="atLimit";visibility=1;order="backward"]
     expected:
       if os == "win" and not debug: [PASS, TIMEOUT, NOTRUN]

Comment 52

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c37170befd12
Fix wpt lint failure/update wpt manifest CLOSED TREE

Comment 53

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/f83ff837ae3e
https://hg.mozilla.org/mozilla-central/rev/33a153e890e7
https://hg.mozilla.org/mozilla-central/rev/d1567c298d34
https://hg.mozilla.org/mozilla-central/rev/9570b1380c92
https://hg.mozilla.org/mozilla-central/rev/c35a98735cfd
https://hg.mozilla.org/mozilla-central/rev/2a76600f28d9
https://hg.mozilla.org/mozilla-central/rev/c37170befd12