1862499 - Hit MOZ_CRASH(assertion failed: `(left == right)` left: `2`, right: `1`: RenderPipeline[32] is no longer alive) at /third_party/rust/wgpu-core/src/storage.rs:111

Status: VERIFIED FIXED

(Core :: Graphics: WebGPU, defect, P2)

Description

[Jason Kratzer [:jkratzer]]

Testcase found while fuzzing mozilla-central rev b73ef4c8979f (built with: --enable-debug --enable-fuzzing).

This is currently the most frequent crasher we have. Please prioritize accordingly.

Testcase can be reproduced using the following commands:

$ pip install fuzzfetch grizzly-framework
$ python -m fuzzfetch --build b73ef4c8979f --debug --fuzzing -n firefox
$ python -m grizzly.replay ./firefox/firefox testcase.html

Hit MOZ_CRASH(assertion failed: `(left == right)`   left: `2`,  right: `1`: RenderPipeline[32] is no longer alive) at /third_party/rust/wgpu-core/src/storage.rs:111

    ==65278==ERROR: UndefinedBehaviorSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7fdc809e6285 bp 0x7fdc1d3ede50 sp 0x7fdc1d3ede40 T65389)
    ==65278==The signal is caused by a WRITE memory access.
    ==65278==Hint: address points to the zero page.
        #0 0x7fdc809e6285 in MOZ_Crash /builds/worker/workspace/obj-build/dist/include/mozilla/Assertions.h:281:3
        #1 0x7fdc809e6285 in RustMozCrash /mozglue/static/rust/wrappers.cpp:18:3
        #2 0x7fdc809e621a in mozglue_static::panic_hook::habfbf582d66d5c86 /mozglue/static/rust/lib.rs:96:9
        #3 0x7fdc809e5c1b in core::ops::function::Fn::call::h081d0c2d4ea076dc /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/ops/function.rs:79:5
        #4 0x7fdc81a5ae7d in _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..Fn$LT$Args$GT$$GT$::call::hb3a915ffd78277c6 /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/alloc/src/boxed.rs:2007:9
        #5 0x7fdc81a5ae7d in std::panicking::rust_panic_with_hook::h75cd912a39a34e8a /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/panicking.rs:709:13
        #6 0x7fdc81a5ac06 in std::panicking::begin_panic_handler::_$u7b$$u7b$closure$u7d$$u7d$::h1498b46f7849e167 /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/panicking.rs:597:13
        #7 0x7fdc81a57ec5 in std::sys_common::backtrace::__rust_end_short_backtrace::hd36a39b27b98086b /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sys_common/backtrace.rs:151:18
        #8 0x7fdc81a5a951 in rust_begin_unwind /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/panicking.rs:593:5
        #9 0x7fdc81aba632 in core::panicking::panic_fmt::h98ef273141454c23 /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/panicking.rs:67:14
        #10 0x7fdc81ababc0 in core::panicking::assert_failed_inner::hb4b889049d545ac5 /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/panicking.rs
        #11 0x7fdc7fa967d6 in core::panicking::assert_failed::h2ce2b84e2f90e695 /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/panicking.rs:229:5
        #12 0x7fdc7fb4e1ed in wgpu_core::storage::Storage$LT$T$C$I$GT$::get::h871b33a5824861ab /third_party/rust/wgpu-core/src/storage.rs:111:9
        #13 0x7fdc7fb13c30 in wgpu_core::device::global::_$LT$impl$u20$wgpu_core..global..Global$LT$G$GT$$GT$::render_pipeline_get_bind_group_layout::hd5c9833d4dc3048f /third_party/rust/wgpu-core/src/device/global.rs:1928:38
        #14 0x7fdc7fb73ce3 in wgpu_bindings::server::Global::device_action::hdb477728c8072fd3 /gfx/wgpu_bindings/src/server.rs:684:34
        #15 0x7fdc7fb8a755 in wgpu_server_device_action /gfx/wgpu_bindings/src/server.rs:922:5
        #16 0x7fdc79de41d4 in mozilla::webgpu::WebGPUParent::RecvDeviceAction(unsigned long, mozilla::ipc::ByteBuf const&) /dom/webgpu/ipc/WebGPUParent.cpp:1259:3
        #17 0x7fdc79defb1b in mozilla::webgpu::PWebGPUParent::OnMessageReceived(IPC::Message const&) /builds/worker/workspace/obj-build/ipc/ipdl/PWebGPUParent.cpp:276:80
        #18 0x7fdc77e6babd in mozilla::gfx::PCanvasManagerParent::OnMessageReceived(IPC::Message const&) /builds/worker/workspace/obj-build/ipc/ipdl/PCanvasManagerParent.cpp:269:32
        #19 0x7fdc773dbc9f in mozilla::ipc::MessageChannel::DispatchAsyncMessage(mozilla::ipc::ActorLifecycleProxy*, IPC::Message const&) /ipc/glue/MessageChannel.cpp:1800:25
        #20 0x7fdc773d89f2 in mozilla::ipc::MessageChannel::DispatchMessage(mozilla::ipc::ActorLifecycleProxy*, mozilla::UniquePtr<IPC::Message, mozilla::DefaultDelete<IPC::Message>>) /ipc/glue/MessageChannel.cpp:1725:9
        #21 0x7fdc773d9672 in mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::ActorLifecycleProxy*, mozilla::ipc::MessageChannel::MessageTask&) /ipc/glue/MessageChannel.cpp:1525:3
        #22 0x7fdc773da7bf in mozilla::ipc::MessageChannel::MessageTask::Run() /ipc/glue/MessageChannel.cpp:1623:14
        #23 0x7fdc7672246d in nsThread::ProcessNextEvent(bool, bool*) /xpcom/threads/nsThread.cpp:1192:16
        #24 0x7fdc767293fd in NS_ProcessNextEvent(nsIThread*, bool) /xpcom/threads/nsThreadUtils.cpp:480:10
        #25 0x7fdc773e2e55 in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /ipc/glue/MessagePump.cpp:330:5
        #26 0x7fdc772fbbf1 in RunHandler /ipc/chromium/src/base/message_loop.cc:363:3
        #27 0x7fdc772fbbf1 in MessageLoop::Run() /ipc/chromium/src/base/message_loop.cc:345:3
        #28 0x7fdc7671d753 in nsThread::ThreadFunc(void*) /xpcom/threads/nsThread.cpp:370:10
        #29 0x7fdc8b51dd0f in _pt_root /nsprpub/pr/src/pthreads/ptthread.c:201:5
        #30 0x7fdc8bdbeac2 in start_thread nptl/pthread_create.c:442:8
        #31 0x7fdc8be50a3f  misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
    
    UndefinedBehaviorSanitizer can not provide additional info.
    SUMMARY: UndefinedBehaviorSanitizer: SEGV /builds/worker/workspace/obj-build/dist/include/mozilla/Assertions.h:281:3 in MOZ_Crash
    ==65278==ABORTING

Comment 1

[Jason Kratzer [:jkratzer]]

Attachment: Testcase

Comment 2

[Tyson Smith [:tsmith]]

NOTE: This is a major fuzzblocker. It is by far the top reported issue and has multiple signatures. Please prioritize it appropriately.

I am assuming this is a dupe or at least related to bug 1861985 since we started seeing it at the same time and is high volume.

Comment 3

[Bugmon [:jkratzer for issues]]

Verified bug as reproducible on mozilla-central 20231101093520-b73ef4c8979f.
The bug appears to have been introduced in the following build range:

Start: e0dd0b10e8fd0ea751f11fb0a6548ad9b6780e16 (20231016153418)
End: fa12efd7ca249d06b27ea86690ae0d0478f5dcce (20231016182434)
Pushlog: https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=e0dd0b10e8fd0ea751f11fb0a6548ad9b6780e16&tochange=fa12efd7ca249d06b27ea86690ae0d0478f5dcce

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1838693

Comment 5

[Brad Werth [:bradwerth]]

This is affected by, but not wholly fixed by the wgpu changes in https://github.com/gfx-rs/wgpu/pull/4624. I'm still trying to figure out what kind of device resource exhaustion is triggering the crash in this testcase.

Comment 6

[Brad Werth [:bradwerth]]

Typo in code, easy to fix. ComputePipeline::GetBindGroupLayout is using the wrong method, requesting a bind group layout from a render pipeline, not a compute pipeline.

Comment 7

[Brad Werth [:bradwerth]]

Attachment: Bug 1862499: Make ComputePipeline::GetBindGroupLayout request from the compute pipeline.

This is just a fixup for a typo. The method was originally implemented
requesting a bind group layout from the render pipelines, not from the
compute pipelines, as intended.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/180ec2f72f11
Make ComputePipeline::GetBindGroupLayout request from the compute pipeline. r=webgpu-reviewers,ErichDonGubler

Comment 9

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/180ec2f72f11

Comment 10

[Bugmon [:jkratzer for issues]]

Verified bug as fixed on rev mozilla-central 20231104091937-fa8ebe703963.
Removing bugmon keyword as no further action possible. Please review the bug and re-add the keyword for further analysis.