Closed Bug 2049803 Opened 4 months ago Closed 4 months ago

WebGPU stale end_of_pass_timer_query causes vkCmdWriteTimestamp on freed vk::QueryPool (heap UAF in GPU process)

Categories

(Core :: Graphics: WebGPU, defect, P1)

Unspecified
Linux
defect

Tracking

()

RESOLVED FIXED
154 Branch
Tracking Status
firefox-esr115 --- disabled
firefox-esr140 --- disabled
firefox-esr153 154+ fixed
firefox152 --- disabled
firefox153 --- disabled
firefox154 + fixed

People

(Reporter: tjr, Assigned: aleiserson)

References

(Blocks 1 open bug)

Details

(6 keywords, Whiteboard: [adv-esr153.1-])

Attachments

(5 files)

wgpu-hal's Vulkan CommandEncoder caches a raw, non-refcounted vk::QueryPool handle in end_of_pass_timer_query when a render/compute pass is begun with timestampWrites.endOfPassWriteIndex. The field is consumed by end_{render,compute}_pass via take(), but neither begin_encoding nor discard_encoding resets it (in contrast to rpass_debug_marker_active, which begin_encoding resets explicitly with the comment "Reset this in case the last renderpass was never ended"). In wgpu-core, encode_compute_pass calls HAL begin_compute_pass (setting the field) and then iterates recorded commands; any validation failure inside the loop (e.g. DispatchError::MissingPipeline) propagates with ? before HAL end_compute_pass runs. The owning CommandBufferMutable is then dropped during CommandEncoder::finish, whose InnerCommandEncoder::drop calls discard_encoding and pushes the boxed HAL encoder back into the per-device LIFO CommandAllocator pool with the stale handle still set.

Web content can therefore: (1) begin a compute pass with timestampWrites:{querySet:qs, endOfPassWriteIndex:0} and a dispatchWorkgroups with no pipeline so the pass fails after caching qs.raw; (2) drop qs so vkDestroyQueryPool frees the driver object; (3) create a second encoder which pops the same HAL encoder from the pool, begin a compute pass without timestampWrites (so the field is not reassigned), end() it (which records vkCmdWriteTimestamp(freed_pool, 0)), and submit(). When the driver executes the command buffer it dereferences the freed query-pool object.

This is reachable from unmodified web content using the spec timestamp-query feature (implemented in Adapter.cpp). On a Linux ASAN build with Mesa lavapipe the result is a SEGV in libvulkan_lvp.so with r14 = 0xe5e5e5e5e5e5e5e5 — the ASAN free_fill_byte=229 poison pattern — i.e. a heap use-after-free inside the (uninstrumented) driver. The PWebGPU actor is [ParentProc=compositor], so the victim is the GPU process (in-parent fallback on Linux, dedicated GPU process on Windows/Android). A control run where the first pass is valid (so end_compute_pass clears the field) does not crash, confirming the stale field as the root cause.

Build Info

Affected Code

File: third_party/rust/wgpu-hal/src/vulkan/mod.rs, line 1017-1021

    rpass_debug_marker_active: bool,

    /// If set, the end of the next render/compute pass will write a timestamp at
    /// the given pool & location.
    end_of_pass_timer_query: Option<(vk::QueryPool, u32)>,

File: third_party/rust/wgpu-hal/src/vulkan/command.rs, line 122-173

unsafe fn begin_encoding(&mut self, label: crate::Label) -> Result<(), crate::DeviceError> {
    ...
    // Reset this in case the last renderpass was never ended.
    self.rpass_debug_marker_active = false;
    // [BUG] end_of_pass_timer_query is NOT reset here
    ...
}

unsafe fn discard_encoding(&mut self) {
    assert_ne!(self.active, vk::CommandBuffer::null());
    self.discarded.push(self.active);
    self.active = vk::CommandBuffer::null();
    // [BUG] end_of_pass_timer_query is NOT reset here either
}

File: third_party/rust/wgpu-hal/src/vulkan/command.rs, line 1303-1331

unsafe fn begin_compute_pass(&mut self, desc: &crate::ComputePassDescriptor<'_, super::QuerySet>) {
    self.bind_point = vk::PipelineBindPoint::COMPUTE;
    ...
    if let Some(timestamp_writes) = desc.timestamp_writes.as_ref() {
        ...
        self.end_of_pass_timer_query = timestamp_writes
            .end_of_pass_write_index
            .map(|index| (timestamp_writes.query_set.raw, index));
    }
    // [BUG] when desc.timestamp_writes is None the stale value is left intact
}
unsafe fn end_compute_pass(&mut self) {
    self.write_pass_end_timestamp_if_requested();
    ...
}

File: third_party/rust/wgpu-hal/src/vulkan/command.rs, line 42-53

fn write_pass_end_timestamp_if_requested(&mut self) {
    if let Some((query_set, index)) = self.end_of_pass_timer_query.take() {
        unsafe {
            self.device.raw.cmd_write_timestamp(
                self.active,
                vk::PipelineStageFlags::BOTTOM_OF_PIPE,
                query_set,   // <-- stale, freed vk::QueryPool
                index,
            );
        }
    }
}

File: third_party/rust/wgpu-core/src/command/compute.rs, line 716-842

unsafe {
    state.pass.base.raw_encoder.begin_compute_pass(&hal_desc);  // sets end_of_pass_timer_query
}

for command in base.commands.drain(..) {
    match command {
        ...
        ArcComputeCommand::DispatchWorkgroups(groups) => {
            let scope = PassErrorScope::Dispatch { indirect: false };
            dispatch_workgroups(&mut state, groups).map_pass_err(scope)?;  // <-- early return on MissingPipeline
        }
        ...
    }
}
...
unsafe {
    state.pass.base.raw_encoder.end_compute_pass();  // never reached on error
}

File: third_party/rust/wgpu-core/src/command/mod.rs, line 786-797

impl Drop for InnerCommandEncoder {
    fn drop(&mut self) {
        if self.is_open {
            unsafe { self.raw.discard_encoding() };
        }
        unsafe { self.raw.reset_all(mem::take(&mut self.list)); }
        let raw = unsafe { ManuallyDrop::take(&mut self.raw) };
        self.device.command_allocator.release_encoder(raw);  // back to LIFO pool with stale field
    }
}

File: third_party/rust/wgpu-core/src/command/allocator.rs, line 32-51

pub(crate) fn acquire_encoder(...) -> Result<Box<dyn hal::DynCommandEncoder>, hal::DeviceError> {
    let mut free_encoders = self.free_encoders.lock();
    match free_encoders.pop() {           // LIFO -> next createCommandEncoder reuses it
        Some(encoder) => Ok(encoder),
        None => unsafe { ... device.create_command_encoder(&hal_desc) },
    }
}
pub(crate) fn release_encoder(&self, encoder: Box<dyn hal::DynCommandEncoder>) {
    self.free_encoders.lock().push(encoder);
}

begin_encoding resets rpass_debug_marker_active but not end_of_pass_timer_query; begin_compute_pass only assigns the field inside the Some branch; encode_compute_pass bails out via ? before end_compute_pass; the HAL encoder is returned to a per-device pool on drop.

Exploit Chain

  1. Attacker page calls navigator.gpu.requestAdapter() / requestDevice({requiredFeatures:['timestamp-query']}) — feature is web-exposed (dom/webgpu/Adapter.cpp:211).
  2. Create qs = device.createQuerySet({type:'timestamp', count:2}); GPU process allocates a vk::QueryPool.
  3. Create encoder E1, beginComputePass({timestampWrites:{querySet:qs, endOfPassWriteIndex:0}}), dispatchWorkgroups(1) WITHOUT setPipeline, end(), finish(). In the GPU process, encode_compute_pass calls HAL begin_compute_pass (caching qs.raw in end_of_pass_timer_query), then the dispatch command fails is_ready() with MissingPipeline and ?-returns before end_compute_pass.
  4. E1's CommandBufferMutable is dropped inside finish(); InnerCommandEncoder::drop calls discard_encoding (does not clear the field) and pushes the HAL encoder back into device.command_allocator. The encoder's tracker (which held the only extra Arc<QuerySet>) is also dropped.
  5. Drop the JS reference to qs and force GC -> content sends DeviceAction::DropQuerySet over PWebGPU -> wgpu-core drops the last Arc<QuerySet> -> vkDestroyQueryPool frees the driver object that end_of_pass_timer_query still points at.
  6. Optionally spray device.createQuerySet({type:'timestamp', count:2}) to reclaim the freed driver allocation with attacker-influenced contents.
  7. Create encoder E2 = device.createCommandEncoder() — acquire_encoder pops the same HAL encoder from the LIFO pool. beginComputePass() with NO timestampWrites leaves end_of_pass_timer_query untouched; end() calls write_pass_end_timestamp_if_requested which records vkCmdWriteTimestamp(freed_pool, 0) into E2's command buffer.
  8. device.queue.submit([e2.finish()]) executes the command buffer; the Vulkan driver dereferences the freed query-pool object on its worker thread -> heap use-after-free in the GPU/compositor process.

IPC Path

  • PWebGPU (via PCanvasManager)
    • Protocol: PWebGPU
    • Parent process: GPU
    • Child process: Content
    • Message flow: Content -> GPU: DeviceAction(CreateQuerySet/CreateCommandEncoder/DropQuerySet) and CommandEncoderFinish/QueueSubmit driving wgpu-core in the compositor

Steps to Reproduce

  1. Build Firefox with --enable-address-sanitizer --enable-fuzzing (mozconfig.linux.asan.fuzzing).
  2. Install Mesa lavapipe and create a wrapper that sets VK_ICD_FILENAMES / VK_DRIVER_FILES to the lavapipe ICD JSON and prepends its lib dir to LD_LIBRARY_PATH before exec'ing dist/bin/firefox (any Vulkan driver works; lavapipe lets the UAF be observed headless).
  3. Set prefs: dom.webgpu.enabled=true, gfx.webgpu.ignore-blocklist=true, fuzzing.enabled=true.
  4. Load testcase.html (creates a timestamp query set, runs a compute pass with endOfPassWriteIndex and no pipeline, GCs the query set, then runs and submits a second compute pass with no timestampWrites).
  5. Observe ASAN SEGV in libvulkan_lvp.so on the compositor's worker thread with r14=0xe5e5e5e5e5e5e5e5 (ASAN free_fill_byte=229 poison) and the preceding 'Compute pipeline must be set' validation warning in stderr.

Security Impact

  • Severity: High
  • Attacker capability: Web content triggers a heap use-after-free of a Vulkan driver query-pool object inside the GPU/compositor process. The freed allocation can be reclaimed with attacker-influenced data (createQuerySet spray) before vkCmdWriteTimestamp dereferences and writes through it, giving a driver-dependent memory-corruption primitive in the GPU process and a stepping stone toward GPU-process code execution / sandbox-escape chaining.
  • Preconditions: WebGPU enabled (default on Windows and macOS-aarch64; default on Nightly/early-Beta Linux) with a Vulkan backend whose adapter exposes the standard 'timestamp-query' feature (lavapipe/RADV/ANV/NVIDIA). No source patches, no user interaction; FuzzingFunctions is used only to make GC deterministic.

ASAN Report

=================================================================
==465425==ERROR: AddressSanitizer: SEGV on unknown address (pc 0x7bb7dcdd3929 bp 0x7bb7dc9feb20 sp 0x7bb7dc9feaf0 T88)
==465425==The signal is caused by a READ memory access.
==465425==Hint: this fault was caused by a dereference of a high value address (see register values below).  Disassemble the provided pc to learn which register was used.
    #0 0x7bb7dcdd3929 in std::_UninitDestroyGuard<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, void>::~_UninitDestroyGuard() ??:?
    #1 0x7bb7dcdc005c in std::_UninitDestroyGuard<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, void>::~_UninitDestroyGuard() ??:?
    #2 0x7bb7dcaab35e in vk_icdGetInstanceProcAddr ??:?
    #3 0x7bb7dcaae87a in vk_icdGetInstanceProcAddr ??:?
    #4 0x7bb7dca9a8eb in vk_icdGetPhysicalDeviceProcAddr ??:?
    #5 0x7bb7dca2fd9a in ?? ??:0
    #6 0x7bb7dca303cc in ?? ??:0
    #7 0x7bb7dcb9301b in std::vector<unsigned char, std::allocator<unsigned char> >::_M_default_append(unsigned long) ??:?
    #8 0x563bb37f0e26 in asan_thread_start(void*) _asan_rtl_:28
    #9 0x7fb8a86ce904 in pthread_condattr_setpshared ??:?
    #10 0x7fb8a873d00b in __clone ??:?

==465425==Register values:
rax = 0x00007bb7fbe068f8  rbx = 0x00007e88aa55a400  rcx = 0x0000000000000000  rdx = 0x00007bb7dd5a5580
rdi = 0x0000000000000010  rsi = 0x00007bb7dce9472d  rbp = 0x00007bb7dc9feb20  rsp = 0x00007bb7dc9feaf0
 r8 = 0x00007bb7dce8ea44   r9 = 0x00007bb7fbe068f8  r10 = 0x00000000000001f8  r11 = 0x0000000000000000
r12 = 0x00007c78a8148280  r13 = 0x00007bb7dcf316ac  r14 = 0xe5e5e5e5e5e5e5e5  r15 = 0x0000000000000001
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (/work/scratch/mesa/usr/lib64/libvulkan_lvp.so+0x3d3929) (BuildId: 8189e9ce072907025a6bca78b99ca7f98e3ef2fb)
Thread T88 created by T61 here:
    #0 0x563bb37d8fd1 in ___interceptor_pthread_create _asan_rtl_:3
    #1 0x7bb7dcb93224 in std::vector<unsigned char, std::allocator<unsigned char> >::_M_default_append(unsigned long) ??:?
    #2 0x7bb7dca2eb28 in ?? ??:0
    #3 0x7bb7dca30b1c in ?? ??:0
    #4 0x7bb7dca9c952 in vk_icdGetInstanceProcAddr ??:?
    #5 0x7bb7fda503c2 in ?? ??:0
    #6 0x7bb7fda5b4bd in ?? ??:0
    #7 0x7bb7fda5c3b9 in ?? ??:0
    #8 0x7bb7fda65f09 in vkCreateDevice ??:0:0

Thread T61 created by T0 here:
    #0 0x563bb37d8fd1 in ___interceptor_pthread_create _asan_rtl_:3
    #1 0x7fb8a81893c8 in _PR_CreateThread nsprpub/pr/src/pthreads/ptthread.c:428:10
    #2 0x7fb8a817765e in PR_CreateThread nsprpub/pr/src/pthreads/ptthread.c:495:10
    #3 0x7bb87c3aaa80 in nsThread::Init(nsTSubstring<char> const&) xpcom/threads/nsThread.cpp:623:20
    #4 0x7bb87c3b7c35 in nsThreadManager::NewNamedThread(...) xpcom/threads/nsThreadManager.cpp:618:22
    #5 0x7bb87c3c34df in NS_NewNamedThread(...) xpcom/threads/nsThreadUtils.cpp:186:57
    #6 0x7bb87e8e36de in mozilla::gfx::CanvasRenderThread::Start() gfx/ipc/CanvasRenderThread.cpp:80:17
    #7 0x7bb87e68fddf in gfxPlatform::Init() gfx/thebes/gfxPlatform.cpp:995:3

==465425==ABORTING

[r14 = 0xe5e5e5e5e5e5e5e5 is the ASAN free_fill_byte=229 poison; lavapipe is not ASAN-instrumented so the UAF surfaces as a SEGV on the poisoned pointer rather than a heap-use-after-free report. crashed_parent=true because the compositor runs in-process on Linux; classify as GPU process per the [ParentProc=compositor] annotation.]

Preceding stderr:
[Child 474347: Main Thread]: W/PageMessages [JavaScript Warning: "Uncaptured WebGPU error: In a dispatch command, indirect:false, caused by: Compute pipeline must be set"]
Attached file trace.jsonl —
Attached file testcase.html —
Attached file prefs.js —
Attached file firefox-wrapper.sh —
Attached file crash_stack.txt —
Attached patch fix.patch — — Splinter Review
Group: core-security → firefox-core-security
Group: firefox-core-security → gfx-core-security
Component: General → Graphics: WebGPU
Product: Firefox Build System → Core

Setting severity S2 / priority P1.

sec-high heap use-after-free reachable from unmodified web content via the spec timestamp-query feature, crashing the GPU/parent process. wgpu-hal's Vulkan CommandEncoder caches a raw, non-refcounted vk::QueryPool handle in end_of_pass_timer_query; begin_encoding/discard_encoding never reset it (unlike rpass_debug_marker_active). A compute pass that fails validation mid-loop (e.g. MissingPipeline) recycles the encoder into the per-device pool with the stale handle set. Content can then drop the QuerySet (freeing the driver pool), pop the same encoder for a new pass without timestampWrites, end() it (records vkCmdWriteTimestamp(freed_pool)), and submit() -- the driver dereferences the freed object. ASAN-confirmed (SEGV in libvulkan_lvp.so, r14 = 0xe5e5e5e5e5e5e5e5 free-fill poison); a control run with a valid first pass does not crash, confirming the stale field as root cause. Deterministic (not a race), no user workaround. Comparable to the accepted-S2 content-process UAF bug 2022381 and WebGPU bug 2018451. Not S1: driver query-pool UAF, sec-high not sec-critical, escalation needs heap grooming.

Adding See Also -> bug 2049799, a sibling WebGPU GPU-process bug from the same batch (distinct root cause: that one is the override @workgroup_size device-limit validation bypass).

Jim, needinfo for review. A fix.patch is attached and looks correct/minimal: it resets end_of_pass_timer_query = None in begin_encoding, and changes begin_render_pass/begin_compute_pass to always assign the field (clearing it when timestampWrites is None) rather than only inside the if let Some(...) arm. Two questions: (1) should discard_encoding also defensively clear the field for belt-and-suspenders, and (2) should this land upstream in wgpu? (The 17.5 KB diff is mostly the .cargo-checksum.json regen; the actual code change is small.)

Note: filed by clauditor (AI tooling); crash is ASAN-confirmed and severity human-reviewed, but please confirm the analysis/patch before landing.

Severity: -- → S2
Flags: needinfo?(jimb)
Priority: -- → P1

Relevant code was changed recently in https://github.com/gfx-rs/wgpu/pull/9671, but that won't fix this issue.

OS: Unspecified → Linux

Stale endOfPassWriteIndex is an issue in all three backends, but because dx12 and Metal resources are reference counted, it is only a security issue on Vulkan.

Assignee: nobody → aleiserson
Flags: needinfo?(jimb) → needinfo?(aleiserson)

Moved needinfo since Andy is working on this bug.

Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Flags: needinfo?(aleiserson)
Depends on: 2052109
No longer depends on: webgpu-update-wgpu

This should be fixed by the most recent wgpu update (bug 2052109).

Status: ASSIGNED → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED
Group: gfx-core-security → core-security-release
QA Whiteboard: [qa-triage-done-c155/b154][sec]
Target Milestone: --- → 154 Branch
See Also: → 2059531
Whiteboard: [adv-esr153.1-]
Group: core-security-release
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: