Closed Bug 2046660 Opened 5 months ago Closed 5 months ago

Fix Vulkan hw decode shutdown ordering: wait for compositor release, Cleanup() must run before av_buffer_unref destroys the VkDevice

Categories

(Core :: Audio/Video: Playback, defect)

defect

Tracking

()

RESOLVED FIXED
153 Branch
Tracking Status
firefox-esr140 --- unaffected
firefox151 --- unaffected
firefox152 --- unaffected
firefox153 --- fixed

People

(Reporter: tboiko, Assigned: tboiko)

References

(Regression)

Details

(4 keywords)

Crash Data

Attachments

(1 external file)

Steps to reproduce:

Build Firefox with Vulkan hardware video decode enabled (Bug 2021722), e.g. set media.hardware-video-decoding-vulkan.enabled to true.
Open YouTube (or another site using hw-decoded video with DMA-BUF export).
Confirm Vulkan hw decode is active.
Navigate quickly between videos/content pages, or close the tab while playback is active.

Actual results:

During FFmpegVideoDecoder::ProcessShutdown(), teardown order can be wrong:

mVideoFramePool is dropped before waiting for the compositor.
avcodec_free_context() runs (via ProcessShutdown()).
av_buffer_unref(&mVulkanDeviceContext) runs before mVulkanDecoder.Cleanup().
If step 3 is the last reference to the AVHWDeviceContext, FFmpeg calls vkDestroyDevice immediately. Step 4 then calls vkDeviceWaitIdle() and destroys Firefox-owned Vulkan objects (command pools, fences, copy-ring resources) on an already-destroyed VkDevice — undefined behavior per the Vulkan spec.

Expected results:

Shutdown should:

Wait for the compositor to release exported DMA-BUF surfaces (and copy-ring slots).
Run avcodec_free_context() so FFmpeg finishes its own Vulkan teardown (ff_vk_uninit) first.
Call mVulkanDecoder.Cleanup() while mVulkanDeviceContext still keeps the VkDevice alive.
Only then call av_buffer_unref(&mVulkanDeviceContext)

Wait for the compositor to release DMA-BUF surfaces before destroying
Vulkan objects in ProcessShutdown(). Keep mVideoFramePool alive until
after Cleanup() so copy-ring slot tracking remains valid. Add
WaitUntilSurfacesReleasedByRenderer() / WaitUntilCopyRingReleasedByRenderer()
that poll the cross-process eventfd refcount before teardown.

Assignee: nobody → tboiko
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true

I think we should swap Cleanup() and av_buffer_unref. Cleanup() must run before av_buffer_unref destroys the VkDevice

The rest of the changes may be unnecessary, I will double check

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Component: Untriaged → Audio/Video: Playback
Product: Firefox → Core
Attachment #9596295 - Attachment description: Bug 2046660 - DOM Media - Fix Vulkan decoder shutdown ordering to avoid use-after-free. r=stransky,alwu → Bug 2046660 - DOM Media - Fix Vulkan decoder shutdown ordering. r=stransky,alwu

Hi Martin,
we also found that remote Youtube video was not working reliably (fallbacks to CPU decoding). I assume this happened after we removed some whitelisted signals in RDD Sandbox just before landing the patch. With MOZ_RDD_DISABLE_SANBOX=1 everything works reliably again. I will look into this issue and send another patch soon.

Status: ASSIGNED → RESOLVED
Closed: 5 months ago
Resolution: --- → FIXED
Target Milestone: --- → 153 Branch

Thank you for the fix! I'm adding a RDD crash signature that I confirmed was linked to this bug.

I'm also marking this as a security bug because the crashes are use-after-free and the trigger (start hw-decoded video, then navigate away or close the tab during playback) is reachable from ordinary web content.

Could someone with sec-triage authority assign a security rating? Factors for the rater's consideration:

  • The free (vkDestroyDevice) and the bad dispatch (Cleanup()) are back-to-back synchronous calls on the same TaskQueue thread, so there is effectively no window for an attacker-controlled allocation to reoccupy the freed memory. The diverse, unmapped faulting addresses are consistent with reading a freed-but-still-largely-intact dispatch table, not an attacker-placed value.
  • It occurs in the RDD process.

(Edit: Removed some too-broad claims that I need to reconfirm.)

Group: core-security
Crash Signature: [@ FFmpegVideoDecoder<T>::ProcessShutdown]
Keywords: regression
Regressed by: 2021722
Crash Signature: [@ FFmpegVideoDecoder<T>::ProcessShutdown] → [@ mozilla::FFmpegVideoDecoder<T>::ProcessShutdown]

Set release status flags based on info from the regressing bug 2021722

Martin, Tymur, do you know if this feature is enabled for real users on some distributions? With the pref being off by default, I'm struggling to understand why we get this crash volume.

Flags: needinfo?(tboiko)
Flags: needinfo?(stransky)

Hi Yannis,
My understanding is:
The pref is off by default in upstream Firefox. However, some distributions may ship Firefox with the pref enabled (e.g. using mozilla.cfg, etc). Users can also enable it manually via about:config (It's distribution-patched or manually enabled installs, not the default upstream config. I saw the similar https://bugzilla.mozilla.org/show_bug.cgi?id=2047016

Flags: needinfo?(tboiko)
Group: core-security → media-core-security

(In reply to Yannis Juglaret [:yannis] from comment #10)

Martin, Tymur, do you know if this feature is enabled for real users on some distributions? With the pref being off by default, I'm struggling to understand why we get this crash volume.

It was advertised here: https://www.phoronix.com/news/Firefox-Vulkan-Video-Merged

Flags: needinfo?(stransky)
Group: media-core-security → core-security-release
QA Whiteboard: [qa-triage-done-c154/b153][sec]
Duplicate of this bug: 2049260

Copying crash signatures from duplicate bugs.

Crash Signature: [@ mozilla::FFmpegVideoDecoder<T>::ProcessShutdown] → [@ mozilla::FFmpegVideoDecoder<T>::ProcessShutdown] [@ <name omitted> | vk_common_DeviceWaitIdle] [@ ac_init_tracked_regs] [@ libvulkan_radeon.so] [@ mozilla::FFmpegVideoDecoder<T>::FFmpegVulkanVideoDecoder::Cleanup] [@ vk_queue_to_handle]
Duplicate of this bug: 2050361

Copying crash signatures from duplicate bugs.

Crash Signature: [@ mozilla::FFmpegVideoDecoder<T>::ProcessShutdown] [@ <name omitted> | vk_common_DeviceWaitIdle] [@ ac_init_tracked_regs] [@ libvulkan_radeon.so] [@ mozilla::FFmpegVideoDecoder<T>::FFmpegVulkanVideoDecoder::Cleanup] [@ vk_queue_to_handle] → [@ mozilla::FFmpegVideoDecoder<T>::ProcessShutdown] [@ <name omitted> | vk_common_DeviceWaitIdle] [@ ac_init_tracked_regs] [@ libvulkan_radeon.so] [@ mozilla::FFmpegVideoDecoder<T>::FFmpegVulkanVideoDecoder::Cleanup] [@ vk_queue_to_handle] [@ vk_optimusGe…
See Also: → CVE-2026-84123
Group: core-security-release
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: