1723207 - Avoid copying hardware decoded video data if possible on Windows

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, enhancement)

Description

[Jeff Muizelaar [:jrmuizel]]

This is the copy here:
https://searchfox.org/mozilla-central/rev/9c91451cc2392d942a42493fc895f5aeeddde45d/dom/media/platforms/wmf/DXVA2Manager.cpp#911

We currently reuse the device on AMD hardware and could potentially elsewhere if this turns out to work well.

Comment 1

[Jeff Muizelaar [:jrmuizel]]

On my Skylake machine, this copy takes about 2ms of GPU time for a 4k frame.

Comment 2

[Sotaro Ikeda [:sotaro]]

See also Bug 1724949 comment 7.

Comment 3

[Sotaro Ikeda [:sotaro]]

Attachment: WIP: Bug 1723207 - Avoid copying hardware decoded video data into shared texture if possible on Windows

Comment 4

[Sotaro Ikeda [:sotaro]]

These diagrams might help

• Diagram of Firefox video decoding with Microsoft Media Foundation
  • https://github.com/sotaroikeda/firefox-diagrams/blob/master/media/dom_media_WMFVideoMFTManager_97.pdf
• Diagram of Chromium video decoding with Microsoft Media Foundation
  • https://github.com/sotaroikeda/chromium-diagrams/blob/master/media_DXVAVideoDecodeAccelerator.pdf

Another diagrams

• https://github.com/sotaroikeda/firefox-diagrams/wiki/Firefox-Diagrams
• https://github.com/sotaroikeda/chromium-diagrams/wiki/Chromium-diagrams

Comment 5

[Sotaro Ikeda [:sotaro]]

Video overlay could be 3 types. And decode swap chains do not support shared resources. It causes a problem to video rendering to WebGL texture.

• RGB SwapChain overlay

• YUV SwapChain overlay

• DecodeSwapChain overlay

• https://source.chromium.org/chromium/chromium/src/+/main:media/gpu/windows/dxva_video_decode_accelerator_win.cc;l=1754

WebGL uses different D3D11Device, then we need to get SharedHandle from ID3D11Texture2D.

• https://searchfox.org/mozilla-central/rev/8f42809e51cb07aa4f5739932a06d14581e9dd4a/gfx/gl/GLBlitHelperD3D.cpp#227

Then for now, I am going to disable DecodeSwapChain overlay for using SharedHandle of ID3D11Texture2D.

Comment 6

[Sotaro Ikeda [:sotaro]]

Attachment: Bug 1723207 - Avoid copying hardware decoded video data if possible on Windows

Comment 7

[Sotaro Ikeda [:sotaro]]

Current implementation uses shared handle for ID3D11Texture2D. Then, Bug 1722987 is not necessary for now.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3ad7a42b1ad1
Avoid copying hardware decoded video data if possible on Windows r=media-playback-reviewers,gfx-reviewers,jrmuizel,alwu

Comment 9

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/3ad7a42b1ad1