1722447 - Add YUV SwapChain support to hardware decoded video

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, enhancement)

Description

[Sotaro Ikeda [:sotaro]]

YUV SwapChain might provide better performance than RGB SwapChain for hardware decoded video.

Comment 1

[Jeff Muizelaar [:jrmuizel]]

This sample app has good GPU performance on my Skylake desktop machine. (I adjusted the sample to do bilinear filtering)

This machine supports DXGI_FORMAT_YUY2 overlays but not DXGI_FORMAT_NV12 ones. The sample app seems to try use a DXGI_FORMAT_YUY2 SwapChain but then seems to switch to DXGI_FORMAT_B8G8R8A8_UNORM based on the results of IDXGISwapChainMedia::GetFrameStatisticsMedia

Comment 2

[Sotaro Ikeda [:sotaro]]

:jrmuizel, how do you get IDXGISwapChainMedia? I could not find an API to get it.

Comment 3

[Jeff Muizelaar [:jrmuizel]]

It looks like you can QueryInterface as IDXGIDecodeSwapChain: https://source.chromium.org/chromium/chromium/src/+/main:ui/gl/swap_chain_presenter.cc;l=1628?q=GetSwapChainMedia&ss=chromium

Comment 4

[Sotaro Ikeda [:sotaro]]

:jrmuizel, thank you for the info! I have one more question. How did you get a SwapChain to QueryInterface? The sample code dose not get the SwapChain directly. It get SurfaceHandle by calling mediaEngineEx->GetVideoSwapchainHandle(). Is there an API to get the SwapChain?
https://github.com/microsoft/media-foundation/blob/master/samples/MediaEngineDCompWin32Sample/src/media/MediaEngineWrapper.cpp#L281

Comment 5

[Jeff Muizelaar [:jrmuizel]]

It looks like Chrome does something similar:
https://source.chromium.org/chromium/chromium/src/+/main:ui/gl/swap_chain_presenter.cc;l=1149;drc=fab2b93d31954030689e2b3eafff4ca810914d58

they track the kind of Presentation here: https://source.chromium.org/chromium/chromium/src/+/main:ui/gl/swap_chain_presenter.cc;l=213;drc=fab2b93d31954030689e2b3eafff4ca810914d58

and do the switching here: https://source.chromium.org/chromium/chromium/src/+/main:ui/gl/swap_chain_presenter.cc;l=285;drc=fab2b93d31954030689e2b3eafff4ca810914d58

Comment 6

[Sotaro Ikeda [:sotaro]]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #1)

This machine supports DXGI_FORMAT_YUY2 overlays but not DXGI_FORMAT_NV12 ones. The sample app seems to try use a DXGI_FORMAT_YUY2 SwapChain but then seems to switch to DXGI_FORMAT_B8G8R8A8_UNORM based on the results of IDXGISwapChainMedia::GetFrameStatisticsMedia

Taking on riot with :jrmuizel. It was got from GPUView.

• https://docs.microsoft.com/en-us/windows-hardware/drivers/display/using-gpuview

I misunderstood the comment as the SwapChain was got by modifying the sample app :(

Comment 7

[Sotaro Ikeda [:sotaro]]

When Video processor was used for video presentation, it seemed that "Direct3d11 ID3D11VideoContext_VideoProcessorBltParameters" could be used to check video color conversion.

Comment 8

[Sotaro Ikeda [:sotaro]]

With GPUView, I tested the sample app and chrome on 2 Win10 PC(NV12 support and YUY2 support).

• Win10 PC(NV12 support): VideoProcessorBltParameters was DXGI_FORMAT_NV12 to DXGI_FORMAT_NV12
• Win10 PC(YUY2 support ): VideoProcessorBltParameters was DXGI_FORMAT_NV12 to DXGI_FORMAT_YUY2

Comment 9

[Sotaro Ikeda [:sotaro]]

Attachment: Bug 1722447 - Add YUV SwapChain support to hardware decoded video

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a97943782c68
Add YUV SwapChain support to hardware decoded video r=gfx-reviewers,nical

Comment 11

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/a97943782c68