1405562 - RenderDXGIYCbCrTextureHostOGL is not used with FFmpegVideoDecoder

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, defect, P1)

Description

[Sotaro Ikeda [:sotaro]]

RenderDXGIYCbCrTextureHostOGL was added by Bug 1388240. But RenderDXGIYCbCrTextureHostOGL was not used when video was decoded by FFmpegVideoDecoder.

Comment 1

[Sotaro Ikeda [:sotaro]]

Attachment: patch - Enable D3D11YCbCrImage usage with WebRender

Comment 2

[Sotaro Ikeda [:sotaro]]

When I tested attachment 8915024 [details] [diff] [review] on my Win PC(P50) with youtube https://www.youtube.com/watch?v=3tmd-ClpJxA , I faced the following problems.
- Green flickering occasionally happened to video area.
- AcquireSync failed with the log.
   + GP+[GFX1]: RenderDXGIYCbCrTextureHostOGL AcquireSync timeout, hr=0x80070057
- The performance seemed not to become better than without attachment 8915024 [details] [diff] [review].

I did not see the problem on another Win desktop pc. The problem seems to related to cpu/GPU performance.

Comment 3

[Sotaro Ikeda [:sotaro]]

:jerry, is RenderDXGIYCbCrTextureHostOGL actually used on current gecko? In which use case? Do you notice the problem like Comment 2?

Comment 4

[Jerry Shih[:jerry] (UTC+8) (inactive)]

Attachment: Test for d3d-ycbcr format texture.

Hi sotaro,
With this patch, we could test the d3d-ycbcr path from a regular h.264 video(which will use nv12 format).
I still don't know which video format could go through the d3d-ycbcr directly.

Comment 5

[Sotaro Ikeda [:sotaro]]

(In reply to Jerry Shih[:jerry] (UTC+8) from comment #4)
> Created attachment 8915950 [details] [diff] [review]
> Test for d3d-ycbcr format texture.
> 
> Hi sotaro,
> With this patch, we could test the d3d-ycbcr path from a regular h.264
> video(which will use nv12 format).
> I still don't know which video format could go through the d3d-ycbcr
> directly.

One user seems like FFmpegVideoDecoder<LIBAV_VER>::DoDecode(). It calls VideoData::CreateAndCopyData() and the CreateAndCopyData() creates D3D11YCbCrImage() on windows.
   https://dxr.mozilla.org/mozilla-central/source/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.cpp#399
   https://dxr.mozilla.org/mozilla-central/source/dom/media/MediaData.cpp#324

I confirmed that FFmpegVideoDecoder is used for youtube playback on my laptop P50. FFmpegVideoDecoder seems to be used for VP8/VP9 if hw accelerated video decoding is not used.

Comment 6

[Sotaro Ikeda [:sotaro]]

Attachment: patch - Enable D3D11YCbCrImage usage with WebRender

Comment 7

[Sotaro Ikeda [:sotaro]]

With latest m-c, attachment 8933183 [details] [diff] [review] works well. With https://www.youtube.com/watch?v=0wCC3aLXdOw , I confirmed that performance of software decoded video was improved on windows with the patch.

[4k video on P50(Win10) default]
- FPS: 26fps
- WebRender cpu(compositor) time: 40ms-37ms

[4k video on P50(Win10) with attachment 8933183 [details] [diff] [review]]
- FPS: 32fps
- WebRender cpu(compositor) time: 20ms

Comment 8

[Matt Woodrow (:mattwoodrow)]

Comment on attachment 8933183 [details] [diff] [review]
patch - Enable D3D11YCbCrImage usage with WebRender

Review of attachment 8933183 [details] [diff] [review]:
-----------------------------------------------------------------

This works, it's pretty painful for media code to have to understand these details though.

Would be nice if aAllocator had a function that returned what the underlying OS API we're targeting is, and have us return LAYERS_D3D11 from that for the WR+ANGLE case.

Advanced Layers (LayerManagerMLGPU) identifies as LAYERS_D3D11 already, for this reason. Most of the callers don't care what algorithm we're using to batch drawing, and only want to know what surface types they can use.

Can we make WR return LAYERS_D3D11 (or whatever is appropriate) from GetCompositorBackendType? Maybe add something new to identify WR for the callers that really need to know.

Comment 9

[Sotaro Ikeda [:sotaro]]

(In reply to Matt Woodrow (:mattwoodrow) from comment #8)
> Comment on attachment 8933183 [details] [diff] [review]
> patch - Enable D3D11YCbCrImage usage with WebRender
> 
> 
> Can we make WR return LAYERS_D3D11 (or whatever is appropriate) from
> GetCompositorBackendType? Maybe add something new to identify WR for the
> callers that really need to know.

We could not change as to return LAYERS_D3D11, since it is already used in another place that expects WR.

Instead, we could add a function like SupportsD3D11() to KnowsCompositor.

Comment 10

[Sotaro Ikeda [:sotaro]]

Attachment: patch - Enable D3D11YCbCrImage usage with WebRender

Addressed the comment.

Comment 11

[Sotaro Ikeda [:sotaro]]

Attachment: patch - Enable D3D11YCbCrImage usage with WebRender

Comment 12

[Sotaro Ikeda [:sotaro]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=7167a3113dc1d36171b628594b25c1b6be2c970b

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/994147e6fd70
Enable D3D11YCbCrImage usage with WebRender r=mattwoodrow

Comment 14

[Cosmin Sabou [:CosminS]]

Backed out for build bustages on obj-firefox/dist/include/mozilla/layers/KnowsCompositor.h:100:75

https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=994147e6fd70f7b6f685637156a23546704a4020&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception

https://treeherder.mozilla.org/logviewer.html#?job_id=149006143&repo=mozilla-inbound&lineNumber=10504

https://hg.mozilla.org/integration/mozilla-inbound/rev/dd549b14646ce686ad4df1a6a1f9ddb164d4eed8

Comment 15

[Sotaro Ikeda [:sotaro]]

Oh, I checked build only on windows :(

Comment 16

[Sotaro Ikeda [:sotaro]]

Attachment: patch - Enable D3D11YCbCrImage usage with WebRender

Fixed build failure.

Comment 17

[Sotaro Ikeda [:sotaro]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=d0296bbb6d588211409b731123c7262264ff156e

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/53ae8f24dcc5
Enable D3D11YCbCrImage usage with WebRender r=mattwoodrow

Comment 19

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/53ae8f24dcc5