1839509 - [wmfme] SWDRM video always shows block frames on my secondary monitor

Status: VERIFIED FIXED

(Core :: Audio/Video: Playback, defect, P2)

Description

[Alastor Wu [:alwu]]

Attachment: about:support

STR.

0. have two monitors connected to your computer
1. on the latest Nightly
2. set the pref media.eme.playready.enabled=true, media.wmf.media-engine.enabled=1
3. go to https://refapp.hbbtv.org/dash/test_dashjs_sl3000.html?video=sl2000b
4. play the video on the primary monitor and make sure it can display correctly
5. drag the entire window to another monitor

Expected.

5. video still displays correctly

Actual

5. video becomes black frame

---

Note, for my setup, there is only one monitor (display0, see my about:support) can display the video correctly. Video always display black frame on my second monitor. This issue only happens on DRM video via the media engine playback, which uses Direct Composition surface. But this issue won't happen on non-encrypted playback. If I turn off my primary monitor to allow only the secondary monitor working, then the video would be able to display. So the problem only happens when there are multiple monitors.

However, on my another laptop, which supports PlayReady HWDRM. (You can check that by using Edge, visiting edge://media-internals/ >> CDMs. Then you can see if there is any capabilities for Hardware Secure) When you play this video which uses HWDRM, then this issue WON'T happen. Both monitors can display the video correctly. But if you play other videos using SWDRM, then this issue would happen again.

Comment 1

[Alastor Wu [:alwu]]

Hello Sotaro,

Could you help me check this issue as well? I want to know whether we are missing something while rendering the Dcomp surface, but I don't know how to verify that. If we can make sure this is not the Dcomp problem, then I will check our media engine implementation to see if I did anything wrong.

Thank you so much!

Comment 2

[Alastor Wu [:alwu]]

Unlike software DRM (SWDRM), output protections are enforced on all monitors based on the least capable monitor. For example, if the user has two monitors connected where one of the monitors supports HDCP and the other does not, playback will fail if the license requires HDCP even if the content is only being rendered on the monitor thaat supports HDCP. In software DRM, content would play back as long as it is only being rendered on the monitor that supports HDCP.

Above information is from here. I can't reproduce this issue on Edge, and my secondary monitor can also work if I turn off the primary monitor. That should hint my secondary monitor also support HDCP. I wonder if we need to set something in the graphic setting in order to render video on both monitors.

Comment 3

[Alastor Wu [:alwu]]

It's also not related with sandbox setting. I tried to disable sandbox for the MFCDM process, but the result is still the same.

Comment 4

[Alastor Wu [:alwu]]

I suspect that the Dcomp surface is probably related with the display monitor, so maybe we need to recreate it when moving to another window? When using Edge, If you move only the half window to the secondary monitor (but still keep half window in the primary monitor), then you can observe the same issue. There would be one side black (the monitor you're moving to), and another side (where video comes from) showing the video content. But if you move the entire window to another monitor, then the video shows correctly.

Comment 5

[Alastor Wu [:alwu]]

I tried to recreate new Dcomp surface when the video is in the secondary monitor, but it didn't help. We also didn't receive any error when switching the monitor. I will keep debugging to see what I can find.

Comment 6

[Sotaro Ikeda [:sotaro]]

I confirmed the problem. The video seemed to be shown only on main display. When I changed main display to 2nd display from Windows's setting, video was shown only on 2nd display.

Comment 7

[Sotaro Ikeda [:sotaro]]

From comment 6, I suspect that ::SetWindowPos() might be related like MediaFoundationRenderer::SetOutputRect()

• https://source.chromium.org/chromium/chromium/src/+/main:media/renderers/win/media_foundation_renderer.cc;l=662

And I manually added calling ::SetWindowPos() in MFMediaEngineParent::InitializeVirtualVideoWindow() after creating mVirtualVideoWindow with rect of 2nd display. Then the video was shown in 2nd display. The rect might be simply used to detect belonging display.

By the way, for some reason pref media.wmf.media-engine.enabled was sometimes reset to 0 during testing.

Comment 8

[Alastor Wu [:alwu]]

(In reply to Sotaro Ikeda [:sotaro] from comment #7)

From comment 6, I suspect that ::SetWindowPos() might be related like MediaFoundationRenderer::SetOutputRect()

• https://source.chromium.org/chromium/chromium/src/+/main:media/renderers/win/media_foundation_renderer.cc;l=662

And I manually added calling ::SetWindowPos() in MFMediaEngineParent::InitializeVirtualVideoWindow() after creating mVirtualVideoWindow with rect of 2nd display. Then the video was shown in 2nd display. The rect might be simply used to detect belonging display.

Thank you for help! I will try to add that. But do you know why adding that can solve the problem? Do we need to set it everytime when the video moving to another monitor?

By the way, for some reason pref media.wmf.media-engine.enabled was sometimes reset to 0 during testing.

I also experienced this weird behavior, but not sure what causes that...

Comment 9

[Alastor Wu [:alwu]]

(In reply to Alastor Wu [:alwu] from comment #8)

By the way, for some reason pref media.wmf.media-engine.enabled was sometimes reset to 0 during testing.

I also experienced this weird behavior, but not sure what causes that...

Ah, maybe related with this! Filed bug 1839938.

Comment 10

[Alastor Wu [:alwu]]

(In reply to Sotaro Ikeda [:sotaro] from comment #7)

And I manually added calling ::SetWindowPos() in MFMediaEngineParent::InitializeVirtualVideoWindow() after creating mVirtualVideoWindow with rect of 2nd display. Then the video was shown in 2nd display. The rect might be simply used to detect belonging display.

I tried to add ::SetWindowPos() after this line, for the rect size, I tried to simply give it 0 or any arbitrary value, but none of them worked. No matter I started video in the secondary monitor or moving it from the primary monitor to the secondary one, in both cases I still only saw black frames.

Did I do something wrong? Would you mind to share your patch to me? Thank you!

Comment 11

[Sotaro Ikeda [:sotaro]]

Attachment: patch - Add log

I got a window rect size from logout with the patch.

Comment 12

[Sotaro Ikeda [:sotaro]]

With Attachment 9340602 [details] [diff], I manually got and set window rect of window in 2nd display . I did re-build to update the rect of mVirtualVideoWindow just for testing.

Comment 13

[Alastor Wu [:alwu]]

Attachment: Bug 1839509 - no need to use virtual window, because the media engine would determine where the video should be rendered.

The reason of using this virtual window is for Output Protection
Manager (OPM). However, based on [1], "If your application uses the
protected media path (PMP) to play video content, you do not have to use
the OPM API, because the PMP makes all of the required OPM calls", we
can acutally play DRM video correctly without using the virtual window.

I tested it on SWDRM and HWDRM, they both work successfully. Therefore,
we can remove the virtual window usage completely.

[1]
https://learn.microsoft.com/en-us/windows/win32/medfound/using-output-protection-manager

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/077875ace55f
no need to use virtual window, because the media engine would determine where the video should be rendered. r=jolin

Comment 15

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/077875ace55f

Comment 16

[Timea Zsoldos [:zstimi/tzsoldos], Desktop QA]

I have reproduced this issue using Firefox 116.0a1 (2023.06.20) on Win 10 x64.
I can confirm this issue is fixed, I verified using Firefox 116.0 - build2 on Win 10 x64, setting the pref's from description, video displays correctly on a secondary monitor.
I tried to verify on Ubuntu and on macOS, but there I assume are other settings or the expectations are different, because the video was not played.