1853448 - Support HEVC via Windows MFT

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, task, P1)

Description

[Alastor Wu [:alwu]]

We will support HEVC playback via Media Foundation transform (MFT).

Comment 1

[Alastor Wu [:alwu]]

Attachment: Bug 1853448 - part1 : support HEVC in WMF decoder module.

HEVC playback will be supported via the Media Foundation Transform (MFT)
and WMF decoder module will check if there is any avaliable MFT which
can be used for HEVC then reports the support information.

Depends on D188143

Comment 2

[Alastor Wu [:alwu]]

Attachment: Bug 1853448 - part2 : check if HEVC DXVA decoder exists.

HEVC playback can only be support on
(1) users have purchased paid HEVC extension on their computer (SW decoding)
(2) HEVC hardware decoding is available on users' computer

This patch checks whether HEVC hardware decoders exist. Decoder GUIDs
are listed in Microsoft's document [1].

[1] https://learn.microsoft.com/en-us/windows/win32/medfound/direct3d-12-video-guids

Depends on D188389

Comment 3

[Alastor Wu [:alwu]]

Attachment: Bug 1853448 - part3 : return HEVC support for GPU process.

As now we can support HEVC playback via MFT as well, we should allow GPU
process as well (HEVC hardware decoding performs only in GPU process).

If the HEVC is not supported, the playback would be abort at the stage
of creating a decoder.

Depends on D188390

Comment 4

[Alastor Wu [:alwu]]

Attachment: Bug 1853448 - part4 : modify pref to allow more flexible strategy for enabling HEVC.

For now, I'd like to only enable HEVC for the media engine playback, but
keep the HEVC default off on the MFT. Because the media engine is an
experimental feature, which is off by default, it's fine to enable HEVC
for that.

Depends on D188391

Comment 5

[Alastor Wu [:alwu]]

Attachment: Bug 1853448 - part5 : make hevc playback test for wmfme only.

Initially the plan is to make this test work on both Window MFT and the
Windows Media Foundation Media Engine (wmfme). However, the machines on
the try server don't support the hardware decoding, we can't find the decoder for HEVC.

The interesting thing is the media engine can still decode hevc on the
try server, that could be many possiblities. One guess is the hevc
SW support still exists on Windows, but it's just not exposed via the
MFT (which you have to buy paid extenstion first). Or we have other
setting on try server to prevent us from using HW decoding.

For now, let's make this test for wmfme only.

Depends on D188540

Comment 6

[:jmoss]

(In reply to Alastor Wu [:alwu] from comment #5)

Created attachment 9353811 [details]
Bug 1853448 - part5 : make hevc playback test for wmfme only.

Initially the plan is to make this test work on both Window MFT and the
Windows Media Foundation Media Engine (wmfme). However, the machines on
the try server don't support the hardware decoding, we can't find the decoder for HEVC.

The interesting thing is the media engine can still decode hevc on the
try server, that could be many possiblities. One guess is the hevc
SW support still exists on Windows, but it's just not exposed via the
MFT (which you have to buy paid extenstion first). Or we have other
setting on try server to prevent us from using HW decoding.

For now, let's make this test for wmfme only.

Depends on D188540

:jmaher Do we need to add an HEVC decoder codec to the win10/win11 pools to get this working? I spun up a loaner for :alwu to test on.

Comment 7

[Alastor Wu [:alwu]]

We've discussed how to run media tasks on the GPU worker this morning, and Joel has helped me to find the solution! Thanks!

Comment 8

[Alastor Wu [:alwu]]

Attachment: Bug 1853448 - part5 : make HEVC test run on all platforms, but only check playback on Windows.

HEVC playback needs hardware decoding, and it currently only support on
Windows. HEVC playback check would be run when the task is in the
mda-gpu, which has the ability for hardware decoding. On other
platforms, HEVC should not be supported.

We also need to set the pref in the ini file, instead of setting it in
file. Because that pref would be checked while creating GPU process,
setting in file would be too late for us.

In addition, interesting thing for HEVC on media engine is that, the
media engine can even play HEVC on the machine which doesn't support
hardware decoding. I suspect that media engine still supports SW HEVC
decoding but just not expose that feature to other applications.

Depends on D188540

Comment 9

[Alastor Wu [:alwu]]

Attachment: Bug 1853448 - part6 : add MSE hevc test.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ead3d69aa2f4
part1 : support HEVC in WMF decoder module. r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/2b587345da78
part2 : check if HEVC DXVA decoder exists. r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/4c1f0e5cdd7a
part3 : return HEVC support for GPU process. r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/c803ab3b6d3c
part4 : modify pref to allow more flexible strategy for enabling HEVC. r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/dcc60b34e9a9
part5 : make HEVC test run on all platforms, but only check playback on Windows. r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/cbef3d8b5de8
part6 : add MSE hevc test. r=media-playback-reviewers,padenot

Comment 11

[Cristian Tuns]

Backed out for causing mochitest failures in test_MediaSource_hevc_mp4.html and test_hevc_playback.html

• Backout link
• Push with failures
• Failure Log
• Failure lines: TEST-UNEXPECTED-FAIL | dom/media/mediasource/test/test_MediaSource_hevc_mp4.html | Test timed out. -
  TEST-UNEXPECTED-FAIL | dom/media/test/test_hevc_playback.html | video started playing

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/355dbe483757
part1 : support HEVC in WMF decoder module. r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/6564e4052fa8
part2 : check if HEVC DXVA decoder exists. r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/d30955261a8c
part3 : return HEVC support for GPU process. r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/96193a1a7df8
part4 : modify pref to allow more flexible strategy for enabling HEVC. r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/b1b8fbac2eab
part5 : make HEVC test run on all platforms, but only check playback on Windows. r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/c8e55bf06bea
part6 : add MSE hevc test. r=media-playback-reviewers,padenot

Comment 13

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/355dbe483757
https://hg.mozilla.org/mozilla-central/rev/6564e4052fa8
https://hg.mozilla.org/mozilla-central/rev/d30955261a8c
https://hg.mozilla.org/mozilla-central/rev/96193a1a7df8
https://hg.mozilla.org/mozilla-central/rev/b1b8fbac2eab
https://hg.mozilla.org/mozilla-central/rev/c8e55bf06bea