1764200 - Consider to prompt users to install Windows Video Extensions (VP9/AV1) if they don't have that on their system

Status: NEW

(Core :: Audio/Video: Playback, enhancement, P2)

Description

[Alastor Wu [:alwu]]

Forked from bug 1762125.

Reporter's about:support is here

From the last log the reporter provided, it showed that Firefox couldn't find the correct MFT for VPX (MF_E_TOPO_CODEC_NOT_FOUND, 0xC00D5212) and couldn't find the correct component for AV1 (WINCODEC_ERR_COMPONENTNOTFOUND, 0x88982F50).

Comment 1

[Zaggy1024]

(In reply to :alwu on the other bug)

But I suppose Chromium also uses MFT for VPX decoding, we would need to figure out why we couldn't find MFT on your system.

I know that Chromium implements at least AV1 directly as a D3D11 decoder as well as through WMF, and I think that's the case for VP9 as well. I think this issue is invalid unless there are plans to implement those decoders in Firefox as well.

We may want to implement some sort of pop up to notify users that they can install the media extensions, though.

Comment 2

[Zaggy1024]

Confirmed:

• VP9: https://chromium.googlesource.com/chromium/src/+/refs/heads/main/media/gpu/vp9_decoder.cc
• AV1: https://chromium.googlesource.com/chromium/src/+/refs/heads/main/media/gpu/av1_decoder.cc

Comment 3

[Zaggy1024]

Links to the Microsoft media extensions are:
https://www.microsoft.com/en-us/p/vp9-video-extensions/9n4d0msmp0pt
https://www.microsoft.com/en-us/p/av1-video-extension/9mvzqvxjbq9v

Comment 4

[Alastor Wu [:alwu]]

Ah, thank you!

dragos, could you help me confirm that if installing these extensions (VP9 / AV1) helps your Firefox perform HW decoding?

Comment 5

[dragos.slash]

Attachment: Untitled.png

Thank you for that solution. I can confirm that now I'm getting both VP9 and AV1 HW decoding in Nightly 101. I am even getting VP9 decoding in ESR 91.

Comment 6

[Zaggy1024]

It's worth noting that there have been two issues already where people were unaware they need install the AV1 Video Extension from the store:

• https://www.reddit.com/r/firefox/comments/u17udy/installed_firefox_100_beta_and_av1_doesnt_seem_to/
• https://bugzilla.mozilla.org/show_bug.cgi?id=1762125

There'll probably be more support requests about this in the future without a way to indicate that people need to install them.

Comment 7

[Zaggy1024]

It would also probably be good to use the Windows Store link handler to open a small dialog to prompt to install the extensions for convenience for the end user:

https://docs.microsoft.com/en-us/windows/uwp/launch-resume/launch-store-app

An example link would be ms-windows-store://pdp/?ProductId=9MVZQVXJBQ9V&mode=mini, which will open the AV1 Video Extensions store page in a small dialog with an option to "Get" the app.

Comment 8

[Alastor Wu [:alwu]]

Maybe we should consider to have this in recent upcoming versions, which can help more users to use hardware decoding.

Comment 9

[Artem S. Tashkinov]

Prompts may be considered intrusive but I'd absolutely show this information in about:support.

Comment 10

[Artem S. Tashkinov]

BTW, I've not noticed any settings in Firefox in regard to HW video decoding.

Would be nice to have something at least in about:config.

Rationale:

1. To avoid any possible problems with HW decoding which can be not as robust as the software one
2. To test e.g. system power consumption
3. To test how your CPU handles video decoding and if it's better than your GPU.

Comment 11

[Alastor Wu [:alwu]]

Would be nice to have something at least in about:config.

FYI, we have that already. In the about:config you posted before, you can see following information

"codecSupportInfo": "H264 SW\nH264 HW\nVP8 SW\nVP9 SW\nAV1 SW\nTheora SW\nAAC SW\nFLAC SW\nMP3 SW\nOpus SW\nVorbis SW\nWave SW"

Comment 12

[Artem S. Tashkinov]

VP9 SW doesn't says a lot especially considering when other applications offer HW encoding regardless ;-)

I meant showing something like: VP9: SW ⚠️ (where the alert emoji is clickable and it explains what's going on/missing).

Comment 13

[Alastor Wu [:alwu]]

Sounds like a good idea, NI :az who implemented the part of codec support information to see what she thinks.

Comment 14

[Zaggy1024]

It's definitely a good idea to get that information into about:support, but I would argue it doesn't solve the problem. Most users won't know to open about:support until they ask for help, by which point time has already been wasted troubleshooting. If a notification is shown the first time that a video of a certain codec is shown, then nobody will end up needing to create reddit posts or bugs.

As far as annoying factor, I feel like it wouldn't be an issue if it was shown once and then once dismissed was never shown again, but perhaps there's a better way to make users aware without opening a prompt, like an icon in the address bar?

Comment 15

[Artem S. Tashkinov]

(In reply to Zaggy1024 from comment #14)

It's definitely a good idea to get that information into about:support, but I would argue it doesn't solve the problem. Most users won't know to open about:support until they ask for help, by which point time has already been wasted troubleshooting. If a notification is shown the first time that a video of a certain codec is shown, then nobody will end up needing to create reddit posts or bugs.

As far as annoying factor, I feel like it wouldn't be an issue if it was shown once and then once dismissed was never shown again, but perhaps there's a better way to make users aware without opening a prompt, like an icon in the address bar?

Most people dismiss one time notifications and God forbid you show a popup window or anything.

Firefox has an option of notifying the user via a narrow yellow strip at the top of the browser - this could work. It's not intrusive and people would more likely to pay attention to it.

Comment 16

[az [:az]]

Adding a notification / additional info for this sounds like a good idea to me as well. Might be able to share code with bug 1766637 since that'll also involve informing the user regarding files needed to improve media playback.

Comment 17

[dragos.slash]

I'm all for more details on the status of h/w decoding in the about:support page. LTSC doesn't have Store, so a Store link wouldn't do much for me and other users of this SKU, without any further details on what is actually missing/broken/happening (GPU/driver blacklist, missing media extensions, etc.).
In my case it was a combination of both driver version blacklisted and missing media extensions. I had to manually download those extensions and install them through PS. I was completely unaware that Firefox needs them.

Comment 18

[zacharyachurchill]

I'm very in favor of doing this, discovering that https://bugzilla.mozilla.org/show_bug.cgi?id=1813339 was the cause of hw acceleration not working took me over 2 years. It's completely opaque to me, and I assume most users, that software from the Microsoft store would be necessary for hardware accelerated video decoding of any kind to work in firefox. The connection is not obvious to someone who doesn't know

1. Firefox relies on Windows media framework to do video decoding, which
2. requires additional software to do av9 decoding, which is the video codec of choice for the most popular video sharing website on the planet

Comment 19

[Jim Mathies [:jimm]]

We have built-in software decoding for both VP8/9 and AV1, so this is just about access to hardware features.

I don't see how we would know the underlying hardware supported hardware decoding before we prompt?

Comment 20

[Jeff Muizelaar [:jrmuizel]]

We should be able to detect support with ID3D11VideoDevice::GetVideoDecoderProfile like Chrome does here: https://source.chromium.org/chromium/chromium/src/+/refs/heads/main:gpu/config/gpu_info.h;l=482;drc=0340dd86c6c1f01ccd1a0b983633cfef5fcf6697

Comment 21

[pexu+moz-bugzilla]

For the current implementation the AV1 extension package (Microsoft.AV1VideoExtension_8wekyb3d8bbwe) works if installed but the VP9 extension package (Microsoft.VP9VideoExtensions_8wekyb3d8bbwe) will not.

The issue is that VP9 decoder is instanced using CoCreateInstance via CLSID_MSVPxDecoder/CLSID_CMSVPXDecMFT ({E3AAF548-C9A4-4C6E-234D-5ADA374B0000}) whereas AV1 (and HVEC) use MFTEnumEx.

At least on Windows 11 the shipped VP9 "decoder" (%SystemRoot%\System32\MSVP9DEC.dll), that gets picked up by that CLSID, is just a small wrapper, not the codec itself. One could guess this from a small filesize and imported MFTEnumEx. Also, the GUID looks a generated one and not like pseudo-FourCC seen below. Besides some desktop versions, the file is also missing from Windows Server platforms.

This MSVP9DEC.dll exports DllGetClassObject so that a COM instance can be created. A quick inspection with Ghidra reveals that it calls MFTEnumEx(MFT_CATEGORY_VIDEO_DECODER, MFT_ENUM_FLAG_SORTANDFILTER, pInputType, pOutputType = nullptr, ...) where the referenced pInputType (MFT_REGISTER_TYPE_INFO) has guidMajorType set to MFMediaType_Video ({73646976-0000-0010-8000-00AA00089B71}, "vids") and guidSubtype set to MFVideoFormat_VP90 ({30395056-0000-0010-8000-00AA00389B71}, "VP90"). Besides the VP9 GUID, pretty much what is already being done for AV1 and HVEC.

I did not study the "encoder" (%SystemRoot%\System32\MSVPXENC.dll), but given the premise (differs only by a few bytes) I presume it is the same story.

Both VP9 GUID and VP8 GUID MFVideoFormat_VP80 ({30385056-0000-0010-8000-00AA00389B71}, "VP80") are defined in VP9 package manifest AppxManifest.xml at <uap4:MediaCodec> extension node. However, the manifest does define any COM interfaces thru <com4:Class>, so installing the package does not create any HKCR\CLSID\E3AAF548-C9A4-4C6E-234D-5ADA374B0000 (or corresponding WOW6432Node) keys. Even if these were created manually, loading msvp9dec_store.dll directly is not possible, as it only exports the typical UWP entrypoint DllGetActivationFactory; CoCreateInstance will simply error out with CO_E_ERRORINDLL (0x800401F9).

So, the comment "MFTEnumEx is slower than creating by CLSID" for AV1 at WMFDecoderModule.cpp, might not be true at all. The AV1 extension is similar to VP9, so there are no exported or usable classic COM CLSIDs or entrypoints. Perhaps the CLSID_MSVPxDecoder is some leftover from previous generation, which predated the current UWP implementation?

Meanwhile these shipped VP9 wrappers can be technically (but not necessarely legally) used on another machine by simply copying the DLLs and creating the associated HKCR keys manually. There is no DllRegisterServer (at least directly exported), so one cannot use regsvr32.exe for this purpose. Given the task the wrappers do is not very complicated, these appear to work between different systems (tried Windows 11 version on Windows Server 2022; no errors, DLLs are picked up, about:support shows VP8/VP9 HW as supported and GPU shows decode activity on a VP9 encoded video).

Comment 22

[Artem S. Tashkinov]

The fact that AV1/VP9 extensions are installed doesn't mean your HW is actually capable of HW decoding of these codecs.

Case in point: I have GTX 1660 Ti and am running Windows 10 LTSC with the AV1 extension and Firefox happily shows that AV1 HW decoding is available despite nothing in my system offering it.

But that's in version 122.