1682834 - Media playback fails when antivirus is installed and RDD is enabled

Status: VERIFIED FIXED

(Core :: Security: Process Sandboxing, defect, P1)

Description

[toddthedeveloper]

Attachment: Firefox85YouTubeError.png

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:85.0) Gecko/20100101 Firefox/85.0

Steps to reproduce:

Click on a video on YouTube

Actual results:

Black screen. No playback.

Expected results:

Video plays.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 2

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Could you navigate to about:support and use one of the buttons there to copy the information then paste it onto this bug?

Comment 3

[yuanhaoyoung]

This happened to me just this week, too. In the debug console, I get
Cannot play media. No decoders for requested formats: audio/webm; codecs="opus", video/mp4; codecs="avc1.4d400c"

This is on Firefox 85.0b2. The URL I'm trying to play is: https://www.youtube.com/watch?v=M8G_LIH-8Nc&feature=youtu.be

The video works fine on Android Firefox (84.0.0-beta 4)

Comment 4

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

As well as the information from about:support, if anyone encountering the issue could navigate to about:config and set the media.rdd-process.enabled pref to false and see if that impacts the problem, that would be useful (the pref can be set back after testing).

Comment 5

[yuanhaoyoung]

Attachment: about:support contents for 1682834

Thanks, Bryce. Setting media.rdd-process.enabled to false fixed it for me. about:support reports the following:

Comment 6

[yuanhaoyoung]

Thanks, Bryce. Setting media.rdd-process.enabled to false fixed it for me. about:support reports the following:

Comment 7

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Thanks, that's helpful.

If you're willing to test further, could you check if

• media.rdd-process.enabled = true
• media.rdd-wmf.enabled = false

works?

Comment 8

[yuanhaoyoung]

I just tried

media.rdd-process.enabled = true
media.rdd-wmf.enabled = false

but that doesn't work. There's a spinning progress symbol under the YouTube logo (which is what I got before), and debug shows a similar message about No decoders for requested formats.

Comment 9

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Thanks for checking that. If you could it would be useful to check

• media.rdd-process.enabled = true
• media.rdd-ffvpx.enabled = false
• media.rdd-wmf.enabled = false

and if that doesn't work setting ontop of the above

• media.rdd-opus.enabled = false

---

In case you're interested in what we're testing (if not you can skip this): we've recently enabled functionality to decode audio and video in a new architecture that seems to be leading to this problem. media.rdd-process.enabled lets us turn on or off the whole arhitecture, and since that seems to help solve the issue, the other prefs are letting us turn off more specific parts to help us try narrow the problem to more specific areas.

Comment 10

[yuanhaoyoung]

Bryce, I tried disabling ffvpx and wmf. That doesn't work. I then also disabled opus, and it still doesn't work. Right now, it looks like I have to disable the process.enabled flag to make it work at all.

Comment 11

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Thanks for confirming that. We've had some other reports that indicate similar issues and suggest they may be specific to beta. It would be helpful to know if this doesn't happen in nightly (Firefox 86).

Comment 13

[Jean-Yves Avenard [:jya]]

It's bizarre that it would work in Nightly but not in Beta. Should be the same code now

Comment 14

[Jean-Yves Avenard [:jya]]

Could you please try with a new profile and also using Firefox nightly?

Comment 15

[Richard]

As I mentioned, I did a full "Refresh Firefox" from the About:support page.... this created a new profile today and it still have the issue.

Comment 16

[Richard]

(In reply to Richard from comment #15)

As I mentioned, I did a full "Refresh Firefox" from the About:support page.... this created a new profile today and it still have the issue.

After that I got the info to change about:configuration which solved the issue.

Comment 17

[Richard]

PS: I looked at the Nightly version and media.rdd-process.enabled = true is the way I found it... and it works.

Comment 18

[Jean-Yves Avenard [:jya]]

I assume that about:support was collected after you had disabled the rdd pref.

Could you try the following:

1. re-enable media.rdd-process.enabled to true in about:config
2. Close Firefox
3. Browser to the location of your Firefox install using Explorer
4. Shift + Right-click in the folder window where firefox.exe is located, select "Open command window here"
5. Add the environment variable(s) you wish to set to your command window -

set MOZ_DISABLE_RDD_SANDBOX=1(return)

6. enter firefox.exe and press enter to launch Firefox with your custom environment

And see if it now works for you.

It's the only thing I can think of that would explain a difference between nightly and beta.

Comment 19

[Jean-Yves Avenard [:jya]]

(In reply to Richard from comment #17)

PS: I looked at the Nightly version and media.rdd-process.enabled = true is the way I found it... and it works.

I'm confused.
are you saying that with nightly and media.rdd-process.enabled = true (the default value) then it works for you?

Comment 20

[Richard]

I have both FF Beta and Nightly on my machine. FF Beta is V85... Nightly V86.

After I changed V85 to the media.rdd-process.enabled to False, V85 worked fine.

V86 always worked... no adjustments needed. So I got curious as to what V86's media.rdd-process.enabled value was, so I looked... it was set to True.

do you still want me to do those steps above???

Comment 21

[donqk]

(In reply to Jean-Yves Avenard [:jya] from comment #18)

I assume that about:support was collected after you had disabled the rdd pref.

Could you try the following:

1. re-enable media.rdd-process.enabled to true in about:config
2. Close Firefox
3. Browser to the location of your Firefox install using Explorer
4. Shift + Right-click in the folder window where firefox.exe is located, select "Open command window here"
5. Add the environment variable(s) you wish to set to your command window -

set MOZ_DISABLE_RDD_SANDBOX=1(return)

6. enter firefox.exe and press enter to launch Firefox with your custom environment

And see if it now works for you.

It's the only thing I can think of that would explain a difference between nightly and beta.

I've the same problem as Richard, but i'm using Nightly, fission enabled. I set MOZ_DISABLE_RDD_SANDBOX=1 and then run firefox.exe and video playback didn't work. If media.rdd-process.enable is set to false, video playback works as intended.

Comment 22

[Richard]

I did the steps above on V85, still did not work with media.rdd-process.enabled = true. Change it back to false in V85, and videos work again.

Comment 23

[Jean-Yves Avenard [:jya]]

Do you have any crashes showing in about:crashes?

Comment 24

[donqk]

(In reply to Jean-Yves Avenard [:jya] from comment #23)

Do you have any crashes showing in about:crashes?

I tested what you suggested again, because i didn't set the environment variable correctly. Used '$Env:MOZ_DISABLE_RDD_SANDBOX = 1' in the powershell console, then executed Nightly ('firefox.exe') with 'media.rdd-process.enabled = true' and video playback works fine.

Also, no crashes reported.

Comment 25

[Jean-Yves Avenard [:jya]]

In beta, is the pref media.rdd-retryonfailure.enabled set to true?

Comment 26

[Jean-Yves Avenard [:jya]]

(In reply to donqk from comment #24)

(In reply to Jean-Yves Avenard [:jya] from comment #23)

Do you have any crashes showing in about:crashes?

I tested what you suggested again, because i didn't set the environment variable correctly. Used '$Env:MOZ_DISABLE_RDD_SANDBOX = 1' in the powershell console, then executed Nightly ('firefox.exe') with 'media.rdd-process.enabled = true' and video playback works fine.

Also, no crashes reported.

And it didn't work without setting that pref in Nightly before?

There's now too many people writing their "me too" to easily track what's going on.

Can you try the same with beta? Does it work then?

Comment 27

[Jean-Yves Avenard [:jya]]

This could be a sandboxing issue.what could explain diffences between beta and nightly. Could you help troubleshoot this? I can't reproduce. But I have a different Intel machine, a few reports all appear to use older Intel 620

Comment 29

[donqk]

(In reply to Jean-Yves Avenard [:jya] from comment #26)

(In reply to donqk from comment #24)

(In reply to Jean-Yves Avenard [:jya] from comment #23)

Do you have any crashes showing in about:crashes?

I tested what you suggested again, because i didn't set the environment variable correctly. Used '$Env:MOZ_DISABLE_RDD_SANDBOX = 1' in the powershell console, then executed Nightly ('firefox.exe') with 'media.rdd-process.enabled = true' and video playback works fine.

Also, no crashes reported.

And it didn't work without setting that pref in Nightly before?

There's now too many people writing their "me too" to easily track what's going on.

Can you try the same with beta? Does it work then?

If "MOZ_DISABLE_RDD_SANDBOX = 1" is not defined, and "media.rdd-process.enabled" is enabled, video playback doesn't work in Nightly, in my pc (Intel i7 7500u, Intel 620).
In Firefox Beta, video playback works fine with default configuration, no changes needed. Same with Firefox stable.

Comment 31

[Jean-Yves Avenard [:jya]]

If "MOZ_DISABLE_RDD_SANDBOX = 1" is not defined, and "media.rdd-process.enabled" is enabled, video playback doesn't work in Nightly, in my pc (Intel i7 7500u, Intel 620).

And it works in Nightly if you set MOZ_DISABLE_RDD_SANDBOX?

Comment 32

[Jean-Yves Avenard [:jya]]

Please try with the following prefs being set in about:config
media.rdd-process.enabled : true
media.rdd-wmf.enabled: false
media.rdd-ffvpx.enabled: false

You must restart Firefox after changing those preferences.

Comment 33

[korossyg]

(In reply to Jean-Yves Avenard [:jya] from comment #32)

Please try with the following prefs being set in about:config
media.rdd-process.enabled : true
media.rdd-wmf.enabled: false
media.rdd-ffvpx.enabled: false

You must restart Firefox after changing those preferences.

This doesn't work. When the media.rdd-process.enabled is set to false, youtube videos play, however, youtube music doesn't work even with those settings.

Comment 34

[Jean-Yves Avenard [:jya]]

Can someone try the moz regression tool to determine which change made it stop working?.
https://mozilla.github.io/mozregression/

Comment 35

[Richard]

PS: Just to add to some additional info:
My V85b2 has the media.rdd-process.enabled set to false and works... mostly in YouTube. However, no videos in Facebook work.

My V86a1 has the media.rdd-process.enabled set to true and works... everywhere, including Facebook.

Comment 36

[Jean-Yves Avenard [:jya]]

Are you all using Windows N édition ?

Comment 37

[korossyg]

(In reply to Jean-Yves Avenard [:jya] from comment #36)

Are you all using Windows N édition ?

I do: Windows 10 N Pro 64-bit (version 20H2, build 19042.685). The Media Feature Pack is enabled.

Comment 38

[Jean-Yves Avenard [:jya]]

[.] removed

Comment 39

[Jean-Yves Avenard [:jya]]

I could reproduce !

Install Norton 360 trial.

Playback won't start in beta.
It starts with nightly however.

Beta compiled locally works too for some reason.

Just suspending Norton 360 doesn't allow playback to start; it must be uninstalled.

Disabling the sandbox with the following steps allowed playback to work:

1- Close Firefox
2- Browser to the location of your Firefox install using Explorer
3- Shift + Right-click in the folder window where firefox.exe is located, select "Open powershell window here"
4- Add the environment variable(s) you wish to set to your command window with "$Env:MOZ_DISABLE_RDD_SANDBOX = 1"
5- Start firefox by typing .\firefox.exe

Comment 40

[korossyg]

(In reply to Jean-Yves Avenard [:jya] from comment #39)

I could reproduce !

Install Norton 360 trial.

Playback won't start in beta.
It starts with nightly however.

Beta compiled locally works too for some reason.

Just suspending Norton 360 doesn't allow playback to start; it must be uninstalled.

Disabling the sandbox with the following steps allowed playback to work:

1- Close Firefox
2- Browser to the location of your Firefox install using Explorer
3- Shift + Right-click in the folder window where firefox.exe is located, select "Open powershell window here"
4- Add the environment variable(s) you wish to set to your command window with "$Env:MOZ_DISABLE_RDD_SANDBOX = 1"
5- Start firefox by typing .\firefox.exe

This seems to be working. I have Norton 360 as well.

Comment 41

[Jean-Yves Avenard [:jya]]

For now, we're just going to backout bug 1620114 in beta only ; once bug 1682609 lands in Nightly and ride the train, we will be good to go.

Comment 42

[Jean-Yves Avenard [:jya]]

Attachment: Bug 1682834 - "Revert 1620114 - Enable pre-spawn CIG in RDD." r=bobowen

This reverts commit https://hg.mozilla.org/mozilla-central/rev/9df12e4b7a00

for causing RDD's decoder failures and total media playback failures when an antivirus is installed.

Comment 43

[Jean-Yves Avenard [:jya]]

Comment on attachment 9193915 [details]
Bug 1682834 - "Revert 1620114 - Enable pre-spawn CIG in RDD." r=bobowen

Beta/Release Uplift Approval Request

• User impact if declined: No audio nor video can be played (YouTube, Facebook, two rather popular sites would become unusable).
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: See https://bugzilla.mozilla.org/show_bug.cgi?id=1682834#c39 ; Install Norton 360 ; it doesn't even need to be enabled.

If you install firefox in C:\Program Files\Mozilla Firefox it will fail to run.

If installed in "C:\Program Files\Firefox Nightly" or any other locations; it will work !

• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): We revert bug 1620114 ; which was only recently added a couple of weeks ago. So we go back to an existing, and tested code path.
• String changes made/needed: None

Comment 44

[Gian-Carlo Pascutto [:gcp]]

If you install firefox in C:\Program Files\Mozilla Firefox it will fail to run.
If installed in "C:\Program Files\Firefox Nightly" or any other locations; it will work !

This is really interesting! We've been trying for months to get https://bugzilla.mozilla.org/show_bug.cgi?id=1347710 enabled and saw similar behavior there.

Comment 45

[Gian-Carlo Pascutto [:gcp]]

once bug 1682609 lands in Nightly and ride the train, we will be good to go.

There are no plans to land this in Nightly, or ever, if it can be helped.

Comment 46

[Gian-Carlo Pascutto [:gcp]]

Disabling full CIG in beta seems like the best way forward.

For Nightly, we'll have to debug what Norton is doing that causes them to break security features. The failures here suggest that they inject themselves into RDD in a way that we can't see (even if the product is disabled!), and things then totally break if Windows tries to apply CIG protections to the RDD process because Nortons' product has already compromised it. We'll have to investigate what is up.

Comment 47

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9193915 [details]
Bug 1682834 - "Revert 1620114 - Enable pre-spawn CIG in RDD." r=bobowen

Approved for 85.0b4.

Comment 48

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/7de611ce368d

Comment 50

[Toshihito Kikuchi [:toshi]]

Thank you for taking care of this while I'm on PTO!

(In reply to Gian-Carlo Pascutto [:gcp] from comment #44)

If you install firefox in C:\Program Files\Mozilla Firefox it will fail to run.
If installed in "C:\Program Files\Firefox Nightly" or any other locations; it will work !

This is really interesting! We've been trying for months to get https://bugzilla.mozilla.org/show_bug.cgi?id=1347710 enabled and saw similar behavior there.

With Norton's trial (I needed to put my creditcard information to start download though), I could confirm their injection behavior was actually different between Nightly and Beta. While only symamsi.dll was injected into the browser process of Nightly, IPSEng64.dll (which is causing another bug 1626993) was injected into all processes of Beta. And, I reproduced the issue: RDD of Beta failed to start. Let me analyze their behavior.

(In reply to Gian-Carlo Pascutto [:gcp] from comment #46)

Disabling full CIG in beta seems like the best way forward.

For Nightly, we'll have to debug what Norton is doing that causes them to break security features. The failures here suggest that they inject themselves into RDD in a way that we can't see (even if the product is disabled!), and things then totally break if Windows tries to apply CIG protections to the RDD process because Nortons' junk product has already compromised it. We'll have to investigate what is up.

Now that my patch was backed out, we still enable CIG in RDD after a process launched. This means third-party modules are still blocked unless they are signed by Microsoft or are injected before we enable CIG. Media modules should be the former case, and IPSEng64.dll should be the latter. The problem here is blocking IPSEng64.dll somehow blocks an rdd process from launching. I got the repro. It should be a fun debug.

Comment 51

[Toshihito Kikuchi [:toshi]]

(In reply to Gian-Carlo Pascutto [:gcp] from comment #45)

once bug 1682609 lands in Nightly and ride the train, we will be good to go.

There are no plans to land this in Nightly, or ever, if it can be helped.

Unfortunately disabling pre-spawn CIG will not suffice to solve the compat issue with IBM. I'm landing bug 1682609 now. Once it's done, we don't need to back out the patch from 86+.

Comment 52

[donqk]

(In reply to Toshihito Kikuchi [:toshi] from comment #50)

Thank you for taking care of this while I'm on PTO!

(In reply to Gian-Carlo Pascutto [:gcp] from comment #44)

If you install firefox in C:\Program Files\Mozilla Firefox it will fail to run.
If installed in "C:\Program Files\Firefox Nightly" or any other locations; it will work !

This is really interesting! We've been trying for months to get https://bugzilla.mozilla.org/show_bug.cgi?id=1347710 enabled and saw similar behavior there.

With Norton's trial (I needed to put my creditcard information to start download though), I could confirm their injection behavior was actually different between Nightly and Beta. While only symamsi.dll was injected into the browser process of Nightly, IPSEng64.dll (which is causing another bug 1626993) was injected into all processes of Beta. And, I reproduced the issue: RDD of Beta failed to start. Let me analyze their behavior.

(In reply to Gian-Carlo Pascutto [:gcp] from comment #46)

Disabling full CIG in beta seems like the best way forward.

For Nightly, we'll have to debug what Norton is doing that causes them to break security features. The failures here suggest that they inject themselves into RDD in a way that we can't see (even if the product is disabled!), and things then totally break if Windows tries to apply CIG protections to the RDD process because Nortons' junk product has already compromised it. We'll have to investigate what is up.

Now that my patch was backed out, we still enable CIG in RDD after a process launched. This means third-party modules are still blocked unless they are signed by Microsoft or are injected before we enable CIG. Media modules should be the former case, and IPSEng64.dll should be the latter. The problem here is blocking IPSEng64.dll somehow blocks an rdd process from launching. I got the repro. It should be a fun debug.

I don't know if this helps, but if there's a registered "amsi" provider (most AVs), and the process calls amsi, then the amsi dll gets injected (both dlls: Windows Defender's and the commercial AV dll).
The thing is, many AVs "amsi" dll are not signed properly. For example, the amsi dll of Eset fails in the automatic security audit of Windows. That may explain why Windows Defender amsi dll doesn't fail with CIG enabled before process is started, but a commercial av amsi.dll does.
Read here: https://forum.eset.com/topic/25035-program-fileseseteset-nod32-antiviruseamsidll/

Comment 53

[Arthur K. (he/him)]

Confirming that this now works again in 85.0b4 as it used to.

Comment 54

[donqk]

Fix works in Firefox Beta. Doesn't work on Nightly

Comment 55

[Jean-Yves Avenard [:jya]]

(In reply to donqk from comment #54)

Fix works in Firefox Beta. Doesn't work on Nightly

until a fix is found for getting around Norton/Symantec, I suggest you uninstall your antivirus. Or you can look in the earlier message on how to start Firefox with the RDD sandbox disabled.

Comment 56

[Rares Doghi, Desktop QA]

Hi, This issue is Verified as Fixed in Beta 85.0b4 - tested on a Windows 10 x32, Please note that this issue was tested only with Norton 360 since Symantec needs the University Systems credentials to access and download the installer.

Please also note that our latest Nightly build installed in C:\Program Files\Firefox Nightly folder also works without issues.

Comment 57

[donqk]

(In reply to Jean-Yves Avenard [:jya] from comment #55)

(In reply to donqk from comment #54)

Fix works in Firefox Beta. Doesn't work on Nightly

until a fix is found for getting around Norton/Symantec, I suggest you uninstall your antivirus. Or you can look in the earlier message on how to start Firefox with the RDD sandbox disabled.

I've Eset Nod AV installed, so this problem is not unique to Norton users. I'll use it with RDD sandbox disabled, thanks.

Comment 58

[Toshihito Kikuchi [:toshi]]

(In reply to Gian-Carlo Pascutto [:gcp] from comment #44)

If you install firefox in C:\Program Files\Mozilla Firefox it will fail to run.
If installed in "C:\Program Files\Firefox Nightly" or any other locations; it will work !

This is really interesting! We've been trying for months to get https://bugzilla.mozilla.org/show_bug.cgi?id=1347710 enabled and saw similar behavior there.

Finally, I found the condition. Norton checks the combination of an executable's name and its path. If a process is firefox.exe, Norton injects IPSEng64.dll if the path contains a string \mozilla firefox\ (case-insensitive).

Comment 59

[Toshihito Kikuchi [:toshi]]

In v86, the compat issue with Norton 360 should be fixed by bug 1684532.

Comment 60

[Rares Doghi, Desktop QA]

Hi, I have retested this issue by installing older versions of Nightly in the C:\Program Files\Mozilla Firefox\ path and I was able to reproduce it, I can also confirm that the issue is Verified as Fixed in our latest version of Nightly 86.0a1 (2021-01-18).

Comment 61

[Toshihito Kikuchi [:toshi]]

(In reply to Rares Doghi from comment #60)

Hi, I have retested this issue by installing older versions of Nightly in the C:\Program Files\Mozilla Firefox\ path and I was able to reproduce it, I can also confirm that the issue is Verified as Fixed in our latest version of Nightly 86.0a1 (2021-01-18).

Thank you for verifying the latest Nightly! Glad to hear the issue was fixed.

Comment 62

[Rares Doghi, Desktop QA]

Hi, can we update the Tracking and Status flags for this issue to Verified ?

Comment 63

[Rares Doghi, Desktop QA]

Removing the Qe-Verify flags, Thank you