Closed Bug 1682834 Opened 7 months ago Closed 7 months ago

Media playback fails when antivirus is installed and RDD is enabled

Categories

(Core :: Security: Process Sandboxing, defect, P1)

Firefox 85
defect

Tracking

()

VERIFIED FIXED
Tracking Status
firefox-esr78 --- unaffected
firefox84 --- unaffected
firefox85 blocking verified
firefox86 + verified

People

(Reporter: toddthedeveloper, Assigned: jya)

References

(Regression)

Details

(Keywords: regression)

Attachments

(3 files)

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:85.0) Gecko/20100101 Firefox/85.0

Steps to reproduce:

Click on a video on YouTube

Actual results:

Black screen. No playback.

Expected results:

Video plays.

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Component: Untriaged → Audio/Video: Playback
Product: Firefox → Core

Could you navigate to about:support and use one of the buttons there to copy the information then paste it onto this bug?

Flags: needinfo?(toddthedeveloper)

This happened to me just this week, too. In the debug console, I get
Cannot play media. No decoders for requested formats: audio/webm; codecs="opus", video/mp4; codecs="avc1.4d400c"

This is on Firefox 85.0b2. The URL I'm trying to play is: https://www.youtube.com/watch?v=M8G_LIH-8Nc&feature=youtu.be

The video works fine on Android Firefox (84.0.0-beta 4)

As well as the information from about:support, if anyone encountering the issue could navigate to about:config and set the media.rdd-process.enabled pref to false and see if that impacts the problem, that would be useful (the pref can be set back after testing).

Thanks, Bryce. Setting media.rdd-process.enabled to false fixed it for me. about:support reports the following:

Thanks, Bryce. Setting media.rdd-process.enabled to false fixed it for me. about:support reports the following:

Thanks, that's helpful.

If you're willing to test further, could you check if

  • media.rdd-process.enabled = true
  • media.rdd-wmf.enabled = false

works?

I just tried

media.rdd-process.enabled = true
media.rdd-wmf.enabled = false

but that doesn't work. There's a spinning progress symbol under the YouTube logo (which is what I got before), and debug shows a similar message about No decoders for requested formats.

Thanks for checking that. If you could it would be useful to check

  • media.rdd-process.enabled = true
  • media.rdd-ffvpx.enabled = false
  • media.rdd-wmf.enabled = false

and if that doesn't work setting ontop of the above

  • media.rdd-opus.enabled = false

In case you're interested in what we're testing (if not you can skip this): we've recently enabled functionality to decode audio and video in a new architecture that seems to be leading to this problem. media.rdd-process.enabled lets us turn on or off the whole arhitecture, and since that seems to help solve the issue, the other prefs are letting us turn off more specific parts to help us try narrow the problem to more specific areas.

Bryce, I tried disabling ffvpx and wmf. That doesn't work. I then also disabled opus, and it still doesn't work. Right now, it looks like I have to disable the process.enabled flag to make it work at all.

Thanks for confirming that. We've had some other reports that indicate similar issues and suggest they may be specific to beta. It would be helpful to know if this doesn't happen in nightly (Firefox 86).

Status: UNCONFIRMED → NEW
Ever confirmed: true
Summary: No video playback on YouTube → Media playback fails when RDD is enabled
Duplicate of this bug: 1682870

It's bizarre that it would work in Nightly but not in Beta. Should be the same code now

Could you please try with a new profile and also using Firefox nightly?

As I mentioned, I did a full "Refresh Firefox" from the About:support page.... this created a new profile today and it still have the issue.

(In reply to Richard from comment #15)

As I mentioned, I did a full "Refresh Firefox" from the About:support page.... this created a new profile today and it still have the issue.

After that I got the info to change about:configuration which solved the issue.

PS: I looked at the Nightly version and media.rdd-process.enabled = true is the way I found it... and it works.

I assume that about:support was collected after you had disabled the rdd pref.

Could you try the following:

  1. re-enable media.rdd-process.enabled to true in about:config
  2. Close Firefox
  3. Browser to the location of your Firefox install using Explorer
  4. Shift + Right-click in the folder window where firefox.exe is located, select "Open command window here"
  5. Add the environment variable(s) you wish to set to your command window -

set MOZ_DISABLE_RDD_SANDBOX=1(return)

  1. enter firefox.exe and press enter to launch Firefox with your custom environment

And see if it now works for you.

It's the only thing I can think of that would explain a difference between nightly and beta.

(In reply to Richard from comment #17)

PS: I looked at the Nightly version and media.rdd-process.enabled = true is the way I found it... and it works.

I'm confused.
are you saying that with nightly and media.rdd-process.enabled = true (the default value) then it works for you?

Flags: needinfo?(rhmorris)

I have both FF Beta and Nightly on my machine. FF Beta is V85... Nightly V86.

After I changed V85 to the media.rdd-process.enabled to False, V85 worked fine.

V86 always worked... no adjustments needed. So I got curious as to what V86's media.rdd-process.enabled value was, so I looked... it was set to True.

do you still want me to do those steps above???

Flags: needinfo?(rhmorris)

(In reply to Jean-Yves Avenard [:jya] from comment #18)

I assume that about:support was collected after you had disabled the rdd pref.

Could you try the following:

  1. re-enable media.rdd-process.enabled to true in about:config
  2. Close Firefox
  3. Browser to the location of your Firefox install using Explorer
  4. Shift + Right-click in the folder window where firefox.exe is located, select "Open command window here"
  5. Add the environment variable(s) you wish to set to your command window -

set MOZ_DISABLE_RDD_SANDBOX=1(return)

  1. enter firefox.exe and press enter to launch Firefox with your custom environment

And see if it now works for you.

It's the only thing I can think of that would explain a difference between nightly and beta.

I've the same problem as Richard, but i'm using Nightly, fission enabled. I set MOZ_DISABLE_RDD_SANDBOX=1 and then run firefox.exe and video playback didn't work. If media.rdd-process.enable is set to false, video playback works as intended.

I did the steps above on V85, still did not work with media.rdd-process.enabled = true. Change it back to false in V85, and videos work again.

Do you have any crashes showing in about:crashes?

(In reply to Jean-Yves Avenard [:jya] from comment #23)

Do you have any crashes showing in about:crashes?

I tested what you suggested again, because i didn't set the environment variable correctly. Used '$Env:MOZ_DISABLE_RDD_SANDBOX = 1' in the powershell console, then executed Nightly ('firefox.exe') with 'media.rdd-process.enabled = true' and video playback works fine.

Also, no crashes reported.

In beta, is the pref media.rdd-retryonfailure.enabled set to true?

(In reply to donqk from comment #24)

(In reply to Jean-Yves Avenard [:jya] from comment #23)

Do you have any crashes showing in about:crashes?

I tested what you suggested again, because i didn't set the environment variable correctly. Used '$Env:MOZ_DISABLE_RDD_SANDBOX = 1' in the powershell console, then executed Nightly ('firefox.exe') with 'media.rdd-process.enabled = true' and video playback works fine.

Also, no crashes reported.

And it didn't work without setting that pref in Nightly before?

There's now too many people writing their "me too" to easily track what's going on.

Can you try the same with beta? Does it work then?

This could be a sandboxing issue.what could explain diffences between beta and nightly. Could you help troubleshoot this? I can't reproduce. But I have a different Intel machine, a few reports all appear to use older Intel 620

Flags: needinfo?(bobowencode)
Duplicate of this bug: 1682791

(In reply to Jean-Yves Avenard [:jya] from comment #26)

(In reply to donqk from comment #24)

(In reply to Jean-Yves Avenard [:jya] from comment #23)

Do you have any crashes showing in about:crashes?

I tested what you suggested again, because i didn't set the environment variable correctly. Used '$Env:MOZ_DISABLE_RDD_SANDBOX = 1' in the powershell console, then executed Nightly ('firefox.exe') with 'media.rdd-process.enabled = true' and video playback works fine.

Also, no crashes reported.

And it didn't work without setting that pref in Nightly before?

There's now too many people writing their "me too" to easily track what's going on.

Can you try the same with beta? Does it work then?

If "MOZ_DISABLE_RDD_SANDBOX = 1" is not defined, and "media.rdd-process.enabled" is enabled, video playback doesn't work in Nightly, in my pc (Intel i7 7500u, Intel 620).
In Firefox Beta, video playback works fine with default configuration, no changes needed. Same with Firefox stable.

Duplicate of this bug: 1683076

If "MOZ_DISABLE_RDD_SANDBOX = 1" is not defined, and "media.rdd-process.enabled" is enabled, video playback doesn't work in Nightly, in my pc (Intel i7 7500u, Intel 620).

And it works in Nightly if you set MOZ_DISABLE_RDD_SANDBOX?

Please try with the following prefs being set in about:config
media.rdd-process.enabled : true
media.rdd-wmf.enabled: false
media.rdd-ffvpx.enabled: false

You must restart Firefox after changing those preferences.

(In reply to Jean-Yves Avenard [:jya] from comment #32)

Please try with the following prefs being set in about:config
media.rdd-process.enabled : true
media.rdd-wmf.enabled: false
media.rdd-ffvpx.enabled: false

You must restart Firefox after changing those preferences.

This doesn't work. When the media.rdd-process.enabled is set to false, youtube videos play, however, youtube music doesn't work even with those settings.

Can someone try the moz regression tool to determine which change made it stop working?.
https://mozilla.github.io/mozregression/

PS: Just to add to some additional info:
My V85b2 has the media.rdd-process.enabled set to false and works... mostly in YouTube. However, no videos in Facebook work.

My V86a1 has the media.rdd-process.enabled set to true and works... everywhere, including Facebook.

Are you all using Windows N édition ?

(In reply to Jean-Yves Avenard [:jya] from comment #36)

Are you all using Windows N édition ?

I do: Windows 10 N Pro 64-bit (version 20H2, build 19042.685). The Media Feature Pack is enabled.

[.] removed

I could reproduce !

Install Norton 360 trial.

Playback won't start in beta.
It starts with nightly however.

Beta compiled locally works too for some reason.

Just suspending Norton 360 doesn't allow playback to start; it must be uninstalled.

Disabling the sandbox with the following steps allowed playback to work:

1- Close Firefox
2- Browser to the location of your Firefox install using Explorer
3- Shift + Right-click in the folder window where firefox.exe is located, select "Open powershell window here"
4- Add the environment variable(s) you wish to set to your command window with "$Env:MOZ_DISABLE_RDD_SANDBOX = 1"
5- Start firefox by typing .\firefox.exe

Severity: -- → S1
Component: Audio/Video: Playback → Security: Process Sandboxing
Priority: -- → P1
Summary: Media playback fails when RDD is enabled → Media playback fails when antivirus is installed and RDD is enabled

(In reply to Jean-Yves Avenard [:jya] from comment #39)

I could reproduce !

Install Norton 360 trial.

Playback won't start in beta.
It starts with nightly however.

Beta compiled locally works too for some reason.

Just suspending Norton 360 doesn't allow playback to start; it must be uninstalled.

Disabling the sandbox with the following steps allowed playback to work:

1- Close Firefox
2- Browser to the location of your Firefox install using Explorer
3- Shift + Right-click in the folder window where firefox.exe is located, select "Open powershell window here"
4- Add the environment variable(s) you wish to set to your command window with "$Env:MOZ_DISABLE_RDD_SANDBOX = 1"
5- Start firefox by typing .\firefox.exe

This seems to be working. I have Norton 360 as well.

Regressed by: 1620114

For now, we're just going to backout bug 1620114 in beta only ; once bug 1682609 lands in Nightly and ride the train, we will be good to go.

Assignee: nobody → jyavenard
Flags: needinfo?(toddthedeveloper)
Flags: needinfo?(bobowencode)
See Also: 16827911682609

This reverts commit https://hg.mozilla.org/mozilla-central/rev/9df12e4b7a00

for causing RDD's decoder failures and total media playback failures when an antivirus is installed.

Comment on attachment 9193915 [details]
Bug 1682834 - "Revert 1620114 - Enable pre-spawn CIG in RDD." r=bobowen

Beta/Release Uplift Approval Request

  • User impact if declined: No audio nor video can be played (YouTube, Facebook, two rather popular sites would become unusable).
  • Is this code covered by automated tests?: No
  • Has the fix been verified in Nightly?: No
  • Needs manual test from QE?: Yes
  • If yes, steps to reproduce: See https://bugzilla.mozilla.org/show_bug.cgi?id=1682834#c39 ; Install Norton 360 ; it doesn't even need to be enabled.

If you install firefox in C:\Program Files\Mozilla Firefox it will fail to run.

If installed in "C:\Program Files\Firefox Nightly" or any other locations; it will work !

  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): We revert bug 1620114 ; which was only recently added a couple of weeks ago. So we go back to an existing, and tested code path.
  • String changes made/needed: None
Attachment #9193915 - Flags: approval-mozilla-beta?
Flags: qe-verify+
Attachment #9193915 - Attachment description: Bug 1682834 - "Revert 1620114 - Enable pre-spawn CIG in RDD." r=bobowen" → Bug 1682834 - "Revert 1620114 - Enable pre-spawn CIG in RDD." r=bobowen

If you install firefox in C:\Program Files\Mozilla Firefox it will fail to run.
If installed in "C:\Program Files\Firefox Nightly" or any other locations; it will work !

This is really interesting! We've been trying for months to get https://bugzilla.mozilla.org/show_bug.cgi?id=1347710 enabled and saw similar behavior there.

once bug 1682609 lands in Nightly and ride the train, we will be good to go.

There are no plans to land this in Nightly, or ever, if it can be helped.

Disabling full CIG in beta seems like the best way forward.

For Nightly, we'll have to debug what Norton is doing that causes them to break security features. The failures here suggest that they inject themselves into RDD in a way that we can't see (even if the product is disabled!), and things then totally break if Windows tries to apply CIG protections to the RDD process because Nortons' product has already compromised it. We'll have to investigate what is up.

Comment on attachment 9193915 [details]
Bug 1682834 - "Revert 1620114 - Enable pre-spawn CIG in RDD." r=bobowen

Approved for 85.0b4.

Attachment #9193915 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Duplicate of this bug: 1682983

Thank you for taking care of this while I'm on PTO!

(In reply to Gian-Carlo Pascutto [:gcp] from comment #44)

If you install firefox in C:\Program Files\Mozilla Firefox it will fail to run.
If installed in "C:\Program Files\Firefox Nightly" or any other locations; it will work !

This is really interesting! We've been trying for months to get https://bugzilla.mozilla.org/show_bug.cgi?id=1347710 enabled and saw similar behavior there.

With Norton's trial (I needed to put my creditcard information to start download though), I could confirm their injection behavior was actually different between Nightly and Beta. While only symamsi.dll was injected into the browser process of Nightly, IPSEng64.dll (which is causing another bug 1626993) was injected into all processes of Beta. And, I reproduced the issue: RDD of Beta failed to start. Let me analyze their behavior.

(In reply to Gian-Carlo Pascutto [:gcp] from comment #46)

Disabling full CIG in beta seems like the best way forward.

For Nightly, we'll have to debug what Norton is doing that causes them to break security features. The failures here suggest that they inject themselves into RDD in a way that we can't see (even if the product is disabled!), and things then totally break if Windows tries to apply CIG protections to the RDD process because Nortons' junk product has already compromised it. We'll have to investigate what is up.

Now that my patch was backed out, we still enable CIG in RDD after a process launched. This means third-party modules are still blocked unless they are signed by Microsoft or are injected before we enable CIG. Media modules should be the former case, and IPSEng64.dll should be the latter. The problem here is blocking IPSEng64.dll somehow blocks an rdd process from launching. I got the repro. It should be a fun debug.

(In reply to Gian-Carlo Pascutto [:gcp] from comment #45)

once bug 1682609 lands in Nightly and ride the train, we will be good to go.

There are no plans to land this in Nightly, or ever, if it can be helped.

Unfortunately disabling pre-spawn CIG will not suffice to solve the compat issue with IBM. I'm landing bug 1682609 now. Once it's done, we don't need to back out the patch from 86+.

(In reply to Toshihito Kikuchi [:toshi] from comment #50)

Thank you for taking care of this while I'm on PTO!

(In reply to Gian-Carlo Pascutto [:gcp] from comment #44)

If you install firefox in C:\Program Files\Mozilla Firefox it will fail to run.
If installed in "C:\Program Files\Firefox Nightly" or any other locations; it will work !

This is really interesting! We've been trying for months to get https://bugzilla.mozilla.org/show_bug.cgi?id=1347710 enabled and saw similar behavior there.

With Norton's trial (I needed to put my creditcard information to start download though), I could confirm their injection behavior was actually different between Nightly and Beta. While only symamsi.dll was injected into the browser process of Nightly, IPSEng64.dll (which is causing another bug 1626993) was injected into all processes of Beta. And, I reproduced the issue: RDD of Beta failed to start. Let me analyze their behavior.

(In reply to Gian-Carlo Pascutto [:gcp] from comment #46)

Disabling full CIG in beta seems like the best way forward.

For Nightly, we'll have to debug what Norton is doing that causes them to break security features. The failures here suggest that they inject themselves into RDD in a way that we can't see (even if the product is disabled!), and things then totally break if Windows tries to apply CIG protections to the RDD process because Nortons' junk product has already compromised it. We'll have to investigate what is up.

Now that my patch was backed out, we still enable CIG in RDD after a process launched. This means third-party modules are still blocked unless they are signed by Microsoft or are injected before we enable CIG. Media modules should be the former case, and IPSEng64.dll should be the latter. The problem here is blocking IPSEng64.dll somehow blocks an rdd process from launching. I got the repro. It should be a fun debug.

I don't know if this helps, but if there's a registered "amsi" provider (most AVs), and the process calls amsi, then the amsi dll gets injected (both dlls: Windows Defender's and the commercial AV dll).
The thing is, many AVs "amsi" dll are not signed properly. For example, the amsi dll of Eset fails in the automatic security audit of Windows. That may explain why Windows Defender amsi dll doesn't fail with CIG enabled before process is started, but a commercial av amsi.dll does.
Read here: https://forum.eset.com/topic/25035-program-fileseseteset-nod32-antiviruseamsidll/

Status: NEW → RESOLVED
Closed: 7 months ago
Resolution: --- → FIXED
QA Whiteboard: [qa-triaged]

Confirming that this now works again in 85.0b4 as it used to.

Fix works in Firefox Beta. Doesn't work on Nightly

(In reply to donqk from comment #54)

Fix works in Firefox Beta. Doesn't work on Nightly

until a fix is found for getting around Norton/Symantec, I suggest you uninstall your antivirus. Or you can look in the earlier message on how to start Firefox with the RDD sandbox disabled.

Hi, This issue is Verified as Fixed in Beta 85.0b4 - tested on a Windows 10 x32, Please note that this issue was tested only with Norton 360 since Symantec needs the University Systems credentials to access and download the installer.

Please also note that our latest Nightly build installed in C:\Program Files\Firefox Nightly folder also works without issues.

(In reply to Jean-Yves Avenard [:jya] from comment #55)

(In reply to donqk from comment #54)

Fix works in Firefox Beta. Doesn't work on Nightly

until a fix is found for getting around Norton/Symantec, I suggest you uninstall your antivirus. Or you can look in the earlier message on how to start Firefox with the RDD sandbox disabled.

I've Eset Nod AV installed, so this problem is not unique to Norton users. I'll use it with RDD sandbox disabled, thanks.

See Also: → 1684532

(In reply to Gian-Carlo Pascutto [:gcp] from comment #44)

If you install firefox in C:\Program Files\Mozilla Firefox it will fail to run.
If installed in "C:\Program Files\Firefox Nightly" or any other locations; it will work !

This is really interesting! We've been trying for months to get https://bugzilla.mozilla.org/show_bug.cgi?id=1347710 enabled and saw similar behavior there.

Finally, I found the condition. Norton checks the combination of an executable's name and its path. If a process is firefox.exe, Norton injects IPSEng64.dll if the path contains a string \mozilla firefox\ (case-insensitive).

In v86, the compat issue with Norton 360 should be fixed by bug 1684532.

Hi, I have retested this issue by installing older versions of Nightly in the C:\Program Files\Mozilla Firefox\ path and I was able to reproduce it, I can also confirm that the issue is Verified as Fixed in our latest version of Nightly 86.0a1 (2021-01-18).

(In reply to Rares Doghi from comment #60)

Hi, I have retested this issue by installing older versions of Nightly in the C:\Program Files\Mozilla Firefox\ path and I was able to reproduce it, I can also confirm that the issue is Verified as Fixed in our latest version of Nightly 86.0a1 (2021-01-18).

Thank you for verifying the latest Nightly! Glad to hear the issue was fixed.

Hi, can we update the Tracking and Status flags for this issue to Verified ?

Flags: needinfo?(jya-moz)
Status: RESOLVED → VERIFIED
Flags: needinfo?(jya-moz)

Removing the Qe-Verify flags, Thank you

QA Whiteboard: [qa-triaged]
Flags: qe-verify+
You need to log in before you can comment on or make changes to this bug.