Closed Bug 1385207 Opened 3 years ago Closed 2 years ago

Audio over RDP connections not working with sandbox level 3 on Windows

Categories

(Core :: Audio/Video: cubeb, defect, P3)

56 Branch
x86
Windows 10
defect

Tracking

()

RESOLVED WORKSFORME
Tracking Status
relnote-firefox --- 58+
firefox56 --- wontfix
firefox57 + wontfix
firefox58 --- wontfix
firefox59 --- fix-optional
firefox60 --- ?

People

(Reporter: ilja, Assigned: handyman)

References

(Depends on 1 open bug)

Details

(Keywords: regression, Whiteboard: sb+)

Attachments

(1 file)

Remote audio is currently not working in Nightly.
Not entirely sure when the issue occurred, tested and verified on: 56.0a1 (2017-07-27)
Reproduced by streaming audio from Nightly over RDP to a remote connection.
OS Windows 10 Pro x64, version 1703.
Works as expected with latest beta (55.0 beta 13).
Ilja,

Thanks for the report. I can reproduce will the following setup:
- Windows 10 machine running today's Firefox Nightly
- Ubuntu machine that runs `xfreerdp` like so: `xfreerdp   /sound /p:passwd /u:user /v:host`

I have sound in release and beta, but not Nightly.

I'll have a look hopefully soon.
Flags: needinfo?(ilja)
Component: Untriaged → Audio/Video: Playback
Product: Firefox → Core
This is broken by the sandbox level 3. Bob, can you tell me how to go forward here? I've tried a bunch of stuff to try to have a logging of the sandbox violations, but nothing works.
Flags: needinfo?(bobowencode)
(In reply to Paul Adenot (:padenot) from comment #2)
> This is broken by the sandbox level 3. Bob, can you tell me how to go
> forward here? I've tried a bunch of stuff to try to have a logging of the
> sandbox violations, but nothing works.

The only logging we have is for things for which we can add policy rules, where the sandbox hooks various functions.

To do this I tend to set the following env vars.

MOZ_SANDBOX_LOGGING=1
MOZ_LOG=SandboxBroker:5,SandboxTarget:5,sync
MOZ_LOG_FILE=<whatever>

If you're on a DEBUG build then just the first one should do and it logs as warnings, it also logs to the browser console.

But the audio issues that we've had in the past don't show up in the logging that we have, I'm afraid.
Flags: needinfo?(bobowencode)
Ok, found where it errors out manually, it's a call to `IMMDeviceEnumerator::GetDefaultAudioEndpoint` that returns 0x80070490 aka "not found".

Bob, short on finishing the audio remoting, do we have any options here?
Flags: needinfo?(bobowencode)
(In reply to Paul Adenot (:padenot) from comment #4)
> Ok, found where it errors out manually, it's a call to
> `IMMDeviceEnumerator::GetDefaultAudioEndpoint` that returns 0x80070490 aka
> "not found".
> 
> Bob, short on finishing the audio remoting, do we have any options here?

I don't think we'll be able to add policy rules for that.
Could we just proxy that API call? 
I don't suppose the work in bug 1361336 will help us here.
Flags: needinfo?(bobowencode)
[Tracking Requested - why for this release]:

This issue breaks audio for people using RDP.
Component: Audio/Video: Playback → Audio/Video: cubeb
Flags: needinfo?(kinetik)
Keywords: regression
Priority: -- → P1
Flags: needinfo?(kinetik)
It might be worth testing the old "winmm" backend with RDP and the sandbox.  If it works, it'd give us a partial (i.e. no full duplex/capture for WebRTC) fallback until we have a better fix in hand.

It's fairly easy to test by creating a new string preference in about:config called "media.cubeb.backend", setting the value to "winmm", then restarting and testing.
Tracking 57+ based on Comment 6.
Status: UNCONFIRMED → NEW
Ever confirmed: true
I tried this just now by creating media.cubeb.backend and setting it to winmm, then restarting. Same issue unfortunately, no audio over RDP.
Flags: needinfo?(ilja)
I don't think audio over RDP is that critical, but maybe there's a valid use case. ajones, can we ship with this considering we don't want to downgrade sandbox levels and cubeb remoting for windows won't be ready in 57.
Blocks: 1366694
Flags: needinfo?(ajones)
Summary: Remote audio not working → Audio over RDP connections not working in 57
Whiteboard: sb+
(In reply to Jim Mathies [:jimm] from comment #10)
> I don't think audio over RDP is that critical, but maybe there's a valid use
> case. ajones, can we ship with this considering we don't want to downgrade
> sandbox levels and cubeb remoting for windows won't be ready in 57.

We don't know how much audio over RDP is used but I'd rather not take the change in breaking it without having a way to measure the impact. In the absence of numbers that show that this is an obscure use case, I recommend we back out the sandboxing changes that break RDP.
Flags: needinfo?(ajones)
Rank: 23
Priority: P1 → P2
Mass change P2->P3 to align with new Mozilla triage process.
Priority: P2 → P3
(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #11)
> (In reply to Jim Mathies [:jimm] from comment #10)
> > I don't think audio over RDP is that critical, but maybe there's a valid use
> > case. ajones, can we ship with this considering we don't want to downgrade
> > sandbox levels and cubeb remoting for windows won't be ready in 57.
> 
> We don't know how much audio over RDP is used but I'd rather not take the
> change in breaking it without having a way to measure the impact. In the
> absence of numbers that show that this is an obscure use case, I recommend
> we back out the sandboxing changes that break RDP.

We have some evidence.
When I try this I get a 12 (CUBEB_BACKEND_INIT_FAILURE_FIRST) for AUDIOSTREAM_BACKEND_USED.
So looking at the evolution for the percentage of these on Nightly and Beta, there doesn't seem to be a significant change in the percentage of 12s in 56 where we turned this on.
In fact Beta 55 seemed a fair bit worse, although that had gone up since 53 and 54.
(The telemetry dashboards are broken for me at the moment or I'd provide links.)

So, it seems the number of people attempting to use audio over RDP is at least swamped by the usual errors we get for this.

It is the USER_LIMITED access token level that appears to be causing this, so we'd loose the blocking of read access to the user's own files here.
One thought is that we could have something on the release notes about this known issue, with instructions for lowering the sandbox level to 2 for this use case.
Flags: needinfo?(jmathies)
Flags: needinfo?(jmathies)
The code here [1] is working for detecting remote sessions, I was being thrown before because I was expecting the remote session to also be in a job and so hit our current use of GetSystemMetrics(SM_REMOTESESSION).

Here's a try push with a drop to USER_INTERACTIVE for remote sessions so audio isn't broken, this does of course mean that these session also lose the read protection for the user's own files.
However, we might decide to go with this depending on how long this is likely to be broken.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=745b29da039ee1ed478495ef10ac20bdbc796ebe

It's probably only marginally useful but this also sort of works dynamically.
If you start Firefox in a local session and then connect via RDP, while tabs in any existing processes won't have audio, if new processes are launched while in the remote session then the audio works.

I've only tested with Windows RDP at the moment.

While almost certainly too late for Fx56, if we decide to go with this I'll uplift to 57 and it could be considered for a ride along.
I'll file another bug to add some telemetry, so it can be considered for uplift separately.

[1] https://msdn.microsoft.com/en-us/library/aa380798.aspx
Changed this slightly from the try push to move IsCurrentSessionRemote into WinUtils.

Still wonder whether this should be behind a pref.
Attachment #8909774 - Flags: review?(jmathies)
Assignee: nobody → bobowencode
Status: NEW → ASSIGNED
I think we may be looking to get something into the Fx56 release notes for this.

Release Note Request (optional, but appreciated)
[Why is this notable]:
Latest Windows content sandbox policy breaks audio over Remote Desktop Connection.

[Affects Firefox for Android]:
None.

[Suggested wording]:
If you are running Firefox on Windows over a Remote Desktop Connection, audio is currently broken due to new content process sandboxing restrictions.
If you must have audio working then you can reduce the strength of the sandbox policy in about:config by setting the following and restarting Firefox:
  security.sandbox.content.level = 2

Note that this does reduce the protection the process sandbox affords you if your browsing session were to be compromised, so use this option with caution.

[Links (documentation, blog post, etc)]:
relnote-firefox: --- → ?
Comment on attachment 8909774 [details] [diff] [review]
Don't use USER_LIMITED token level for content process when in remote session

will review if we decide to land. I don't want to r+ this now and confuse things.
Attachment #8909774 - Flags: review?(jmathies)
Summary: Audio over RDP connections not working in 57 → Audio over RDP connections not working with sandbox level 3 on Windows
We should likely reword this (I commented in the 56 relnotes draft).
FYI. Rewording by Michelle:  Users running Firefox for Windows over a Remote Desktop Connection (RDP) may find that audio playback is disabled due to increased security restrictions. Learn how to mitigate this issue until it is corrected in an upcoming release. (There will be a link to a SUMO page)
Duplicate of this bug: 1407831
Duplicate of this bug: 1426580
Depends on: 1432303
Hi Matthew, just wondered if you had any insight on this bug, while we're working on fixing it with audio remoting?
Flags: needinfo?(mvaneerde)
Please file a problem report in Feedback Hub with "Audio glitches" logs of a (failed) attempt to play audio.

https://blogs.msdn.microsoft.com/matthew_van_eerde/2016/09/26/report-problems-with-logs-and-suggest-features-with-the-feedback-hub/

Once filed, grab a direct link from Feedback Hub / Feedback / My feedback / (click on the problem report) / Share

Send me the link to the problem report and cite this bug
Flags: needinfo?(mvaneerde)
See Also: → 1432577
Duplicate of this bug: 1410076
Depends on: 1432577
Duplicate of this bug: 1423994
up ! This is flabbergasting that there's no fix yet. Audio & security.. really ? What about video then ?

This is a REAL problem in VDI environment. Obviously noone hears it, as there's no sound ;-)
Assignee: bobowencode → nobody
Status: ASSIGNED → NEW
Great move....
Good News:
since Version Firefox60 > there is at last again sound in RDC-Session.
I´ve checked out in several user sessions, with YouTube and other streams.... it works fine :-)
Thank you very much for supporting us.

Best Regards 
Ralf
(In reply to VGWORT from comment #33)
> Good News:
> since Version Firefox60 > there is at last again sound in RDC-Session.

Looks like this is down to bug 1358372.
I'm not sure if the audio session can ever require re-initialising after the sandbox is lowered, as that might mean audio would be lost at that point.
Depends on: 1358372
Flags: needinfo?(davidp99)
Cool that the volume slider fix helped this as well.  Wish I could say I did it on purpose.

It is rare that the AudioSession is restarted but it definitely happens.  I believe the case is when audio (not just one audio device) is turned off and then restored.  So, e.g. plugging and unplugging headphones should not trigger a restart but disabling all audio devices will.

(I just confirmed this with breakpoints -- OnSessionDisconnected was only called once I disabled all audio devices by opening "Playback devices" from the speaker in the system tray, then right-clicking and disabling all entries.)

The code that restarts it is here [1].  The comment there is telling.  ("If it fails there's not much we can do.")

[1] https://searchfox.org/mozilla-central/rev/00dd116638001772fe354b081353b73f1cad405d/widget/windows/AudioSession.cpp#469
Flags: needinfo?(davidp99)
FYI, Bob mentioned that, after some local experimentation, he believes that the (default) RDP audio endpoint may not be disable-able.  So it may be the case that even the edge case mentioned in comment 35 is not possible.  Even if it is, it is probably extremely rare.  We are resolving this for now.
Assignee: nobody → davidp99
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → WORKSFORME
Audio works if you connect to RDP, then start Firefox. However if you Firefox is already running on the remote computer, audio does not work.
Yes, that sounds like a case that I hadn't fully considered.  The precise (technical) condition would actually be that the tab was running in a content process that had been created before the RDP session was established.  The process' AudioSession will fail to be recreated.  Restarting the browser, or otherwise killing/recreating the tab process, would be required to fix it.

I thought my rare example in comment 35 was going from one device to zero, then back to one.  Bob's discovery (comment 36) suggests I wasn't.  sriram's example does.
(In reply to sriram from comment #37)
> Audio works if you connect to RDP, then start Firefox. However if you
> Firefox is already running on the remote computer, audio does not work.

I can confirm that using Firefox 60.0.1 on a remote desktop session, I could restore the audio of a YouTube video by 
1. closing Firefox
2. closing and re-opening the RDP session
3. opening Firefox again

without lowering the sandbox level. 

The YouTube video was originally opened after the RDP session was established and did not have audio playback, however, I opened this tab in an existing tab group (using Simple Tab Group add-on). After step 3 the tab was again opened in the same tab group, this time with audio playback.
Migrating from W2008 R2 RDSH to W2016. Firefox being our preferred browser. 
We bumped several times that black screen of death. If no fix is provided rapidly we'll have to move to some other browser.

I had open another bug https://bugzilla.mozilla.org/show_bug.cgi?id=1497177
See Also: → 1497177
See Also: → 1553651
You need to log in before you can comment on or make changes to this bug.