1474175 - Wrong audio channel selection on multi channel audio interfaces (MacOS)

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P3)

Description

[bugreports]

Attachment: system.log

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:61.0) Gecko/20100101 Firefox/61.0
Build ID: 20180704003137

Steps to reproduce:

Add an external (multichannel in my case) USB sound interface to a Mac
Go to the system sound setup, select the interface
Open the system Audio MIDI Setup, select the external interface and click on "Configure speakers)
Select the output channel to use for the left and right speaker (17+18 in my case on a Behringer XR18 digital mixer)
All apps (Safari, Chromium, Logic Pro etc.) are now playing sound by default on that outputs, system noise will also be sent there



Actual results:

Starting with version 61.0.0 Firefox does not play sound on MacOS any more when an external USB interface is connected and selected as the default system output. Version 60.x.x worked just fine, the problem started right after the update to 61.0.0. Also, when switching back to internal speakers, Firefox 61.0.0 is playing sound again. This behavior is independent of the web site being used, it happens on all web sites.
Each time I am opening Firefox or a new tab, I am getting entries in the /var/log/system.log (see attached file). Not sure if that's related to the problem though.


Expected results:

Firefox should play sound on the system's selected sound device/channels

Comment 1

[Daniel Bodea [:danibodea]]

We cannot confirm this issue's reproduction due to the lack of hardware needed to reproduce it.

I will set this issue's component as Core: Audio/Video: Playback and hope it can be addressed from there.
This issue appears to only be reproduced on Firefox Release.

@bugreports: Can you please create a new profile, you have the steps here: https://support.mozilla.org/en-US/kb/profile-manager-create-and-remove-firefox-profiles?redirectlocale=en-US&redirectslug=Managing-profiles#w_starting-the-profile-manager
and see if the issue is still reproducing?

Thank you for your contribution!

Comment 2

[Dan Minor [:dminor]]

I was unable to reproduce this using Nightly and a cheap Trond AC2 usb audio adapter, but that setup does not have multiple output channels from which to choose.

Does the issue reproduce with Nightly and the Behringer hardware?

Comment 3

[bugreports]

I created a brand new user account on the Mac to get a fresh profile and started up FireFox 61.0.1. No change in the behavior. It works on internal speakers, no sound on the external audio interface. Same thing with Nightly 63.0a1 (2018-07-11).

BUT: Dan's comment "... output channels... choose..." gave me the idea of switching the Behringer mixer's channels 1+2 from analog input to USB input and voila - Firefox's sound was there. So instead of using the system settings (saying channels 17+18) it just sends the sound to channels 1+2. Maybe the bug description should be changed to something like "wrong audio channel selection on multi channel audio interfaces" then.

Comment 4

[Dan Minor [:dminor]]

Thanks for the additional testing! I'll update the bug summary.

:padenot, does this seem like a known problem?

Comment 5

[Paul Adenot (:padenot)]

Yes. jya has ordered a similar sound card (not exactly the same), and will investigate.

Comment 6

[alfski]

Can I poke this? I am trying to use a SoundBlaster Creative X3 7.1 USB audio device for multichannel web audio on MacOS.

Web audioContext.destination.maxChannelCount reports 8 channels, as expected. I believe this was only fixed a few months ago.
However, when playing multichannel audio however everything seems to be downmixed to stereo.

Adjusting the example at https://developer.mozilla.org/en-US/docs/Web/API/AudioBuffer/getChannelData to play audio on ALL channels, still comes out as stereo.

The MacOS AudioMidi Setup tool and native media players eg. mpv play multichannel audio correctly when using the X3 audio device.

A good test of multichannel audio for the web is here https://www2.iis.fraunhofer.de/AAC/multichannel.html
That pages plays in surround find with the X3 device and Windows 10 browsers, including Firefox.

Cheers, Andrew

Comment 7

[Paul Adenot (:padenot)]

(In reply to alfski from comment #6)

Web audioContext.destination.maxChannelCount reports 8 channels, as expected. I believe this was only fixed a few months ago.
However, when playing multichannel audio however everything seems to be downmixed to stereo.

AudioContext.destination.maxChannelCount tells you the maximum channel count of the configuration, that has always had the correct number (since I implemented it for release in Firefox 25).

The default channel count for an AudioContext is always stereo, but this can be changed up to maxChannelCount. This is controlled by AudioContext.destination.channelCount, that is initially 2, and can be set up to 8 in on your system it seems.

If you feed, say, 8 channels into the destination, that hasn't seen its channelCount changed from the default of 2, it will perform a down-mix step. I'll note in passing that a bug in the Web Audio API specification is to only define channel layouts up to 5.1 (6 channels), this will be fixed at some point.

Comment 8

[alfski]

Hi, apologies I thought I read the maxChannelCount was only working recently -- may be another browser.

Setting the system to 6 channels, and using a modified version of the 'white noise' sample code, all I hear is a 'pop' in the front pair of speakers when maxChannels is 6.

If I set maxChannels to 2 I get 0.5 seconds white noise in front pair, as expected.
Anything beyond 2 is just a 'pop' in the front pair of speakers.

Interestingly, if I skip loading data into first 4 channels and only load up the last two channels which should play in Rear Left/Right, instead I get 0.5 seconds of white noise in the Front speakers.

Sample javascript https://goo.sstars.ws/chivecam/mozilla.html

Everything looks okay in console log, but I just get a 'pop' -

maxChannels 6
now channelCount: 6
channels: 6
frameCount: 24000
sampleRate: 48000
myArrayBuffer: 
AudioBuffer { sampleRate: 48000, length: 24000, duration: 0.5, numberOfChannels: 6 }```

Creative support tell me this is a "problem with browsers" and not their problem. 
The X3 and Omni multichannel sound devices are otherwise great little pieces of kit.. they just don't work in any MacOS browsers!

Comment 9

[Paul Adenot (:padenot)]

Your code in incorrect, and produces noise with a heavy DC offset:

Math.random() * 0.4 - 1

Yields value in [-1, -0.6], and this can provoke a pop (depending on the OS/hardware configuration). The correct code would be:

0.8 * Math.random() * 2 - 1

This produces noise with an amplitude of 0.8.

With this fixed, can you do the following on the command line, with a Firefox Nightly build that installs fine alongside a regular Firefox:

MOZ_DISABLE_CONTENT_SANDBOX=1 MOZ_DUMP_AUDIO=1 /Applications/FirefoxNightly.app/Contents/MacOS/firefox

and run your code, this will drop two files named GraphDriverInput-1.wav and GraphDriverOutput-2.wav that should be WAV files with a number of channels identical to what you've set on the DestinationNode. Can you attach GraphDriverOutput-2.wav here?

Comment 10

[alfski]

Attachment: alfski-2020803-GraphDriverOutput-2.wav

Comment 11

[alfski]

(In reply to Paul Adenot (:padenot) from comment #9)

Your code in incorrect, and produces noise with a heavy DC offset:

Math.random() * 0.4 - 1

Yields value in [-1, -0.6], and this can provoke a pop (depending on the OS/hardware configuration). The correct code would be:

0.8 * Math.random() * 2 - 1

This produces noise with an amplitude of 0.8.

Thanks! code was recycled from a codepen at https://stackoverflow.com/questions/29185251/what-is-the-maximum-number-of-outputs-for-a-web-audio-context-and-buffer

Now plays white noise for 0.5 second even when maxChannels > 2, but still only on front stereo channels.

With this fixed, can you do the following on the command line, with a Firefox Nightly build that installs fine alongside a regular Firefox:

MOZ_DISABLE_CONTENT_SANDBOX=1 MOZ_DUMP_AUDIO=1 /Applications/FirefoxNightly.app/Contents/MacOS/firefox

and run your code, this will drop two files named GraphDriverInput-1.wav and GraphDriverOutput-2.wav that should be WAV files with a number of channels identical to what you've set on the DestinationNode. Can you attach GraphDriverOutput-2.wav here?

oky, output wav attached here. alfski-20200803-GraphDriverOutput-2.wav

ffprobe says 6 channel wav, and so does Audacity. So that looks.. ummm.. correct?

Input #0, wav, from 'alfski-2020803-GraphDriverOutput-2.wav':
  Duration: 00:00:05.49, bitrate: 4608 kb/s
    Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, 6 channels, s16, 4608 kb/s

Comment 12

[Paul Adenot (:padenot)]

Ok that looks correct, Firefox writes 6 channels of white noise to the output. Can you pleaserepeat the procedure with the following command:

MOZ_DISABLE_CONTENT_SANDBOX=1 MOZ_LOG=cubeb:5 MOZ_LOG_FILE=firefox.log /Applications/FirefoxNightly.app/Contents/MacOS/firefox

this will create a number of files called firefox.log with a number at the end, zip them up and attach here, this will have data to explain how the audio IO layer of Firefox decides to mix/route the audio.

Comment 13

[alfski]

Attachment: alfski-20200803-multichannel-logs.zip

Comment 14

[alfski]

oky, zip bundle attached.

The bit that says "Incompatible channel layouts detected, setting up remixer"
and "out channel count: 6, output channels: [FrontLeft, FrontRight, Silence, Silence, Silence, Silence]" looks like a clue!

[Child 2792: CubebOperation #1]: E/cubeb mod.rs:2629: (0x14ee45400) Initialize output by device info: device_info { id: 71, flags: DEV_OUTPUT | DEV_SYSTEM_DEFAULT | DEV_SELECTED_DEFAULT }
[Child 2792: CubebOperation #1]: E/cubeb mod.rs:2640: (0x14ee45400) Opening output side: rate 48000, channels 6, format Float32LE, layout FRONT_LEFT | FRONT_RIGHT | FRONT_CENTER | LOW_FREQUENCY | SIDE_LEFT | SIDE_RIGHT, prefs NONE, latency in frames 128.
[Child 2792: CubebOperation #1]: E/cubeb mod.rs:2678: (0x14ee45400) Output hardware description: AudioStreamBasicDescription { mSampleRate: 48000.0, mFormatID: 1819304813, mFormatFlags: 9, mBytesPerPacket: 24, mFramesPerPacket: 1, mBytesPerFrame: 24, mChannelsPerFrame: 6, mBitsPerChannel: 32, mReserved: 0 }
[Child 2792: CubebOperation #1]: E/cubeb mod.rs:2699: Incompatible channel layouts detected, setting up remixer
[Child 2792: CubebOperation #1]: E/cubeb mixer.rs:182: Create a mixer with input channel count: 6, input layout: FRONT_LEFT | FRONT_RIGHT | FRONT_CENTER | LOW_FREQUENCY | SIDE_LEFT | SIDE_RIGHT, out channel count: 6, output channels: [FrontLeft, FrontRight, Silence, Silence, Silence, Silence]

Comment 15

[Paul Adenot (:padenot)]

Chun-Min, can you find why this is happening ?

The input is 5.1 SMPTE-type mapping on a device that supports it, but the mixer decides that it's incompatible, and drops the 4 last channels. This should probably be pass-through without remixer, or just a shuffling of the channels.

Comment 16

[C.M.Chang[:chunmin]]

Hi Andrew,

What is the channel layout you see from Audio MIDI Setup > Configure Speakers, for 5.1 Surround? That would show the current channel layout in use.

From the log attached, I would expect the current audio layout is: <Left: 1, Right: 2>. Is the layout being used set to stereo? If the layout is set to 5.1 Surround with <Left: 1, Right: 2, Center: 3, Subwoofer: 4, Left surround: 5, Right surround: 6>, does the audio works as what you expect?

According to the log, the mixer is created since the channel-layout of the audio source is different from the current audio layout in use.

• The channel-layout of the audio source is: [FrontLeft, FrontRight, FrontCenter, LowFrequency, SideLeft, SideRight] (6 channels)
• The channel-layout of the output hardware is: [FrontLeft, FrontRight, Silence, Silence, Silence, Silence] (6 channels)

Comment 17

[alfski]

Attachment: alfski-20200819-CreativeX3-AudioLayoutMacOS.jpg

Comment 18

[alfski]

Hi, i've just attached a screenshot of what Audio MIDI Setup Tool is showing me. The 'Sound Blaster X3 Speakers' with '0 ins/6 outs' is the default or selected output device.

This 5.1 speaker config works great for surround audio when using "mpv" to play test files.
I can't see anything that says "stereo output selected" or downmix to stereo?

I'm not sure what else I can give you to show what the "audio layout" is.

Could this be a Creative problem? At the moment they are saying it's not their problem, and to "Contact the browser makers".
Thanks for helping with this.

Andrew

Comment 19

[C.M.Chang[:chunmin]]

Attachment: Screen Shot 2020-08-19 at 9.07.30 AM.png

(In reply to alfski from comment #18)

I'm not sure what else I can give you to show what the "audio layout" is.

Hi,

The channel-layout can be set when clicking the "Configure Speakers" button in the right-bottom of Attachment 9170921 [details]

Can you show the screenshot of the window pop up? The window is something like my attachment here.

Comment 20

[C.M.Chang[:chunmin]]

Attachment: Screen Shot 2020-08-19 at 9.07.16 AM.png

(In reply to C.M.Chang[:chunmin] from comment #19)
And when you click the "Configuration" button on the popup window, you can set the channel layout to "stereo", "2.0 Surround", "5.1 Surround",... etc. I guess your channel layout is set to "stereo" or "2.0 Surround". Can you switch it to "5.1 Surround" and see if you can get what you expect? If not, can you please attach the screenshot of the Speak-Channel map shown in the popup window? It would be helpful to know what the channel layout you are currently using.

Comment 21

[alfski]

Attachment: alfski-20200819-X3ChannelLayout.jpg

Comment 22

[alfski]

Hi, Channel Layout screenshot attached. You can see 5.1 Surround is selected. Have been using this all along.

Interestingly even if I select '2.0 Surround', MPV player still plays audio out ALL 6 speakers!!

Comment 23

[C.M.Chang[:chunmin]]

Hi,

Can you get the log again, by the command in comment #12, after applying the layout to "5.1 Surround"? Does the audio go to the channel as what you expect?

From the screenshot you provided, the hardware channel layout info for the mixer log should be something like:

Create a mixer with .... output channels: [FrontLeft, FrontRight, FrontCenter, LowFrequency, BackLeft, BackRight]

What is the Mac OS version you are currently on? My Mac OS 10.15 maps the channel to a number, while it maps to a label name on your side.

Comment 24

[alfski]

Attachment: alfski_X3_51Channels_firefox.log.child-5.moz_log

Comment 25

[alfski]

Attachment: alfski_X3_2Channels_firefox.log.child-4.moz_log

Comment 26

[alfski]

Hi, log snippets attached.

Interesting... regardless of whether I select 2 Channel or 5.1 Channel with 'Configure Speakers' the log output is the same.

5.1 selected

[Child 13187: CubebOperation #1]: E/cubeb mixer.rs:182: Create a mixer with input channel count: 6, input layout: FRONT_LEFT | FRONT_RIGHT | FRONT_CENTER | LOW_FREQUENCY | SIDE_LEFT | SIDE_RIGHT, out channel count: 6, output channels: [FrontLeft, FrontRight, Silence, Silence, Silence, Silence]

2 channel selected:

[Child 13222: CubebOperation #1]: E/cubeb mixer.rs:182: Create a mixer with input channel count: 6, input layout: FRONT_LEFT | FRONT_RIGHT | FRONT_CENTER | LOW_FREQUENCY | SIDE_LEFT | SIDE_RIGHT, out channel count: 6, output channels: [FrontLeft, FrontRight, Silence, Silence, Silence, Silence]

Comment 27

[C.M.Chang[:chunmin]]

Create a mixer with input channel count: 6, input layout: FRONT_LEFT | FRONT_RIGHT | FRONT_CENTER | LOW_FREQUENCY | SIDE_LEFT | SIDE_RIGHT, out channel count: 6, output channels: [FrontLeft, FrontRight, Silence, Silence, Silence, Silence]

From the two logs uploaded, it seems SoundBlaster Creative X3 7.1 USB is configured to 6 channels with stereo front-left and front-right anyway, even if you set it to [FrontLeft, FrontRight, FrontCenter, LowFrequency, BackLeft, BackRight]. It's weird.

• What is the Mac OS version you have?
• Does the sound go to the right channel when you click the "Test" button on the "Configure Speakers" window?
• How does the audio sound when you shuffle the speaker-channel pair (e.g., Set "Centre" to "Front Left", "Right Surrond" to "Front Right", ...)?

Comment 28

[alfski]

Attachment: alfski_X3_SB2Channel.moz_log

Comment 29

[alfski]

Hi, macOS Catalina 10.15.6. I think Creative may have a driver problem!?

I just found if use the SoundBlaster (SB) Command tool to set the X3 to Stereo THAT does actually make a stereo output device, see X3_SB2Channel log attached.

The SBCommand tool does NOT have a "speaker configurator" like Audio MIDI tool. SB Command has set "2.0, 5.1, or 7.1" output modes.
I can re-assign and test speaker swaps when using stereo (Front Left <> Front Right) within the 'Audio MIDI Setup' tool and it works as you'd expect.

When using multi-channels I use the SB Command Tool set to "5.1" at that immediately shows up in Audio MIDI tool with the 6 channels.

But when in 5.1Channel mode none of the speaker swaps work in Audio MIDI Tool!
They "noise test" correctly, but nothing changes after Apply/Done.
Everything stays on FL=FL,FR=FR,C=C,LFE=LFE, etc and no applications I tested see a swap (vlc, browsers, mpv).

This is getting annoying. Might just have to install Bootcamp hah.

Comment 30

[Oliver Günther]

This issue appears related to my issue: I'm using a Focusrite Vocaster One USB audio interface to connect an external microphone to my Mac and use it in web conferences.

in about:support it is shown as 10-channel input device. When connecting to any WebRTC conference, and selecting the microphone as input, other users can hear my actual microphone channel as well as the loopback channel (computer audio). So when I play music, an alert sound or similar, it is being picked up as if it were microphone input.

Input devices
Name 	Group 	Vendor 	Status 	Preferred 	Format 	Channels 	Rate 	Latency
MacBook Pro-Mikrofon	builtin-internal-mic|spk	Apple Inc.	Enabled	None	default: F32LE, support: S16LE S16BE F32LE F32BE	1	default: 48000, support: 44100 - 96000	2414 - 6495
Vocaster One USB	Vocaster One USB:1235:8216	Focusrite	Enabled	All	default: F32LE, support: S16LE S16BE F32LE F32BE	10	default: 48000, support: 48000 - 48000	63 - 4144

Comment 31

[Josh]

I've got what seems to be a variant of this issue, or something like it. I have an RME Fireface UCX II USB audio interface. It's got a bunch of I/O and exposes 20 discrete channels of "software playback" to the OS. In Audio Midi Setup's Configure Speakers dialog I have it set to:

Configuration: Stereo

Speaker Left: Analog 3
Speaker Right: Analog 4

Every application that I've tried respects this speaker configuration except for Firefox -- instead, it uses Analog 1/2 as Left/Right.

I downloaded Firefox Nightly and ran it with the following env vars set:

MOZ_DISABLE_CONTENT_SANDBOX=1 MOZ_LOG=cubeb:5 MOZ_LOG_FILE=firefox.log

In the logs I see:

[Parent 26361: AudioIPC Server RPC]: E/cubeb mod.rs:248: Using the system default device
[Parent 26361: AudioIPC Server RPC]: E/cubeb mod.rs:2926: Use global latency 512 instead of the requested latency 4800.
[Parent 26361: AudioIPC Server RPC]: E/cubeb mod.rs:3421: Output device ID: 106 (aggregate: false)
[Parent 26361: AudioIPC Server RPC]: E/cubeb mod.rs:3798: (0x14fa52c00) Initialize output by device info: device_info { id: 106, flags: device_flags(DEV_OUTPUT | DEV_SELECTED_DEFAULT) }
[Parent 26361: AudioIPC Server RPC]: E/cubeb mod.rs:3804: (0x14fa52c00) Opening output side: rate 48000, channels 2, format Float32LE, layout FRONT_LEFT | FRONT_RIGHT, prefs 0x0, latency in frames 512, v
oice processing false.
[Parent 26361: AudioIPC Server RPC]: E/cubeb mod.rs:3834: (0x14fa52c00) Output hardware description: AudioStreamBasicDescription { mSampleRate: 44100.0, mFormatID: 1819304813, mFormatFlags: 41, mBytesPerPacket: 4, mFramesPerPacket: 1, mBytesPerFrame: 4, mChannelsPerFrame: 2, mBitsPerChannel: 32, mReserved: 0 }
[Parent 26361: AudioIPC Server RPC]: E/cubeb mod.rs:3903: (0x14fa52c00 Using output device channel layout [Discrete, Discrete, FrontLeft, FrontRight, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete]
[Parent 26361: AudioIPC Server RPC]: E/cubeb mod.rs:3917: Incompatible channel layouts detected, setting up remixer
[Parent 26361: AudioIPC Server RPC]: E/cubeb mixer.rs:183: Creating a mixer with input channel count: 2, input layout: FRONT_LEFT | FRONT_RIGHT,out channel count: 2, output channels: [Discrete, Discrete, FrontLeft, FrontRight, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete, Discrete]
[Parent 26361: AudioIPC Server RPC]: E/cubeb mixer.rs:83: Create an floating type(f32) mixer
[Parent 26361: AudioIPC Server RPC]: E/cubeb mod.rs:1496: The buffer frame size of AudioUnit 0x866f90af for OUTPUT is already 512
[Parent 26361: AudioIPC Server RPC]: E/cubeb mod.rs:3994: (0x14fa52c00) Output audiounit init with device 106 successfully.
[Parent 26361: AudioIPC Server RPC]: E/cubeb cubeb_resampler_internal.h:521:Input and output sample-rate match, target rate of 48000Hz
[Parent 26361: AudioIPC Server RPC]: E/cubeb mod.rs:2970: (0x14fa52c00) Cubeb stream init successful.

Any clues there? It doesn't exactly look like it's obviously doing something wrong, but I'm not familiar with the code. Let me know if there's any other information I can gather for you!

Comment 32

[Paul Adenot (:padenot)]

Patch for this in https://github.com/mozilla/cubeb-coreaudio-rs/pull/222, sorry it took so long.

Comment 33

[Josh]

Thank you!!!

Comment 34

[Paul Adenot (:padenot)]

Attachment: Bug 1474175 - Update cubeb-coreaudio-rs to 4cc24c7. r?#cubeb-reviewers

Comment 35

[Paul Adenot (:padenot)]

Attachment: Bug 1474175 - mach vendor rust. r?#cubeb-reviewers

Depends on D211873

Comment 36

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/cffa168178e1
Update cubeb-coreaudio-rs to 4cc24c7. r=cubeb-reviewers,pehrsons
https://hg.mozilla.org/integration/autoland/rev/cad236ee0207
mach vendor rust. r=cubeb-reviewers,pehrsons

Comment 37

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/cffa168178e1
https://hg.mozilla.org/mozilla-central/rev/cad236ee0207