2012662 - Mutlichannel audio input capture device in Ubuntu

Status: UNCONFIRMED

(Core :: WebRTC: Audio/Video, defect, P3)

Description

[Roberto Aldà]

Attachment: JScode.js

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/144.0.0.0 Safari/537.36

Steps to reproduce:

I wrote a sample JS to capture audio from card Magewell Pro Capture HDMI 4K+ and hear it to multichannel audio output.

I've already tested my output devices playing an audio file in Firefox and all 8 channels are ok (tested with pulsemeter).

I've also verifyed that Magewell captures 8 channels correctly, i've used 'Pulsemeter' to monitor input audio volumes.

The problem is that i can't hear captured audio into all 8 channels of output device, i hear only first 2 channels.

My system is:
os:Ubuntu 24.04.1 LTS
bowser:Mozilla Firefox 142.0.1
capture card: Magewell Pro Capture HDMI 4K+
output audio card: virtual card created with a pactl script
js code: see attached file
log file:see attachments

Actual results:

I can't hear captured multichannel audio into all 8 channels of output device, i hear only first 2 channels.
The console log is:

Adding output device: VirtualMultichannelSink Audio/Sink sink id: IZMY0Dc9gR2R1Q03Fu+nJ6x+kTGLJmA8rj9TMqNn5Jg= testSample.js:33:17
Adding output device: HDA NVidia Digital Stereo (HDMI) id: UljPao0RAril4qaIZ6dTRi7QVQZFc5E840gC50EIf04= testSample.js:33:17
Adding output device: Built-in Audio Digital Stereo (IEC958) id: FqWCtrfrdwx6cG1W3fMQIvlbCSMQjOGZLQ3CojULKMw= testSample.js:33:17
Captured track:
MediaStreamTrack { kind: "audio", id: "{2ae2561b-1e2d-4cc6-a225-09ca48dd4565}", label: "01 Pro Capture HDMI 4K+ Pro", enabled: true, muted: false, onmute: null, onunmute: null, readyState: "live", onended: null }
testSample.js:94:17
Track settings:
Object { autoGainControl: false, channelCount: 8, deviceId: "h5q5pq8pI/B27w/qDTVvkdCT+cbRBt0+qdPDdgSp7Y8=", echoCancellation: false, groupId: "cOMq+mHYGxJgoFbJpyRu3bB4TIQ8BR9htab2AkhS0EI=", noiseSuppression: false }
testSample.js:95:17
Track capabilities:
Object { autoGainControl: (2) [\u2026], channelCount: {\u2026}, deviceId: "h5q5pq8pI/B27w/qDTVvkdCT+cbRBt0+qdPDdgSp7Y8=", echoCancellation: (2) [\u2026], groupId: "cOMq+mHYGxJgoFbJpyRu3bB4TIQ8BR9htab2AkhS0EI=", noiseSuppression: (2) [\u2026] }
testSample.js:96:17
outputDeviceId: IZMY0Dc9gR2R1Q03Fu+nJ6x+kTGLJmA8rj9TMqNn5Jg= testSample.js:97:17
AudioContext.destination.channelCount: 8 testSample.js:157:13
AudioContext.destination.channelCount: 8 testSample.js:168:13

Expected results:

I want to hear all captured 8 channels into audio output multichannel device.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Alastor Wu [:alwu]]

Karl, could you help triage this? I’m not sure whether this should be a WebRTC (getUserMedia) issue or a Web Audio issue. Thanks!

Comment 3

[Karl Tomlinson (back June 29 :karlt)]

Track capabilities:
Object { autoGainControl: (2) [\u2026], channelCount: {\u2026}, deviceId: "h5q5pq8pI/B27w/qDTVvkdCT+cbRBt0+qdPDdgSp7Y8=", echoCancellation: (2) [\u2026], groupId: "cOMq+mHYGxJgoFbJpyRu3bB4TIQ8BR9htab2AkhS0EI=", noiseSuppression: (2) [\u2026] }

What does the channelCount ellipsis expand to?

Does the Input Devices section of about:support show the expected number of channels?

(Tentatively picking WebRTC: Audio/Video, until we know more.)

Comment 4

[Roberto Aldà]

about:support shows "Channels 8" on "01 Pro Capture HDMI 4K+ Pro" under "Input devices"

Track capabilities:
{
"autoGainControl": [
true,
false
],
"channelCount": {
"max": 8,
"min": 1
},
"deviceId": "RkIO25GYd6BRR790u/dTQIzHcjtvwo6InI8Do+veWVs=",
"echoCancellation": [
true,
false
],
"groupId": "4MBwX6iWH4IjfyGQhl83jnFgpWbZu1W7+s0AisOzNmI=",
"noiseSuppression": [
true,
false
]
}

Comment 5

[Karl Tomlinson (back June 29 :karlt)]

Thanks for the extra info. I'm not seeing any evidence of the unexpected except that audio is heard in only two of the eight output channels.

The next thing I suggest is to use ChannelSplitterNode and/or ChannelMergerNode to switch known working output channels to output different input channels and/or switch different input channels into known working output channels.

That would help narrow down which side is losing the signal from additional channels.

Comment 6

[Karl Tomlinson (back June 29 :karlt)]

(In reply to Roberto Aldà from comment #0)

I've already tested my output devices playing an audio file in Firefox and all 8 channels are ok (tested with pulsemeter).

I see async function play8ChannelAudio(file) unused in the testcase.
Was that what you used to test the output device?

If that is what you used, and not an HTMLMediaElement, then the AudioContext output seems to be behaving as expected, and there is not much additional value in the experiments suggested comment 5.
The unverified components in the pipeline would be the MediaStreamTrack from getUserMedia() and MediaStreamAudioSourceNode/AudioNodeExternalInputTrack from Web Audio. The latter could be verified by connecting a MediaStreamAudioDestinationNode connected from an AudioBufferSourceNode like that in play8ChannelAudio().