1567949 - Implement `AudioDestinationNode.channelCount` correctly

Status: RESOLVED FIXED

(Core :: Web Audio, defect, P3)

Description

[Paul Adenot (:padenot)]

This has the following parts:

• Defaults the channelCount to 2 regardless of the hardware. This is what the spec says, in fact.
• Map the channel count to well known channel topologies. It would be best if non-mapped channels was supported, but it does not seem to be the case. For Firefox, this means that 1 would be mono, 2 would be stereo, etc. For now, we always use the UNDEFINED layout, that is correct because it means independent discrete channels, but we've seen drivers that don't like it. This should fix bug 1563906.
• When outputChannel is set on a particular AudioDestinationNode, change the number of output channel the graph outputs to, and perform upmix on the other AudioContext, like we do for input.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug is a enhancement, but please change it back in case of error.

Comment 3

[Paul Adenot (:padenot)]

Attachment: Bug 1567949 - Default to 2 channels when creating an AudioContext. r?pehrsons

Comment 4

[Paul Adenot (:padenot)]

Attachment: Bug 1567949 - Don't change the channel count when it's not necessary. r?pehrsons

Comment 5

[Paul Adenot (:padenot)]

Attachment: Bug 1567949 - When the channel count change on an DestinationNode, change the AudioCallbackDriver channel count. r?pehrsons

Comment 6

[Paul Adenot (:padenot)]

Attachment: Bug 1567949 - Pass in a well known channel mapping based on the number of channels to cubeb in AudioCallbackDriver. r?pehrsons

Comment 7

[Paul Adenot (:padenot)]

Attachment: Bug 1567949 - Pass an output channel count to new AudioCallbackDriver. r?pehrsons

Comment 8

[Paul Adenot (:padenot)]

Attachment: Bug 1567949 - Start testing AudioDestinationNode in WPT. r?pehrsons

Comment 9

[Paul Adenot (:padenot)]

Attachment: Bug 1567949 - Always write the number of audio output channels the AudioCallbackDriver expects. r?pehrsons

Comment 10

[Paul Adenot (:padenot)]

Attachment: Bug 1567949 - Don't use a global for the key in AudioDestinationNode.cpp, nullptr is fine. r?pehrsons

This was initialized to 0 anyways.

Comment 11

[Paul Adenot (:padenot)]

Attachment: Bug 1567949 - Expose the number of input channels of an AudioNode, on the audio rendering thread side. r?pehrsons

This is in line with the meaning of channelCount in the Web Audio API
specification.

Comment 12

[Paul Adenot (:padenot)]

Attachment: Bug 1567949 - Clarify the semantics of AudioSegment::ChannelCount, and rename it. r?pehrsons

It is now the channel count of the underlying chunks or 0 of there are no
segment (but this should be avoided, so the non-fatal assert is kept).

An assert is added to make sure there are no segments with varying number of
chunks.

Comment 13

[Paul Adenot (:padenot)]

Attachment: Bug 1567949 - Don't create/destroy audio outputs during processing, use graph messages to register/unregisters audio outputs instead. r?pehrsons

This is the big part:

• Methods are added to register/unregister an audio output, and to unregister
  all audio output for a stream. The previous way of determining what track to
  output is removed.
• Information about audio output stream now lives on the graph itself, and not
  at multiple locations.
• PlayAudio is refactored slightly but largely untouched
• AudioOutputChannelCount now works in a way that is similar to its input
  counterpart: it's the maximum of the channel count that needs to be output.

Comment 14

[Paul Adenot (:padenot)]

Attachment: Bug 1567949 - Remove the output channel count from the MTG. r?pehrsons

This channel count solely depends on what the graph is doing, it's not static
anymore.

Comment 15

[Paul Adenot (:padenot)]

Attachment: Bug 1567949 - Prevent using an AudioCallbackDriver when not necessary. r?pehrsons

Not stricly needed but easy to do now.

Comment 16

[Paul Adenot (:padenot)]

Attachment: Bug 1567949 - Add a mochitest of a MediaElement playing a MediaSource with a varying number of channels, routed to an AudioContext. r?pehrsons

Comment 17

[Paul Adenot (:padenot)]

Attachment: Bug 1567949 - Fix some includes. r?pehrsons

This is because the resampler is used from a different file and also because
there was a stray #include "AudioStream.h" that was bringing in a bunch of
stuff.

Comment 18

[Paul Adenot (:padenot)]

Attachment: Bug 1567949 - Move resampler setup one layer down to be able to set the channel count per chunk. r?pehrsons

It is technically possible to have different channel count per chunk, so we
should reinitialize the resampler when that happens. This is a bit awkward
because the rates are on the track, but the data is in the chunks, and the track
cannot access the chunks, so we have to plumb the resampler state down.

Comment 19

[Paul Adenot (:padenot)]

Attachment: Bug 1567949 - When the audio output channel count of an output changes, try switching to a new driver with the appropriate number of channels. r?pehrsons

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5eb0391b5e39
Default to 2 channels when creating an AudioContext. r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/1b8eab609e03
Don't change the channel count when it's not necessary.  r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/dcbf81da8664
Pass an output channel count to new AudioCallbackDriver.  r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/b6f5857fef14
Don't use a global for the key in AudioDestinationNode.cpp, nullptr is fine.  r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/9155420178fc
Expose the number of input channels of an AudioNode, on the audio rendering thread side. r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/52fdc0e5fea1
Clarify the semantics of AudioSegment::ChannelCount, and rename it. r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/95dc6464704f
Don't create/destroy audio outputs during processing, use graph messages to register/unregisters audio outputs instead. r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/fd7b69523a09
Prevent using an AudioCallbackDriver when not necessary. r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/3602937f5807
Pass in a well known channel mapping based on the number of channels to cubeb in AudioCallbackDriver. r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/126c112142c7
Start testing AudioDestinationNode in WPT. r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/566a2c6f21db
Add a mochitest of a MediaElement playing a MediaSource with a varying number of channels, routed to an AudioContext. r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/c68d34ef1005
Fix some includes. r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/1bed49fa4f66
Move resampler setup one layer down to be able to set the channel count per chunk. r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/d4d413229820
When the audio output channel count of an output changes, try switching to a new driver with the appropriate number of channels. r=pehrsons

Comment 21

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/20339 for changes under testing/web-platform/tests

Comment 22

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream web-platform-tests status checks passed, PR will merge once commit reaches central.

Comment 23

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/5eb0391b5e39
https://hg.mozilla.org/mozilla-central/rev/1b8eab609e03
https://hg.mozilla.org/mozilla-central/rev/dcbf81da8664
https://hg.mozilla.org/mozilla-central/rev/b6f5857fef14
https://hg.mozilla.org/mozilla-central/rev/9155420178fc
https://hg.mozilla.org/mozilla-central/rev/52fdc0e5fea1
https://hg.mozilla.org/mozilla-central/rev/95dc6464704f
https://hg.mozilla.org/mozilla-central/rev/fd7b69523a09
https://hg.mozilla.org/mozilla-central/rev/3602937f5807
https://hg.mozilla.org/mozilla-central/rev/126c112142c7
https://hg.mozilla.org/mozilla-central/rev/566a2c6f21db
https://hg.mozilla.org/mozilla-central/rev/c68d34ef1005
https://hg.mozilla.org/mozilla-central/rev/1bed49fa4f66
https://hg.mozilla.org/mozilla-central/rev/d4d413229820

Comment 24

[BugBot [:suhaib / :marco/ :calixte]]

Since the status are different for nightly and release, what's the status for beta?
For more information, please visit auto_nag documentation.

Comment 25

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot