Closed Bug 1318628 Opened 8 years ago Closed 8 years ago

Handle downmix/upmix in cubeb

Categories

(Core :: Audio/Video: cubeb, task, P2)

task

Tracking

()

RESOLVED FIXED

People

(Reporter: chunmin, Assigned: chunmin)

References

(Blocks 1 open bug)

Details

Attachments

(1 file, 1 obsolete file)

The AudioConverter processes the downmix/upmix before passing audio data to cubeb because cubeb doesn't support multiple channels. Once multiple channel support is implemented, the downmix/upmix should be moved into cubeb. The backends without multiple channel support should downmix every thing to stereo or mono like before. There are many possible layout combinations for downmix and upmix, from m channels to n channels, where m != n. The stereo, 5.1, and 7.1 are most common channel layout, so we should focus on m,n = 2, 6, 8. Table 2 in ITU-R BS.775-3[1] provides a downmix coefficients matrix to convert from SMPTE 3F2 to 1F, 2F, 3F, 2F1, 3F1 and 2F2. Example: --------------------------------------------- The 3F2's data is { L, R, C, LS, RS } and the downmix coefficients for stereo is: - L: 1, 0, 1/sqrt(2), 1/sqrt(2), 0 - R: 0, 1, 1/sqrt(2), 1/sqrt(2), 0 so the downmixed stereo is - L: L*1 + R*0 + C*(1/sqrt(2)) + LS*(1/sqrt(2)) + RS*0 = L + (C + LS)/sqrt(2) - R: L*0 + R*1 + C*(1/sqrt(2)) + LS*(1/sqrt(2)) + RS*0 = R + (C + RS)/sqrt(2) We could expand this matrix to convert from 3F2-LFE to 1F, 2F, 3F, 2F1, 3F1, 2F2 and their LFEs. If the input layout is 3F2, we just need to append 0 for LFE. By the expanded matrix, we could get a more general matrix that allows to convert more layouts. Example: --------------------------------------------- The 3F2's data is { L, R, C, 0(LFE), LS, RS } and the downmix coefficients for stereo is: - L: 1, 0, 1/sqrt(2), 0, 1/sqrt(2), 0 - R: 0, 1, 1/sqrt(2), 0, 1/sqrt(2), 0 - LFE: 0, 0, 0, 1, 0, 0 so the downmixed stereo is - L: L*1 + R*0 + C*(1/sqrt(2)) + 0*0 + LS*(1/sqrt(2)) + RS*0 = L + (C + LS)/sqrt(2) - R: L*0 + R*1 + C*(1/sqrt(2)) + 0*0 + LS*(1/sqrt(2)) + RS*0 = R + (C + RS)/sqrt(2) - LFE: L*0 + R*0 + C*0 + 0*0 + LS*0 + RS*0 The example code is here[2] [0] http://searchfox.org/mozilla-central/rev/62db1c9021cfbde9fa5e6e9601de16c21f4c7ce4/dom/media/AudioConverter.cpp#141 [1] https://www.itu.int/dms_pubrec/itu-r/rec/bs/R-REC-BS.775-3-201208-I!!PDF-E.pdf [2] https://gist.github.com/ChunMinChang/0de31bd5b3213409544ef89db91b696b
Rank: 25
Priority: -- → P2
Assignee: nobody → cchang
Blocks: 1073786
I think we could provide three mechanisms for mixing: - Specific conversion - Mapping data by channel name - Bypassing data by channel index(fallback plan) Each time when we try to upmix or downmix, we need to try converting data with the above order. That is, we try specific conversion first. If it works, then the job is done. Otherwise, we next try mixing by mapping the channel. If it still doesn't work, then we try mixing by bypassing the channel data. The final mechanism should be our fallback plan and it should always work. Specific conversion ------------------------------------------ Some conversion has its own definition, so we need to implement this. For example, The table 2 in ITU-R BS.775-3 defines the downmix equations from 3F2 to 1F, 2F, 3F, 2F1, 3F1 and 2F2. Mapping data by channel name ------------------------------------------ In most cases, the input and output data can be mapped by its layout setting. For example, if we try downmixing from 3F(L, R, C) to Stereo(L, R), we only need to pass the first two input channel data to output. Bypassing data by channel index(fallback plan) ------------------------------------------ There is some case the above mechanisms don't cover. The downmix from stereo(L, R) to mono(M) is an example. There is no spec and there is no matched channel for this conversion. Especially, WASAPI can support some unmatched speaker settings like 6 channels with stereo layout(stereo should only has 2 channels). In such case, we don't know the mixing policy should follow the layout or channel number. The simplest plan is to follow its channel numbers. If the input has 2 channels and output has 1 channel, then we just need to pass the first data to the output. We just need to pass the channel data by channel index. An alternative way is to define some matrices to compress/expand the audio data. However, the combination is not a small number. Define some matrices for partial common cases like 8~3 to 2 channels may be feasible.
Attached file [WIP] Mixing module for cubeb (obsolete) —
Attachment #8816071 - Attachment is obsolete: true
The some backend may already have mixing support. It's possible to combine the existing mixing support with our own module. See bug 1300023 comment 4.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Type: defect → task
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: