1300018 - Support audio 5.1 on Windows(cubeb)

Status: RESOLVED FIXED

(Core :: Audio/Video: cubeb, task, P2)

Comment 1

[C.M.Chang[:chunmin]]

Attachment: [WIP] support audio 5.1 on Windows

TODO:
- Need to check channel mapping information before setting dwChannelMask. There are different masks for same channel counts(e.g., channels = 3 can be <L, R, C> or <L, R, LOW_FREQ>).
- If the channel counts is less than 6, maybe we should upmix it to simulated 5.1 surround sounds[0]

[0] Real-Time Conversion of Stereo Audio to 5.1 Channel Audio for Providing Realistic Sounds (http://www.sersc.org/journals/IJSIP/vol2_no4/7.pdf)

Comment 2

[C.M.Chang[:chunmin]]

Attachment: Notes for WAS API

Comment 3

[C.M.Chang[:chunmin]]

Attachment: [WIP] support audio 5.1 on Windows

change:
1. New a enum for channal_layout. 
2. New a member for channal_layout in cubeb_stream.
3. Use cubeb_stream.channal_layout to decide the dwChannelMask for WAS API

Comment 4

[C.M.Chang[:chunmin]]

Attachment: [WIP] support audio 5.1 on Windows

fix typo.

Comment 5

[C.M.Chang[:chunmin]]

Attachment: Notes for WAS API

Comment 6

[C.M.Chang[:chunmin]]

Attachment: Support audio 5.1 on Windows

Hi kinetik,
Could you give me some suggestions?

Comment 7

[C.M.Chang[:chunmin]]

Comment on attachment 8802406 [details] [review]
Support audio 5.1 on Windows

https://github.com/kinetiknz/cubeb/pull/171

Comment 8

[Matthew Gregan [:kinetik]]

Comment on attachment 8802406 [details] [review]
Support audio 5.1 on Windows

Feedback in the PR.

Comment 10

[C.M.Chang[:chunmin]]

Attachment: audio 5.1 ogg file for testing

Comment 11

[C.M.Chang[:chunmin]]

Attachment: [WIP] Gecko audio5.1 testing on windows

With this patch for Windows, I could successfully play the attachment 8803285 [details](ogg file) on all 5.1 channels.

Comment 12

[C.M.Chang[:chunmin]]

Attachment: [WIP] play audio 5.1 in gecko(windows)

Comment 13

[C.M.Chang[:chunmin]]

Attachment: [WIP] play audio 5.1 in gecko(windows)

Comment 14

[C.M.Chang[:chunmin]]

Comment on attachment 8802406 [details] [review]
Support audio 5.1 on Windows

Hi kinetik,
Thanks for the feedback. The concerns should have been addressed. Could you help to review the patch? The changes are left in git pull request. Thanks.

Comment 15

[Matthew Gregan [:kinetik]]

Comment on attachment 8802406 [details] [review]
Support audio 5.1 on Windows

I added feedback and questions in the PR.

Comment 16

[C.M.Chang[:chunmin]]

Attachment: [WIP] Multiple channel in SMPTE layout format

Comment 17

[C.M.Chang[:chunmin]]

Attachment: [WIP] Multiple channel in SMPTE layout format

I'll add downmix matrix for 5.1 in next patch.

Comment 18

[C.M.Chang[:chunmin]]

Attachment: [WIP] Multiple channel in SMPTE layout format

Changes:
- Add downmix from 3F2(-LFE) to 1F, 2F, 3F, 2F1, 3F1, 2F2 and their LFEs.
- Replace "layout_default" with "layout_unsupported". The cubeb user needs to set the layout explicitly.

TODO:
The downmix/upmix code should be separated into a independent module from WASAPI for following reasons:
- It can be reused for all backends. 
  The downmix/upmix is refered to some standards, so it should be same for each backend.
- It's easier for testing. 
  We can ignore the internal process of backends and just focus on the correctness of supported layout conversion.
- It's flexible to extend. 
  If we want to add some fancy features like stereo-to-5.1 simulation, then only this module needs to be modified, and it will be applied to all backends.

Comment 19

[C.M.Chang[:chunmin]]

Attachment: Part 1 : Add SMPTE layout to configure multiple channels

Comment 20

[C.M.Chang[:chunmin]]

Attachment: Part 2 : Separate downmix amd upmix into an independent module

Comment 21

[C.M.Chang[:chunmin]]

Attachment: Part 2 : Separate downmix amd upmix into an independent module

Forget to delete unnecessary code from cubeb_wasapi.cpp last time.

Comment 22

[C.M.Chang[:chunmin]]

Attachment: [WIP] Play audio 5.1 on Windows

changes:
- Convert channelMap(AudioConfig.ChannlLayout().Map()) to SMPTE layout and pass it to cubeb
- Use |defined(XP_WIN)| to enable multiple channel on Windows only.

Comment 23

[C.M.Chang[:chunmin]]

Attachment: Part 2 : Separate downmix amd upmix into an independent module

Move downmix matrix into .cpp instead of .h because there is no reason to expose it.

Comment 24

[C.M.Chang[:chunmin]]

Attachment: [WIP] Play audio 5.1 on Windows

Remove windows' invisible "Cr" characters.

Comment 25

[C.M.Chang[:chunmin]]

Comment on attachment 8802406 [details] [review]
Support audio 5.1 on Windows

Hi Matthew,
I was wondering if you could review this patch.
I add a mixing module in cubeb and there are three policies. The detail is described in [0].

[0] https://wiki.mozilla.org/Media/multichannel#Mixing_Policy

Comment 26

[Matthew Gregan [:kinetik]]

Comment on attachment 8802406 [details] [review]
Support audio 5.1 on Windows

Review comments are on the PR

Comment 27

[C.M.Chang[:chunmin]]

Comment on attachment 8802406 [details] [review]
Support audio 5.1 on Windows

Thanks for your help. The patch is revised.

Comment 28

[Matthew Gregan [:kinetik]]

Comment on attachment 8802406 [details] [review]
Support audio 5.1 on Windows

Merged in https://github.com/kinetiknz/cubeb/commit/a900d6e511c3286956c38ca60162790ca6e8e18d

Thanks!

Comment 29

[Danial Horton]

Is this reopenable?

I just reported bug 1368938