1221574 - Write a full-duplex Android OpenSL ES cubeb backend

Status: RESOLVED FIXED

(Core :: Audio/Video: cubeb, defect, P1)

Comment 1

[Maire Reavy [:mreavy]]

Alex is going to take this; our goal is to get this landed before the London work week (in June).

Comment 2

[Alex Chronopoulos [:achronop]]

Patch is up for review PR: https://github.com/kinetiknz/cubeb/pull/114

Comment 3

[Alex Chronopoulos [:achronop]]

Try run with pref off:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=722486f12441
and the updated debug build
https://treeherder.mozilla.org/#/jobs?repo=try&revision=60257532a844

Comment 4

[Alex Chronopoulos [:achronop]]

New run with cubeb and life cycle logs:

Full duplex pref set to true:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=a766aeb6b65c

Full duplex pref set to false:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=ca29617838132b9fe6ede15a4792fc2927c7fe20

Comment 5

[Alex Chronopoulos [:achronop]]

New try run with the assert issue loved and some logs activated:
Full duplex pref off:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=2f35bc1d4e72
Full duplex pref on:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=e036aba33867

Comment 6

[Alex Chronopoulos [:achronop]]

New run with less logging and one crash fix:
Full duplex off:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=a13fae117867
Full duplex on:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=e76495725513

Full duplex on + C2 crashtest fix:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=7bd946c6eaf9

Comment 7

[Alex Chronopoulos [:achronop]]

Degub run with mda2 problem fixed:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=746d5a694b48

Comment 8

[Alex Chronopoulos [:achronop]]

Full run with full duplex pref off:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c553bf42ded8

Comment 9

[Alex Chronopoulos [:achronop]]

Full run with full duplex pref on:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=287aa9cb0078

Comment 10

[Alex Chronopoulos [:achronop]]

Latest try:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=cab4bbde79b65b872f0181b2076d49b3b8df231d
Latest autophone try:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=ae10057481627d78a7b47483adfc27768c985d79

Comment 11

[Alex Chronopoulos [:achronop]]

Attachment: 001-gecko-change-update-script

Update the update.sh script

Comment 12

[Alex Chronopoulos [:achronop]]

Attachment: 002-gecko-build-errors

Fix build errors found with the new backend

Comment 13

[Alex Chronopoulos [:achronop]]

Attachment: 003-gecko-not-found-error

In case of Android there is not device enumeration available so it is set hardcoded to return the one mic.

Comment 14

[Alex Chronopoulos [:achronop]]

Attachment: 004-gecko-crash-error

Due to lack of device enumeration the assert changes no to expect anything in case of Android.

Comment 15

[Alex Chronopoulos [:achronop]]

Attachment: 005-gecko-msg-warnings-as-error

Update lifecylce logs in respect to android log macros.

Comment 16

[Alex Chronopoulos [:achronop]]

Attachment: 006-gecko-skip-addTrackRemoveTrack-test

Disable one test that fails sporadically in android. No failure in autophone.

Comment 17

[Randell Jesup [:jesup] (needinfo me)]

Comment on attachment 8810937 [details] [diff] [review]
004-gecko-crash-error

Review of attachment 8810937 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/media/webrtc/MediaEngineWebRTC.h
@@ +268,5 @@
>  
>    void StartRecording(SourceMediaStream *aStream, AudioDataListener *aListener)
>    {
> +#ifdef MOZ_WIDGET_ANDROID
> +    // OpenSL ES does not support enumerate devices.

"enumerating"

Comment 18

[Alex Chronopoulos [:achronop]]

(In reply to Randell Jesup [:jesup] from comment #17)
> Comment on attachment 8810937 [details] [diff] [review]
> 004-gecko-crash-error
> 
> Review of attachment 8810937 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: dom/media/webrtc/MediaEngineWebRTC.h
> @@ +268,5 @@
> >  
> >    void StartRecording(SourceMediaStream *aStream, AudioDataListener *aListener)
> >    {
> > +#ifdef MOZ_WIDGET_ANDROID
> > +    // OpenSL ES does not support enumerate devices.
> 
> "enumerating"

Patch has been updated

Comment 19

[Alex Chronopoulos [:achronop]]

Bug 1318619 update the cubeb upstream up to the patch before the OpenSL ES full duplex implementation.

Comment 20

[Alex Chronopoulos [:achronop]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/3765376057ce889747ddf549bcc248eb3086b624
Bug 1221574 - [01] Add queue array in update.sh script. r=padenot

https://hg.mozilla.org/integration/mozilla-inbound/rev/5a104f59c9f3c946276b448330eb9a839144e7ba
Bug 1221574 - [02] Fix build erros with the new backend. r=jesup

https://hg.mozilla.org/integration/mozilla-inbound/rev/bd59a9fb070a85ba8d03357bb3b516d38ee462db
Bug 1221574 - [03] Hardcode expected mic device on Android. r=jesup

https://hg.mozilla.org/integration/mozilla-inbound/rev/d0269eb23e0c23e08b95158c4c125c82a64dfaa0
Bug 1221574 - [04] Change assert of expected devices in case of Android. r=jesup

https://hg.mozilla.org/integration/mozilla-inbound/rev/14af435edc2d399d2834a32d43b0d7d4ffec9782
Bug 1221574 - [05] Fix MediaStreamGraph lifecycle logs in case of Android. r=jesup

https://hg.mozilla.org/integration/mozilla-inbound/rev/36297c58723f978d1924bd7ad74dc25ad34feb4d
Bug 1221574 - [06] Disable one media test in case of Android. r=jesup

https://hg.mozilla.org/integration/mozilla-inbound/rev/0d3f2c421fee39c99140a6c14075814a35476ad6
Bug 1221574 - [07] Update cubeb to b6315bb including full duplex for OpenSL ES. r=kinetik, r=padenot

Comment 21

[Alex Chronopoulos [:achronop]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/a40cac8610de8543bf21eecaa91dacf3ed1b73e7
Bug 1221574 - [08] Add new file cubeb_array_queue.h. r=bustage-fix on a CLOSED TREE

Comment 22

[Matthew Gregan [:kinetik]]

(In reply to Alex Chronopoulos [:achronop] from comment #21)
> https://hg.mozilla.org/integration/mozilla-inbound/rev/
> a40cac8610de8543bf21eecaa91dacf3ed1b73e7
> Bug 1221574 - [08] Add new file cubeb_array_queue.h. r=bustage-fix on a
> CLOSED TREE

update.sh needs to be updated to copy this across during import, too.

Comment 23

[Alex Chronopoulos [:achronop]]

(In reply to Matthew Gregan [:kinetik] from comment #22)
> update.sh needs to be updated to copy this across during import, too.
1st patch does that:
https://hg.mozilla.org/integration/mozilla-inbound/rev/3765376057ce889747ddf549bcc248eb3086b624
Bug 1221574 - [01] Add queue array in update.sh script. r=padenot

Comment 24

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/3765376057ce
https://hg.mozilla.org/mozilla-central/rev/5a104f59c9f3
https://hg.mozilla.org/mozilla-central/rev/bd59a9fb070a
https://hg.mozilla.org/mozilla-central/rev/d0269eb23e0c
https://hg.mozilla.org/mozilla-central/rev/14af435edc2d
https://hg.mozilla.org/mozilla-central/rev/36297c58723f
https://hg.mozilla.org/mozilla-central/rev/0d3f2c421fee
https://hg.mozilla.org/mozilla-central/rev/a40cac8610de

Comment 25

[Oana Horvath [:ohorvath - out of office]]

Hi,
I will take this feature as a QA. Here is the Test plan based on which the testing is made: https://wiki.mozilla.org/QA/Fennec/Full_Duplex_testing