839907 - Android web dev should be able to use getUserMedia to capture video

Status: RESOLVED FIXED

(Core :: WebRTC: Audio/Video, defect)

Description

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: WIP make gUM video work with dummy surface, v1

Here's a patch that makes getUserMedia actually display video, with some limitations:

* you get one call to getUserMedia requesting video.  Then you have to kill and restart the browser for it to work again.  This is because of bugs in the initialization code here which shouldn't be too tough to fix.

* on some devices, the image appears rotated needed degrees

Technical deets: for the Android camera to work, one MUST supply a surface for a preview to be drawn on.  The standard way to do this if we don't want Android controlling doing that rendering for us (which we don't) is to create a dummy surface.  That has to happen on some other thread and generate callbacks here.

Fortunately, Fabrice already wrote code for the still camera stuff to do this in GeckoApp.java.  So we introduce a dependency on Gecko and "borrow" that dummy surface.  This is ok for now, but before uplift, we're probably going to want to generalize the GeckoApp.java code a bit so that it can generate an arbtirary number of these surfaces dynamically for use with multiple cameras (e.g. front and back) on a device.  And also so that we don't have ownership conflicts.

Comment 1

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: WIP make gUM video work using dummy surface, V2 (landed on alder)

Makes multiple invocations of gUM requesting video work without having to restart the browser.  This fixes the capture code to tear down and release resources correctly, which allows them to be re-used in subsequent allocations of the camera.

Comment 2

[Dan Mosedale (:dmosedale, :dmose)]

Landed on alder:

https://hg.mozilla.org/projects/alder/rev/0bd874aa0258

Next steps:

* rebase to trunk
* decide on and implement sharing model for dummy local surfaces
* request review

Comment 3

[Gian-Carlo Pascutto [:gcp]]

Attachment: Patch 1. Enable/disable the camera view when capture starts.

Rebased, also contains a build fix (my patch stack is getting big...).

Comment 4

[Brad Lassey [:blassey] (use needinfo?)]

Comment on attachment 731893 [details] [diff] [review]
Patch 1. Enable/disable the camera view when capture starts.

Review of attachment 731893 [details] [diff] [review]:
-----------------------------------------------------------------

r+ but this will have to be rebased to based on the r- on bug 835973

::: media/webrtc/trunk/webrtc/modules/video_capture/android/java/org/webrtc/videoengine/VideoCaptureAndroid.java
@@ +71,5 @@
>          captureAndroid.context = 0;
> +
> +        GeckoApp.mAppContext.cameraView.getHolder().
> +            removeCallback(captureAndroid);
> +        GeckoAppShell.getMainHandler().post(new Runnable() {

I think this needs to use the new ThreadUtils

@@ +77,5 @@
> +            public void run() {
> +                try {
> +                    GeckoApp.mAppContext.disableCameraView();
> +                } catch (Exception e) {
> +                    Log.e(TAG, 

nit, trailing space here

Comment 5

[Gian-Carlo Pascutto [:gcp]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/d1195963bc7c

Comment 6

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/d1195963bc7c

Comment 7

[Jason Smith [:jsmith]]

Not building WebRTC in Android by default yet, so can't verify.