Android web dev should be able to use getUserMedia to capture video

RESOLVED FIXED in mozilla23

Status

()

defect
RESOLVED FIXED
7 years ago
6 years ago

People

(Reporter: dmose, Assigned: dmose)

Tracking

(Blocks 1 bug)

Trunk
mozilla23
ARM
Android
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [android-trunk-needed][blocking-gum-][android-gum+][qa-])

Attachments

(1 attachment, 2 obsolete attachments)

Here's a patch that makes getUserMedia actually display video, with some limitations:

* you get one call to getUserMedia requesting video.  Then you have to kill and restart the browser for it to work again.  This is because of bugs in the initialization code here which shouldn't be too tough to fix.

* on some devices, the image appears rotated needed degrees

Technical deets: for the Android camera to work, one MUST supply a surface for a preview to be drawn on.  The standard way to do this if we don't want Android controlling doing that rendering for us (which we don't) is to create a dummy surface.  That has to happen on some other thread and generate callbacks here.

Fortunately, Fabrice already wrote code for the still camera stuff to do this in GeckoApp.java.  So we introduce a dependency on Gecko and "borrow" that dummy surface.  This is ok for now, but before uplift, we're probably going to want to generalize the GeckoApp.java code a bit so that it can generate an arbtirary number of these surfaces dynamically for use with multiple cameras (e.g. front and back) on a device.  And also so that we don't have ownership conflicts.
Makes multiple invocations of gUM requesting video work without having to restart the browser.  This fixes the capture code to tear down and release resources correctly, which allows them to be re-used in subsequent allocations of the camera.
Attachment #712273 - Attachment is obsolete: true
Landed on alder:

https://hg.mozilla.org/projects/alder/rev/0bd874aa0258

Next steps:

* rebase to trunk
* decide on and implement sharing model for dummy local surfaces
* request review
Whiteboard: [android-trunk-needed]
Attachment #712278 - Attachment description: WIP make gUM video work using dummy surface, V2 → WIP make gUM video work using dummy surface, V2 (landed on alder)
Blocks: 840131
Blocks: 840244
Whiteboard: [android-trunk-needed] → [android-trunk-needed][blocking-gum-]
Whiteboard: [android-trunk-needed][blocking-gum-] → [android-trunk-needed][blocking-gum-][android-gum+]
Rebased, also contains a build fix (my patch stack is getting big...).
Attachment #712278 - Attachment is obsolete: true
Attachment #731893 - Flags: review?(blassey.bugs)
Comment on attachment 731893 [details] [diff] [review]
Patch 1. Enable/disable the camera view when capture starts.

Review of attachment 731893 [details] [diff] [review]:
-----------------------------------------------------------------

r+ but this will have to be rebased to based on the r- on bug 835973

::: media/webrtc/trunk/webrtc/modules/video_capture/android/java/org/webrtc/videoengine/VideoCaptureAndroid.java
@@ +71,5 @@
>          captureAndroid.context = 0;
> +
> +        GeckoApp.mAppContext.cameraView.getHolder().
> +            removeCallback(captureAndroid);
> +        GeckoAppShell.getMainHandler().post(new Runnable() {

I think this needs to use the new ThreadUtils

@@ +77,5 @@
> +            public void run() {
> +                try {
> +                    GeckoApp.mAppContext.disableCameraView();
> +                } catch (Exception e) {
> +                    Log.e(TAG, 

nit, trailing space here
Attachment #731893 - Flags: review?(blassey.bugs) → review+
https://hg.mozilla.org/mozilla-central/rev/d1195963bc7c
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla23
Keywords: verifyme
Not building WebRTC in Android by default yet, so can't verify.
Keywords: verifyme
Whiteboard: [android-trunk-needed][blocking-gum-][android-gum+] → [android-trunk-needed][blocking-gum-][android-gum+][qa-]
You need to log in before you can comment on or make changes to this bug.