1242816 - Use applyConstraints facingMode to switch front/back camera on Android

Status: NEW

(Core :: WebRTC: Audio/Video, defect, P3)

Description

[Mathieu Hofman]

Once a webcam stream has been acquired on Android, it should be possible to switch between front and back camera by changing the "facingMode" constraint in an applyConstraints() call.

Currently it seems that the "facingMode" constraint only works during the initial gUM call for selection.

This would avoid interruptions of the stream since both cameras cannot be active at the same time (forcing the app to stop the previous stream first), and a permission prompt is normally shown to the user for each gUM call.

See https://github.com/w3c/mediacapture-main/issues/304#issuecomment-174213158 for some more background.

Comment 1

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Unfortunately, I believe it would violate the spec to have track.applyConstraints() switch to another device that's observably different in device enumeration.

To be compliant, this would have to be a new single virtual device abstracting both cameras. Lets call it "Both".

gUM may be fine, but web sites calling enumerateDevices would be affected. Would they see:

 A) three devices ("Front", "Back", and "Both")?
 B) One device ("Both")? or
 C) two devices ("Front (by default)" and "Back (by default)")?

Ironically, sites would probably want to translate this back into a list of two devices for their UX, so maybe C is best?

Furthermore, devinfo.getCapabilities() would have to show { ... facingMode: ["user", "environment"] } on this new thing, which may be fine, except in C, where one wouldn't be able to tell the two apart except by the label.

In the end, web sites would have to recognize this thing and know how it works, which might mean we'd want buy-in from other vendors on this anyway.

Comment 2

[Mathieu Hofman]

What I had in mind was B) a single "Both". For lack of actual experience with Firefox Android, and seeing examples with "facingMode", I thought that was already the case.

As a transition A) seems like an acceptable solution. There is already plenty of cases where 2 different entries in the enumeration are actually the same device, e.g. the "default" option in Chrome, or virtual webcam drivers on desktop.
Indeed developers would need to use getCapabilities() to understand this "Both" device is capable of switching.
As an app developer, I actually might show a completely different UI on mobile, instead of a "list", I'd simply show a "flip" button.

IMHO, C) is not the right approach for the same issues you raise.

I agree that getting buy-in from other vendors would be a good idea.
I just filed https://code.google.com/p/chromium/issues/detail?id=581401 with Chrome. Justin, do you have any opinion on the matter?

Comment 3

[Randell Jesup [:jesup] (needinfo me)]

Any action on that Chrome bug?

Comment 4

[Mathieu Hofman]

I haven't seen much traction on that Google bug. Maybe Justin has more info?

Comment 5

[Justin Uberti]

I looked into it and I agree with the B) proposal - this matches how Android and iOS do camera switching under the covers. Updated the chromium bug as such.

Comment 6

[Bulk Bug Changes for mreavy's org]

Mass change P2->P3 to align with new Mozilla triage process.