901616 - Non-interactive getUserMedia device selection support (persistence)

Status: RESOLVED DUPLICATE of bug 1037389

(Core :: WebRTC: Audio/Video, defect)

Description

[Randell Jesup [:jesup] (needinfo me)]

The Media Capture spec includes support for non-interactive device selection (and also the ability to change defaults to be "the same device I used last time")  We should add support for it to avoid a) support applications/sites given persistent permissions, and b) stop users from having to constantly override the defaults.

Note that the UI needs to learn how to change default selection (bug TBD)

Comment 1

[Patrick Quinn-Graham [:pqg]]

Would this bug also cover an implementation of navigator.getMediaDevices() ?

Comment 2

[Randell Jesup [:jesup] (needinfo me)]

Covered in other bugs (by jib)

Comment 4

[Mathieu Hofman]

The bug this bug is marked as a duplicate of only covers the "requesting a specific device" part.
Is there another bug covering the "non-interactive" part. I would like to be able to request another stream for the same (or another device) without a user prompt if I already have access to a device of the same input type.

Comment 5

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Hi Mathieu, the permission prompt is orthogonal to device selection. Firefox will prompt the user each time you call getUserMedia, unless the user has chosen "Always Allow" (https only) instead of just "Allow" for your site.

If you've already been granted a stream, there are a couple of things you can do to it:

1. You can alter it, using MediaStreamTrack.applyConstraints(constraints) (Bug 912342).
2. You can clone it, using MediaStreamTrack.clone() (once Bug 1208371 is fixed).

I hope that answers your question.

Comment 6

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Additionally, use the 'exact' keyword to make the constraint mandatory:

  { video: { deviceId: { exact: yourId } } } // this device or fail (no user-override)

Comment 7

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

(it wont remove a prompt but it'll remove choices.)

Comment 8

[Mathieu Hofman]

Thanks for the update.

The last call draft suggests the following: "When permission is not stored, permission should last only until such time as all MediaStreamTracks sourced from that device have been stopped." 
I read this as even with non persistent permissions, a gUM call for an active source should be granted without interaction from the user. I agree the only use case for this would be to get a stream with different constraints, which is covered by applyConstraints and clone.

Now while I agree user-agents are free to implement permissions persistence however they want, if at all, figuring out a non-interactive model for device switching could greatly improve the user experience.
Right now the enumerateDevices() API is pretty much worthless for a "device selection" feature since the user would be presented with a device selection prompt from both the application and from the browser when switching devices.

What I would like to see is a permission model based more on the type of device being shared, to allow the application to freely switch between devices of that type without a new browser permission prompt.
Now I realize this might be tricky to implement in the browser prompt, especially since the specs says: "It is a User Agent choice whether it offers functionality to store permission to each device separately, all devices of a given class, or all devices; the choice needs to be apparent to the user, and permission must have been granted for the entire set whose permission is being stored"

Maybe we need a virtual "all" deviceId for the application to request access to all devices of a given type? Or maybe the prompt's UI can present a different question based on the content of the gUM constraints, e.g. if the deviceId is specified (but maybe not constrained to "exact"), the browser assumes that the apps implements its own device selection and ask the user for a blanket permission for all devices of that type by default, with the ability for the user to override the choice to only share this specific device?

Is there any bug already tracking the permission prompts, or should I open a new one with this request?

Comment 9

[Mathieu Hofman]

Also such a "all devices" permission is orthogonal to persistence. If the user doesn't want to remember the choice, the permission is only granted while a device of that class is active on the page. That does still allow for non-interactive device switching.

Comment 10

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Attachment: WebRTC doorhangers on Android, FYI

(In reply to mathieu.hofman from comment #8)

Thanks, I think these are great points! Please file new bugs to organize discussion [1].

I agree that re-asking permission when calling getUserMedia to clone, is superfluous.
I agree that "allow all cameras for this session only" represents a hole, but am unsure about trade-off.
I agree that in-content device selection is un-smooth, but there are multiple reasons / complications:

 1. Our permission-prompt is overly complicated in ways that are solvable (Bug 1004392).
 2. There's an inherent usability hit in limiting permission to single devices for the drive-by web.
 3. Permission models are solved differently on desktop vs mobile.
 4. Most users have one camera on desktop, and two on mobile.
 5. Other browsers split things differently (limits reliance on web-devs).

> enumerateDevices() API is pretty much worthless for a "device selection" feature since
> the user would be presented with a device selection prompt from both the application
> and from the browser when switching devices.

That's your fault for not using the 'exact' keyword as shown in comment 6, and our fault for making a "permission prompt" look like a "device selection" prompt, even when there is no selection (OK mostly our fault). Firefox for Android does slightly better (though leaves out information, see attached screenshot).

There may be UX redesign on-going, and I've heard of maybe moving device selection entirely to the "in-call" doorhanger, but I don't know bug numbers.

Lastly, the word "non-interactive" conflates "selection" and "permission". Lets avoid it. E.g. in-content device selection works just fine once the user has granted "Always Allow".

> Maybe we need a virtual "all" deviceId to request access to all devices of a given type?

A useful straw man, but makes me think Firefox needs to solve this without web-devs coding for it.

[1] https://bugzilla.mozilla.org/enter_bug.cgi?product=Firefox&component=Device%20Permissions

Comment 11

[Mathieu Hofman]

Thanks for the detailed answer and the links to other UI bugs.

I just filed Bug 1212996 detailing more clearly my issue with the current permission flow and suggesting a solution.