914494 - getUserMedia application notification when a higher-priority user grabs the hardware

Status: NEW

(Core :: WebRTC: Audio/Video, defect, P5)

Description

[StevenLee[:slee]]

As discussed in https://bugzilla.mozilla.org/show_bug.cgi?id=904025#c18, we have a follow up bug for this issue.

Comment 1

[Randell Jesup [:jesup] (needinfo me)]

There's an issue when a higher-priority user (such as in incoming phone call) grabs the hardware when it's in-use by getUserMedia.  Temporarily at least we'll land bug 904025 which causes OnNavigation for all streams being captured, but that's not a great solution:

*) it kills video-only captures
*) it doesn't inform the application except very indirectly (onended on MediaStreamTracks, and the app will stop seeing data arrive)
   When we support calls on B2G, the app wants to know so it can do the appropriate thing
*) no option for the application to notice when the call is done and resume
   (Would this require a new permission grant?)

For example, a "background" app listening for voice commands, music to identify, or (video) gestures would end up cut off.

Some of these involve coordination with the Media Capture task force to capture the behavior in the spec.  The MediaCap folk have proposed removing the stream-level "onended", leaving just the track-level ones, to simplify the spec and implementation.  We should probably note to the list that an impact of this proposal would be: apps will need to ask for all the tracks and install onended funcs on all of them, and add a func to fire an application event/call when all tracks have ended.  that's a bunch of work for each app, and effectively boilerplate that could be subsumed once into the browsers.

Comment 2

[StevenLee[:slee]]

(In reply to Randell Jesup [:jesup] from comment #1)
> There's an issue when a higher-priority user (such as in incoming phone
> call) grabs the hardware when it's in-use by getUserMedia.  Temporarily at
> least we'll land bug 904025 which causes OnNavigation for all streams being
> captured, but that's not a great solution:
> *) it kills video-only captures
Agree, we should not kill all MediaStreams.

> *) it doesn't inform the application except very indirectly (onended on
> MediaStreamTracks, and the app will stop seeing data arrive)
>    When we support calls on B2G, the app wants to know so it can do the
> appropriate thing
> *) no option for the application to notice when the call is done and resume
>    (Would this require a new permission grant?)
I can try it and see if we could resume the MS smoothly.

If we want to support pause, does it mean that we need to add attributes/functions to MediaStream or we cannot know/control the state of the MS? Or we could do the resuming task internally. But we need to keep a list of MediaStreams that should be resumed.

> Some of these involve coordination with the Media Capture task force to
> capture the behavior in the spec.  The MediaCap folk have proposed removing
> the stream-level "onended", leaving just the track-level ones, to simplify
> the spec and implementation.  We should probably note to the list that an
> impact of this proposal would be: apps will need to ask for all the tracks
> and install onended funcs on all of them, and add a func to fire an
> application event/call when all tracks have ended.  that's a bunch of work
> for each app, and effectively boilerplate that could be subsumed once into
> the browsers.
Do you mean that we should have a centralized event to inform all apps that the current status of camera/mic is changed? When apps register this event, they can get informed and do what they want to do.

Comment 3

[Maire Reavy [:mreavy]]

We need a proposal, hopefully one we can make to the Media Capture TF

Comment 4

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Could we perhaps use muted [1] and onmute [2] for this?

"A MediaStreamTrack is muted when the source is temporarily unable to provide the track with data."

[1] http://w3c.github.io/mediacapture-main/getusermedia.html#track-muted
[2] http://w3c.github.io/mediacapture-main/getusermedia.html#widl-MediaStreamTrack-onmute

Comment 5

[Bulk Bug Changes for mreavy's org]

Mass change P4->P5 to align with new Mozilla triage process.