910249 - [meta] Finish implementation of MediaStream.webidl and MediaStreamTrack.webidl

Status: NEW

(Core :: WebRTC: Audio/Video, task, P3)

Description

[Adam Roach [:abr]]

Currently, our version of MediaStreamTrack.webidl is out of sync with the most recent editors' draft (in particular, it is missing the "MediaSourceStates states ();" method). It also has a preponderance of its events and methods commented out.

I'm filing this as a monolithic bug rather than giving each attribute/method/event its own bug, based on roc's evaluation that this should be a relatively small task. If that ends up not being the case, we may want to split this up into smaller chunks.

One specific point of pain that is coming up from the missing features is that scripts have no way to determine whether the camera selected by a user is front ("user") facing or rear ("environment") facing. Since some scripts choose to mirror a self-view, it is important to to be able to make this distinction.

Comment 1

[Adam Roach [:abr]]

I'll also note that the absence of clone() -- which should be present on both MediaStream and MediaStreamTrack -- is particularly painful. It would be nice to be able to, say, disable the video track being sent to a remote party while continuing to display a viable self-view. Without cloning, the only way to do this is to ask for the camera twice, which is untenable from a user experience perspective (and might have very surprising results if the user picks two different cameras for the two dialogs).

Comment 3

[Maire Reavy [:mreavy]]

If we think cloning is critical for applications, we should open a separate bug for that.

Jan-Ivar -- can you estimate the effort for this bug and put a p= in the whiteboard?

Comment 4

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

This is a wild guess. Cloning and camera-sharing sounds a bit like an open-ended problem, something best fine-tuned over time perhaps?

If we just want the webidl up and spinning with a small subset of features then that's a p=1.

Comment 5

[damon.oehlman]

I agree, I think the ability to effectively clone a track is pretty important for WebRTC media centric apps.  I think the following illustrates an instance where you would go down this route (has been tested as working this way in Chrome):

- Capture MediaStream (Audio + Video)
- Render MediaStream to "local" video display
- Create a new MediaStream, cloning each of the MediaStreamTracks within the originally captured stream.
- Use this new stream (with cloned tracks) to send over remote peer connections.

Using this approach gives you the advantage of being able to toggle the enabled states of the the cloned tracks without affecting the local rendering.  Taking it a step further, as an application developer you might take the approach of creating MediaStreams for each of the peers you wish to communicate with, thus being able to selectively disable tracks for targeted participants.

Comment 6

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

This bug includes implementing the ConstrainablePattern which must arbitrate camera access between multiple tracks with potentially competing constraints (e.g. users in different tabs and windows), including firing overconstained events on the MediaStreamTracks which constraints can no longer be met due to arbitration or even runtime events like users rotating their phone (affects width/height/aspectRatio).

Comment 7

[Randell Jesup [:jesup] (needinfo me)]

Let's target this for 37 or 38; there will be some backend work to do by me as well to support it.  Also, we may be able to get part of this sooner than the entire set covered by this bug.

Comment 10

[Andreas Pehrson [:pehrsons]]

I just turned this into a meta bug. I think it'll be easier to get things implemented and landed this way.

Comment 11

[rajkumaradass]

I'm waiting for the mediastream 'clone' functionality, as it's blocking controlling local streams in a multiple call scenario. Appreciate if you could provide a rough estimate of the targeted milestone for this?

thanks

Comment 12

[rajkumaradass]

Hi,

Could anyone please let me know if there's a rough estimate for this 'clone' functionality as we are relying on this.

thanks

Comment 13

[Maire Reavy [:mreavy]]

(In reply to rajkumaradass from comment #12)
> Hi,
> 
> Could anyone please let me know if there's a rough estimate for this 'clone'
> functionality as we are relying on this.
> 
> thanks

You want to track Bug 1208371.  The good news is cloning is basically ready to land.  Because there is so much code changing, we'll likely land the patches on that bug early next week (after uplift) so they get maximum bake time in Fx47 (rather than right before Fx46 goes to Dev Edition).

Comment 14

[Bulk Bug Changes for mreavy's org]

Mass change P2->P3 to align with new Mozilla triage process.