Closed Bug 977864 Opened 11 years ago Closed 11 years ago

Create new media streams for unannounced SSRCs received on a peer connection

Categories

(Core :: WebRTC, defect)

Product:
Core
Component:
WebRTC
Version:
27 Branch
Platform:
x86
macOS
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED WONTFIX

People

(Reporter: emcho, Unassigned)

References

(
URL
)

Details

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:27.0) Gecko/20100101 Firefox/27.0 (Beta/Release) Build ID: 20140212131424 Steps to reproduce: Established a PeerConnection and sent one video stream (ssrc=1) to Firefox. Then sent a second video stream within the same peer connection (ssrc=2) to Firefox. Actual results: Video became messed up showing alternating messy segments of the two streams. Expected results: Some day, there will be support for UNIFIED signalling and (if the second SSRC would have been signalled that way) it would have been handled accordingly. UNIFIED however is still not around the corner as suggested by: https://bugzilla.mozilla.org/show_bug.cgi?id=784517 Still, even with UNIFIED, an unannounced SSRC should just trigger a new "anonymous" MediaStream. Implementing this should be much, much simpler and it should provide for basic multi-stream support that would probably be enough to get SFU conferencing applications off the ground.
Forgot to mention. This all originated on #media: mreavy: emcho: I totally understand. I wish we had what you wanted implemented today. emcho: mreavy: there are actually other possibilities too. accepting unannounced SSRCs within a peer connection, even if the signaling is not there, should provide most of what we need to make jitmeet.org work with firefox emcho: (jitmeet.org being the project behind meet.jit.si) mreavy: emcho: do me a favor? File a bug about this: https://bugzilla.mozilla.org/enter_bug.cgi?product=Core&component=WebRTC and cc me (:mreavy) in the bug. We can discuss in the bug with other team members and see what's possible. emcho: mreavy: fair enough. will do.
FWIW the code for that lives at http://jitmeet.org
Alternately, support for ORTC would also provide a solution to the problem.
(In reply to emcho from comment #3) > Alternately, support for ORTC would also provide a solution to the problem. For delete any mistake, ORTC is for "Open Real-time Connectivity" ??? http://www.realtime.co/developers/ortc
No, actually ORTC stands for the Object Real-time Communication API that's currently being developed in a W3C community group: http://www.w3.org/community/ortc/ It is probably not unreasonable to see this as the WebRTC 2.0 API
(In reply to emcho from comment #5) > No, actually ORTC stands for the Object Real-time Communication API that's > currently being developed in a W3C community group: > > http://www.w3.org/community/ortc/ > > It is probably not unreasonable to see this as the WebRTC 2.0 API This statement seems premature at best.
(In reply to emcho from comment #0) > User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:27.0) > Gecko/20100101 Firefox/27.0 (Beta/Release) > Build ID: 20140212131424 > > Steps to reproduce: > > Established a PeerConnection and sent one video stream (ssrc=1) to Firefox. > Then sent a second video stream within the same peer connection (ssrc=2) to > Firefox. > > > Actual results: > > Video became messed up showing alternating messy segments of the two streams. > > > Expected results: > > Some day, there will be support for UNIFIED signalling and (if the second > SSRC would have been signalled that way) it would have been handled > accordingly. UNIFIED however is still not around the corner as suggested by: > > https://bugzilla.mozilla.org/show_bug.cgi?id=784517 > > Still, even with UNIFIED, an unannounced SSRC should just trigger a new > "anonymous" MediaStream. Please provide a link to any specification which suggests that this is the case. As far as I can tell, a major premise of unified plan is precisely the opposite: 1. Each media stream track is represented by its own unique m-line. This is a strict one-to-one mapping; a single media stream track cannot be spread across several m-lines, nor may a single m-line represent multiple media stream tracks. Note that this requires a modification to the way simulcast is currently defined by the individual draft [I-D.westerlund-avtcore-rtp-simulcast]. This does not preclude "application level" simulcasting; i.e., the creation of multiple media stream tracks from a single source. I'm assigning this bug to ABR for a second opinion but I strongly suspect it should be WONTFIXED.
> This statement seems premature at best. Time will show I guess. Until, then how about a more useful comment from Mozilla? You know, something other than: use multiple peer connections or wait 10 years until we are done specifying and implementing unified? ;)
(In reply to emcho from comment #8) > > This statement seems premature at best. > > Time will show I guess. > > Until, then how about a more useful comment from Mozilla? > > You know, something other than: use multiple peer connections or wait 10 > years until we are done specifying and implementing unified? ;) This bug is not an appropriate place to debate specification issues. Please take those to the WebRTC or RTCWEB WGs respectively. As I said in c7, this bug appears to directly contradict unified plan. Unless you can provide some specification-based rationale for this request, it us unlikely we will modify Firefox as requested here.
> This bug is not an appropriate place to debate specification > issues. Don't believe anyone is doing that. > Please provide a link to any specification which suggests that this > is the case. There's this: http://tools.ietf.org/html/draft-ietf-mmusic-msid-05#section-4.1 (first bullet) I also think there was a W3C discussion that went in the same direction. I'll post here if I find any trace of that.
(In reply to emcho from comment #0) > Still, even with UNIFIED, an unannounced SSRC should just trigger a new > "anonymous" MediaStream. No. Absent any signaling that assigns semantics to having multiple SSRCs in a single session (e.g., bundle), sending multiple SSRCs simultaneously is undefined behavior. In fact, this was the key objection that people who cared about legacy devices had to the now discarded "Plan B" that was proposed in (and discarded by) MMUSIC. Your application is sending nonsense, and Firefox is rendering nonsense.
Status: UNCONFIRMED → RESOLVED
Closed: 11 years ago
Resolution: --- → WONTFIX
> Your application is sending nonsense, and Firefox is rendering nonsense. Well that's clearly not true and, as you know, quite a misrepresentation. The application is sending valid RTP. That RTP is fully self-descriptive, decodable and renderable. You insist that Firefox still needs to be told what it gets. Fine: Please provide a way!
(In reply to emcho from comment #12) > > Your application is sending nonsense, and Firefox is rendering nonsense. > > Well that's clearly not true and, as you know, quite a misrepresentation. > The application is sending valid RTP. That RTP is fully self-descriptive, > decodable and renderable. Yes, and we are doing so correctly: by combining them into one video flow. As Adam says, this is at the heart of the reason for Unified Plan. > You insist that Firefox still needs to be told > what it gets. Fine: Please provide a way! The way is Unified Plan, which we are currently working on implementing. If you want it faster, patches to implement it would be welcome.
> Yes, and we are doing so correctly: by combining them into one video flow. That's a very eccentric interpretation of RFC3550 ... ah well ... anyways. Given that we are often asked about Firefox support and that this ticket was about a way to provide it, I’d like to end the discussion with the following summary of the situation: 1. Chrome currently provides support for handling multiple media streams (SSRCs) within a single connection. It does so through a non-standard mechanism known as “Plan B”. We currently rely on this mechanism for efficient multiparty video conferencing in Jitsi Meet. 2. In early 2013 a debate took place on the IETF about finding a standard way for managing multiple streams within a peer connection. There were several proposals, one of which, known as “Plan A” was authored by and very strongly supported by Mozilla. Arguments that “Plan A” would imply substantial complexity were vigorously countered. 3. After long debates the IETF agreed to adopt a “Plan UNIFIED” as a solution, which was basically a somewhat adjusted version of “Plan A”. All other proposals were rejected and in other words: Mozilla won! 4. AFAIK Mozilla has not even started implementing their own Plan UNIFIED and people are cautioned not to hold their breath because its implementation implies substantial complexity. 5. In the mean time, Mozilla is often encouraging people to basically abandon usage of multiple streams within a single PeerConnection and advising adoption of other means for handling multiparty conferencing 6. There seems to be absolutely no inclination in Mozilla to even consider compromises or alternative solutions. As a longtime admirer of Mozilla, as well as a Firefox user and supporter, I am personally quite saddened by the current state of things but … it is what it is.
> AFAIK Mozilla has not even started implementing their own Plan UNIFIED and people are cautioned not to > hold their breath because its implementation implies substantial complexity. I agree 100% with emcho. The progress and answers from Mozilla are just depressing...
If Mozilla is going to fight for a specific implementation, then please at least get onto implementing it as soon as possible. Given Skype/Miscrosoft's position as a key PRISM member, and WebRTC's unique position to help people break the Skype habit, this seems to be very strongly in line with Mozilla's mission statement of innovation and openness. What would it take to help make this happen? Testers? Cash from users? Help from relevant devs? If we know what it will take we can all pitch in and help find solutions to make it happen.
(In reply to user581 from comment #16) > What would it take to help make this happen? Testers? Cash from users? Help > from relevant devs? If we know what it will take we can all pitch in and > help find solutions to make it happen. Ultimately, it's a matter of development cycles. If you want to start by digging into Bug 784517 and crafting patches, I'll personally make sure they get reviewed (be sure to needinfo me, as I get a lot of bugmail, and will frequently miss simple comments). If you want to submit some patches here, the place to start is in media/webrtc/signaling/src/sipcc/core. Basically, the SDP handling needs to be updated to support an arbitrary number of audio, an arbitrary number of video, and one data channel (including the offer/answer handling). The interesting stuff is in the gsm and sdp directories. Then, this needs to be plumbed through to the Media Pipeline API that landed with Bug 786234.
Please remove "WONTFIX" from this bug entry due to public demend. Please implement multistream support in Firefox. Firefox does not support any alternative technology, so Jitsi developers have to rely on this method. Please use this draft standard for the implementation, just as Chrome/ium already did. Thanks!
(In reply to Gerald from comment #19) > Please remove "WONTFIX" from this bug entry due to public demend. Please > implement multistream support in Firefox. Firefox does not support any > alternative technology, so Jitsi developers have to rely on this method. > Please use this draft standard for the implementation, just as Chrome/ium > already did. Thanks! The key issue at this point is that this behavior would be out-of-spec (and I expect Chrome to begin deprecating it sometime soon-ish to match specified behavior). The most recent agreement about how to handle unannounced SSRCs is captured here: https://github.com/w3c/webrtc-pc/pull/29 Note that there has been substantial progress on Bug 784517 following our SDP rewrite, and the proper, spec-compliant mechanism for multistream handling should land very soon now.
You need to log in before you can comment on or make changes to this bug.