1631263 - Support RTCRtpScriptTransform (formerly webrtc insertable streams)

Status: NEW

(Core :: WebRTC: Networking, enhancement, P2)

Description

[Jérôme]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0

Steps to reproduce:

Insertable streams provide a hook between the RTP (de)packetizer and the encoder/decoder as described here :
http://www.w3.org/2019/09/18-mediaprocessing-harald-insertable-media-processing.pdf

The Jisti team implemented an end-to-end encryption by using the insertable streams of Chrome with the Jisti Meet system :
https://jitsi.org/blog/e2ee/

This allows their video bridge (Selective Forwarding Unit) to easily forward packets whatever they are encrypted or not, whatever the receiver quality is.

On the end points, the insertable stream can be processed into a separated thread (or process ?), besides the usual (de)packetizers and encoder/decoder.

Actual results:

It seems Firefox doesn't support it.

Expected results:

Plan a support for this feature ? Are their any security issues ?

Comment 1

[Release mgmt bot [:sylvestre / :calixte / :marco for bugbug]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 2

[Nico Grunbaum [:ng, @chew:mozilla.org]]

Jib, would you mind taking a initial triage pass here?

Comment 3

[Óvári]

Please add insertable streams support to Firefox.

Access give access to:

• end-to-end encryption
• adding more reliability with forward error correction

For more information, please watch the “Jitsi Community Call” (Streamed live on 21 Apr 2020) at 21’30”
https://youtu.be/y4CW3c_Es4I?t=1290

100% support for Firefox (and other non-Chrome browsers)
https://github.com/jitsi/jitsi-meet/issues/4758

It is understood that Chromium has the insertable streams feature.

What do you think?

Thank you

Comment 4

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

I believe Mozilla is working on a position on insertable streams. We should probably open an issue on https://github.com/mozilla/standards-positions/

Comment 5

[Nils Ohlmeier [:drno]]

Correct. Mozilla first needs to decide about it position on https://github.com/mozilla/standards-positions/ what we think about Google insertable streams proposal. These discussions don't happen in bugzilla.

Comment 6

[Óvári]

(In reply to Jan-Ivar Bruaroey [:jib] (needinfo? me) from comment #4) and (In reply to Nils Ohlmeier [:drno] from comment #5)
https://github.com/mozilla/standards-positions/issues/330

Comment 7

[Nico Grunbaum [:ng, @chew:mozilla.org]]

It looks like this is being worked on in the standards-positions tracker. I am closing this as moved for now, it can be reopened or recreated when a position is taken.

Comment 8

[Jasper Knockaert]

I guess this issue can be reopened now that a position is taken.

Comment 9

[Alfredo]

Hi,
any news about this?

Comment 10

[Jasper Hugo]

(In reply to Nico Grunbaum [:ng, @chew:mozilla.org] from comment #7)

It looks like this is being worked on in the standards-positions tracker. I am closing this as moved for now, it can be reopened or recreated when a position is taken.

Now that a position has been taken could this bug be reopened? Or is there another bug tracking work on WebRTC insertable streams?

Comment 11

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Just to update this issue that the plan is to implement the latest spec, which is RTCRtpScriptTransform, and not any previous Chrome API. This is the API we'd implement that most matches the requests from major services.

Separately, the spec also offers the SFrameTransform API, which would provide sframe encoding natively without requiring this to be done by JS. This API is a bit less mature, and we haven't gotten requests for it yet. I'll open a separate issue on that.