1631263 - Support RTCRtpScriptTransform (formerly webrtc insertable streams)

Status: NEW

(Core :: WebRTC: Networking, enhancement, P2)

Description

[Jérôme]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0

Steps to reproduce:

Insertable streams provide a hook between the RTP (de)packetizer and the encoder/decoder as described here :
http://www.w3.org/2019/09/18-mediaprocessing-harald-insertable-media-processing.pdf

The Jisti team implemented an end-to-end encryption by using the insertable streams of Chrome with the Jisti Meet system :
https://jitsi.org/blog/e2ee/

This allows their video bridge (Selective Forwarding Unit) to easily forward packets whatever they are encrypted or not, whatever the receiver quality is.

On the end points, the insertable stream can be processed into a separated thread (or process ?), besides the usual (de)packetizers and encoder/decoder.

Actual results:

It seems Firefox doesn't support it.

Expected results:

Plan a support for this feature ? Are their any security issues ?

Comment 1

[Release mgmt bot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 2

[Nico Grunbaum [:ng, @chew:mozilla.org]]

Jib, would you mind taking a initial triage pass here?

Comment 3

[Óvári]

Please add insertable streams support to Firefox.

Access give access to:

• end-to-end encryption
• adding more reliability with forward error correction

For more information, please watch the “Jitsi Community Call” (Streamed live on 21 Apr 2020) at 21’30”
https://youtu.be/y4CW3c_Es4I?t=1290

100% support for Firefox (and other non-Chrome browsers)
https://github.com/jitsi/jitsi-meet/issues/4758

It is understood that Chromium has the insertable streams feature.

What do you think?

Thank you

Comment 4

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

I believe Mozilla is working on a position on insertable streams. We should probably open an issue on https://github.com/mozilla/standards-positions/

Comment 5

[Nils Ohlmeier [:drno]]

Correct. Mozilla first needs to decide about it position on https://github.com/mozilla/standards-positions/ what we think about Google insertable streams proposal. These discussions don't happen in bugzilla.

Comment 6

[Óvári]

(In reply to Jan-Ivar Bruaroey [:jib] (needinfo? me) from comment #4) and (In reply to Nils Ohlmeier [:drno] from comment #5)
https://github.com/mozilla/standards-positions/issues/330

Comment 7

[Nico Grunbaum [:ng, @chew:mozilla.org]]

It looks like this is being worked on in the standards-positions tracker. I am closing this as moved for now, it can be reopened or recreated when a position is taken.

Comment 8

[Jasper Knockaert]

I guess this issue can be reopened now that a position is taken.

Comment 9

[Alfredo]

Hi,
any news about this?

Comment 10

[Jasper Hugo]

(In reply to Nico Grunbaum [:ng, @chew:mozilla.org] from comment #7)

It looks like this is being worked on in the standards-positions tracker. I am closing this as moved for now, it can be reopened or recreated when a position is taken.

Now that a position has been taken could this bug be reopened? Or is there another bug tracking work on WebRTC insertable streams?

Comment 11

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Just to update this issue that the plan is to implement the latest spec, which is RTCRtpScriptTransform, and not any previous Chrome API. This is the API we'd implement that most matches the requests from major services.

Separately, the spec also offers the SFrameTransform API, which would provide sframe encoding natively without requiring this to be done by JS. This API is a bit less mature, and we haven't gotten requests for it yet. I'll open a separate issue on that.

Comment 13

[Jasper Hugo]

Jitsi Meet has just implemented RTCRtpScriptTransform support, so E2EE on Jitsi Meet will work in Firefox once Firefox gains RTCRtpScriptTransform support.

Comment 14

[pachainti]

Since firefox 96 fixed many issues about webRTC support https://bugzilla.mozilla.org/show_bug.cgi?id=1654112#c371. This is one important remaining bug. Is there any ETA?

Comment 15

[brandoncote]

Any update on this?

Comment 16

[Alfredo]

Any news?

Comment 17

[James Graham [:jgraham]]

https://bugzilla.mozilla.org/show_bug.cgi?id=1715625#c3 says that this is blocking Facetime support in Firefox.

Comment 18

[Ksenia Berezina [:ksenia]]

Given that we can get support for Facetime, setting webcompat priority as P1.

Comment 19

[timo.mayer]

Any news on this? This is very important for firefox compatibility, especially since the not support pushes Firefox in more unsecure areas when using e.g. Jitsi or any other Web-Video-Conference and e2ee is not possible due to the lack of support for this feature in FF.

Comment 20

[throw]

Has there been any progress now that this is P1?

As others have said this is a major blocker with using Firefox for end to end encryption in webRTC.

Comment 21

[Alfredo]

The next version 106 will contain many enhancements of WebRTC https://www.mozilla.org/en-US/firefox/106.0beta/releasenotes/, so maybe they will focus on that later on, I hope.

Comment 23

[Justin Peter]

Sorry to be the person to bother everyone again, but I think people would appreciate if you had some idea of what the priority and potential timeframe on this is, considering that it's the sole blocker for making multiple major services usable (or usable with increased functionality) in Firefox.

Comment 24

[Jim Mathies [:jimm]]

(In reply to Justin Peter from comment #23)

Sorry to be the person to bother everyone again, but I think people would appreciate if you had some idea of what the priority and potential timeframe on this is, considering that it's the sole blocker for making multiple major services usable (or usable with increased functionality) in Firefox.

We're currently working on getting this implemented!