When renegotiating, Firefox does not respect previously negotiated codec payload types

RESOLVED INVALID

Status

()

Core
WebRTC: Signaling
RESOLVED INVALID
a year ago
a year ago

People

(Reporter: Iñaki Baz Castillo, Unassigned)

Tracking

54 Branch
Points:
---

Firefox Tracking Flags

(firefox54 affected)

Details

(Reporter)

Description

a year ago
Below a set of sequential SDP O/A procedures between a SFU server and Firefox. BUNDLE is used for all the SDP media sections.

Note that Firefox is the only "participant" in the SFU so there is no multi-stream in this scenario (to make it simpler for inspection).




1) Firefox receives an SDP offer from the SFU:

--------------------------------------------------------------
v=0
o=mediasoup 1945617887998605 1 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-lite
a=fingerprint:sha-256 01:29:C9:CD:40:21:46:4E:27:D4:28:AF:41:B1:A6:CE:E4:F5:AF:00:C6:E7:28:4B:57:DF:82:CE:75:37:57:4E
a=msid-semantic: WMS *
a=group:BUNDLE recv-audio-track-1 recv-video-track-1

m=audio 7 RTP/SAVPF 100
c=IN IP4 127.0.0.1
a=rtpmap:100 opus/48000/2
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=setup:actpass
a=mid:recv-audio-track-1
a=recvonly
a=ice-ufrag:1zqxjo6cupqr4znj
a=ice-pwd:qwzast7lfuv8pk48ndcf1uwisv8dulll
a=candidate:udpcandidate 1 udp 1078862079 192.168.1.38 48705 typ host
a=rtcp-mux
a=rtcp-rsize

m=video 7 RTP/SAVPF 123
c=IN IP4 127.0.0.1
a=rtpmap:123 VP8/90000
a=rtcp-fb:123 nack
a=rtcp-fb:123 nack pli
a=rtcp-fb:123 ccm fir
a=setup:actpass
a=mid:recv-video-track-1
a=recvonly
a=ice-ufrag:1zqxjo6cupqr4znj
a=ice-pwd:qwzast7lfuv8pk48ndcf1uwisv8dulll
a=candidate:udpcandidate 1 udp 1078862079 192.168.1.38 48705 typ host
a=rtcp-mux
a=rtcp-rsize
--------------------------------------------------------------




2) Firefox creates a PeerConnection and attaches a local MediaStream with audio&video to it. Then it creates an answer and sends it back to the SFU:

--------------------------------------------------------------
v=0
o=mozilla...THIS_IS_SDPARTA-54.0a1 2010012012176775199 1 IN IP4 0.0.0.0
s=-
t=0 0
a=sendrecv
a=fingerprint:sha-256 82:CE:F7:20:EA:A5:B3:E6:FA:7F:77:61:E9:9F:AD:1E:58:CA:DF:04:E3:A6:71:AD:7C:BD:51:00:2A:D4:4F:1A
a=group:BUNDLE recv-audio-track-1 recv-video-track-1
a=ice-options:trickle
a=msid-semantic:WMS *

m=audio 9 RTP/SAVPF 100
c=IN IP4 0.0.0.0
a=sendonly
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=ice-pwd:5e0e5ebbfe3dba80d8f34777d73665f0
a=ice-ufrag:10a8920d
a=mid:recv-audio-track-1
a=msid:{4b39a180-d8d1-e146-a60a-8241c1432e74} {a3477340-ca87-4a49-af03-9519971e5c78}
a=rtcp-mux
a=rtpmap:100 opus/48000/2
a=setup:active
a=ssrc:1070920025 cname:{aa8a2862-f544-5a40-9714-6782289d03ad}

m=video 9 RTP/SAVPF 123
c=IN IP4 0.0.0.0
a=sendonly
a=ice-pwd:5e0e5ebbfe3dba80d8f34777d73665f0
a=ice-ufrag:10a8920d
a=mid:recv-video-track-1
a=msid:{4b39a180-d8d1-e146-a60a-8241c1432e74} {6f6c9232-7c0c-f249-9307-8d6ea9d08f33}
a=rtcp-fb:123 nack
a=rtcp-fb:123 nack pli
a=rtcp-fb:123 ccm fir
a=rtcp-mux
a=rtpmap:123 VP8/90000
a=setup:active
a=ssrc:1562214261 cname:{aa8a2862-f544-5a40-9714-6782289d03ad}
--------------------------------------------------------------

So, Firefox correctly sets codecs payload types to those in the offer (opus PT 100 and VP8 PT 123).




3) Later, Firefox removes its local video (webcam) from the PeerConnection, calls createOffer() and sends the re-offer to the SFU:

--------------------------------------------------------------
v=0
o=mozilla...THIS_IS_SDPARTA-54.0a1 2010012012176775199 2 IN IP4 0.0.0.0
s=-
t=0 0
a=sendrecv
a=fingerprint:sha-256 82:CE:F7:20:EA:A5:B3:E6:FA:7F:77:61:E9:9F:AD:1E:58:CA:DF:04:E3:A6:71:AD:7C:BD:51:00:2A:D4:4F:1A
a=ice-options:trickle
a=msid-semantic:WMS *

m=audio 63752 RTP/SAVPF 100 9 0 8 101 109
c=IN IP4 192.168.1.38
a=candidate:0 1 UDP 2122252543 192.168.1.38 63752 typ host
a=sendrecv
a=extmap:1/sendonly urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=fmtp:109 maxplaybackrate=48000;stereo=1;useinbandfec=1
a=fmtp:101 0-15
a=ice-pwd:5e0e5ebbfe3dba80d8f34777d73665f0
a=ice-ufrag:10a8920d
a=mid:recv-audio-track-1
a=msid:{4b39a180-d8d1-e146-a60a-8241c1432e74} {a3477340-ca87-4a49-af03-9519971e5c78}
a=rtcp-mux
a=rtpmap:100 opus/48000/2
a=rtpmap:9 G722/8000/1
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:101 telephone-event/8000
a=rtpmap:109 opus/48000/2
a=setup:actpass
a=ssrc:1070920025 cname:{aa8a2862-f544-5a40-9714-6782289d03ad}

m=video 0 RTP/SAVPF 120
c=IN IP4 0.0.0.0
a=inactive
a=rtpmap:120 VP8/90000
--------------------------------------------------------------

Here, the removed video track (a=inactive and port 0) has a suspect PT 120... Well, it does not cause any trouble as, at the end, it's a removed track. OK.




4) The SFU replies with its SDP answer:

--------------------------------------------------------------
v=0
o=mediasoup 1945617887998605 0 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-lite
a=fingerprint:sha-256 01:29:C9:CD:40:21:46:4E:27:D4:28:AF:41:B1:A6:CE:E4:F5:AF:00:C6:E7:28:4B:57:DF:82:CE:75:37:57:4E
a=msid-semantic: WMS *
a=group:BUNDLE recv-audio-track-1 recv-video-track-1

m=audio 7 RTP/SAVPF 100
c=IN IP4 127.0.0.1
a=rtpmap:100 opus/48000/2
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=setup:passive
a=mid:recv-audio-track-1
a=recvonly
a=ice-ufrag:1zqxjo6cupqr4znj
a=ice-pwd:qwzast7lfuv8pk48ndcf1uwisv8dulll
a=candidate:udpcandidate 1 udp 1078862079 192.168.1.38 48705 typ host
a=rtcp-mux
a=rtcp-rsize

m=video 0 RTP/SAVPF 123
c=IN IP4 127.0.0.1
a=rtpmap:123 VP8/90000
a=rtcp-fb:123 nack
a=rtcp-fb:123 nack pli
a=rtcp-fb:123 ccm fir
a=mid:recv-video-track-1
a=inactive
--------------------------------------------------------------




5) And later, Firefox adds a new local video track to its PeerConnection by means of addTrack(track, stream), creates a new offer and sends it to the SFU:

--------------------------------------------------------------
v=0
o=mozilla...THIS_IS_SDPARTA-54.0a1 2010012012176775199 3 IN IP4 0.0.0.0
s=-
t=0 0
a=sendrecv
a=fingerprint:sha-256 82:CE:F7:20:EA:A5:B3:E6:FA:7F:77:61:E9:9F:AD:1E:58:CA:DF:04:E3:A6:71:AD:7C:BD:51:00:2A:D4:4F:1A
a=group:BUNDLE recv-audio-track-1 recv-video-track-1
a=ice-options:trickle
a=msid-semantic:WMS *

m=audio 63752 RTP/SAVPF 100 9 0 8 101 109
c=IN IP4 192.168.1.38
a=candidate:0 1 UDP 2122252543 192.168.1.38 63752 typ host
a=sendrecv
a=extmap:1/sendonly urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=fmtp:109 maxplaybackrate=48000;stereo=1;useinbandfec=1
a=fmtp:101 0-15
a=ice-pwd:5e0e5ebbfe3dba80d8f34777d73665f0
a=ice-ufrag:10a8920d
a=mid:recv-audio-track-1
a=msid:{4b39a180-d8d1-e146-a60a-8241c1432e74} {a3477340-ca87-4a49-af03-9519971e5c78}
a=rtcp-mux
a=rtpmap:100 opus/48000/2
a=rtpmap:9 G722/8000/1
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:101 telephone-event/8000
a=rtpmap:109 opus/48000/2
a=setup:actpass
a=ssrc:1070920025 cname:{aa8a2862-f544-5a40-9714-6782289d03ad}

m=video 9 RTP/SAVPF 120 121 126 97
c=IN IP4 0.0.0.0
a=sendrecv
a=fmtp:126 profile-level-id=42e01f;level-asymmetry-allowed=1;packetization-mode=1
a=fmtp:97 profile-level-id=42e01f;level-asymmetry-allowed=1
a=fmtp:120 max-fs=12288;max-fr=60
a=fmtp:121 max-fs=12288;max-fr=60
a=ice-pwd:5e0e5ebbfe3dba80d8f34777d73665f0
a=ice-ufrag:10a8920d
a=mid:recv-video-track-1
a=msid:{4b39a180-d8d1-e146-a60a-8241c1432e74} {b2813275-79f5-734c-b4ba-a0933a6c211d}
a=rtcp-fb:120 nack
a=rtcp-fb:120 nack pli
a=rtcp-fb:120 ccm fir
a=rtcp-fb:120 goog-remb
a=rtcp-fb:121 nack
a=rtcp-fb:121 nack pli
a=rtcp-fb:121 ccm fir
a=rtcp-fb:121 goog-remb
a=rtcp-fb:126 nack
a=rtcp-fb:126 nack pli
a=rtcp-fb:126 ccm fir
a=rtcp-fb:126 goog-remb
a=rtcp-fb:97 nack
a=rtcp-fb:97 nack pli
a=rtcp-fb:97 ccm fir
a=rtcp-fb:97 goog-remb
a=rtcp-mux
a=rtpmap:120 VP8/90000
a=rtpmap:121 VP9/90000
a=rtpmap:126 H264/90000
a=rtpmap:97 H264/90000
a=setup:actpass
a=ssrc:3944619480 cname:{aa8a2862-f544-5a40-9714-6782289d03ad}
--------------------------------------------------------------

So here, Firefox is re-using the inactive video m= section to add its new local video track (that's ok). But: it's NOT respecting the previously negotiated codec payload types. As you can see, it's offering VP8 PT 120 instead of the previously negotiated PT 123.




https://github.com/rtcweb-wg/jsep/issues/266 clearly states that, during a re-offer, the browser "will offer everything it can do at that point of time, but will put the stuff currently in use first". Firefox is not satisfying such a text (it should re-offer with VP8 PT 123 as previously negotiated within the same "bundled" RTP session).
rtpmap is totally meaningless on a disabled m-section, because no codec has actually been negotiated. Payload type 123 is no longer in use once the first renegotiation completes.

I suppose it might be useful to remember the old payload types from the last time an m-section was successfully negotiated, but it seems a whole lot simpler to just start from scratch on both ends. A disabled m-section is supposed to be a blank slate, after all.
(Reporter)

Comment 2

a year ago
Byron, I don't care at all about the disabled m-section. The problem is described in bullet 5) above, when Firefox adds a new video track so the previously disabled m-section becomes enabled again.

As described above, the m-section (enabled) offers VP8 PT 120 rather than 123. That's the problem.
(Reporter)

Comment 3

a year ago
Please check https://github.com/rtcweb-wg/jsep/issues/266. The re-offerer MUST use the same PT values as previously negotiated over the whole RTP session (this is, all the bundled m-sections), so "starting from scratch" is not valid.

A re-negotiation is not a fresh negotiation because there already was an initial negotiation that set up some rules (as PT values, RTP header extension IDs, etc). And according to https://github.com/rtcweb-wg/jsep/issues/266 those values must be kept in future re-negotiations.
Uh, please point to the precise text which you think says this
(Reporter)

Comment 5

a year ago
I understood that from the previous comments. I'll recheck.

BTW: You may also want to take a loot to the bullet 3) above in which Firefox creates a re-offer with:

a=rtpmap:100 opus/48000/2
a=rtpmap:9 G722/8000/1
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:101 telephone-event/8000
a=rtpmap:109 opus/48000/2

(opus appears twice with different PTs).
(Reporter)

Comment 6

a year ago
OK, as Roman Shpount pointed out, RFC 3264 clearly allows it:

-------------------------
RFC3264 8.3.2 Changing the Set of Media Formats

However, in the case of RTP, the mapping from a particular dynamic payload type number to a particular codec within that media stream MUST NOT change for the duration of a session. For example, if A generates an offer with G.711 assigned to dynamic payload type number 46, payload type number 46 MUST refer to G.711 from that point forward in any offers or answers for that media stream within the session. However, it is acceptable for multiple payload type numbers to be mapped to the same codec, so that an updated offer could also use payload type number 72 for G.711.
-------------------------

Fair. But please, don't offer the same codec twice with different PT values...

Thanks a lot.
(In reply to Iñaki Baz Castillo from comment #6)
> Fair. But please, don't offer the same codec twice with different PT
> values...

Lets take that discussion to bug 1247725 (especially comment #3 appears to relevant for that).
Status: NEW → RESOLVED
Last Resolved: a year ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.