976521 - Implement usage of "b=as" parameter to control outbound bandwidth for WebRTC

Status: NEW

(Core :: WebRTC, enhancement, P4)

Description

[Karthikeyan R]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.117 Safari/537.36

Steps to reproduce:

Change the SDP and mention the bandwidth you want to use for Audio/Video/Data channels of WebRTC PeerConnection.
Refer the html attachments in comment #3 of WebRTC's Bug @ https://code.google.com/p/webrtc/issues/detail?id=1846


Actual results:

No changes in bandwidth


Expected results:

Bandwidth should have been limited to the user specified speed.

Comment 1

[Maire Reavy [:mreavy]]

We don't currently support this.

Byron, Nils - What are your thoughts about implementing this?   Do you want the spec to get further along or can we start implementing soon?  Is it worth implementing Chrome's workaround of parsing the SDP?

Comment 2

[Byron Campen [:bwc]]

I suppose we could start implementing this. We seem to have all the parse code in place, so we should only need to plumb those attributes through JsepSessionImpl to the webrtc.org code.

Comment 3

[Byron Campen [:bwc]]

FWIW, we would need to do all that work even if we were only implementing some higher-level API on PeerConnection or RTPSender or what-have-you.

Comment 4

[Robert Jongbloed]

Is there anything happening with this issue?

Bandwidth control is very important to us at Blackboard. We must be able to indicate to the client a limit to it's transmitted it rate. Ideally dynamically via REMB or TMMBR (subject of another bug) but would settle for Mozilla just honouring the b=AS and/or b=TIAS provided in the SDP.

To reiterate the problem; we send SDP with:

m=video 58407 RTP/SAVPF 103
b=AS:200
b=TIAS:200000
.....

and Firefox Developer edition continues to send around 300kbps, sometimes ramping higher.

Comment 5

[Nikolai Alexandrov]

Gentleman,

I'm currently working for a company developing a WEBRTC based system and we are too pretty interested if this is going to happen anytime soon. It is a very nice and needed feature.

---------
Best regards
Nikolai Alexandrov
R&D engineer
Gold Broker 
www.gold-broker.eu

Comment 6

[chris.chen]

Not sure if this is concretely on firefox roadmap...?

Yeah, capability of respect b=AS bandwidth control is very important for us as well...

We are developing our webrtc platform, we had chrome bandwidth controlled, however, when it comes to firefox, it is just super greedy ...

Chris

Comment 7

[Eric Shepherd [:sheppy]]

Is twiddling with the SDP "by hand" something that the docs should be touching on at all? Not necessarily describing in detail every parameter (which may or may not be considered out-of-scope depending on how common a thing it is, although ideally someday this would be covered), but at least that it's a thing you can do and an example or two?

Or should MDN just avoid it entirely? I'm not intimate enough with the state of things and what's considered reasonable for content to do.

Marking dev-doc-needed to keep this on doc team's radar, just in case.

Comment 8

[Olexandr Vynnychenko]

I came across this issue when I tried streaming 720p and 1080p video from the web camera. Even in case of 480p the consumed bandwidth was 1.5-2.5 Mbps. It got really huge in higher resolutions.

We are developing a broadcasting system based on WebRTC. And Firefox is currently the only suitable browser because Chrome cannot forward audio stream. Also Firefox for Android is doing his job well here.

So the only huge problem is that we cannot specify lower bandwidth for the outgoing stream. It would be great to support it.

Olexandr

Comment 9

[Randell Jesup [:jesup] (needinfo me)]

(In reply to Olexandr Vynnychenko from comment #8)
> I came across this issue when I tried streaming 720p and 1080p video from
> the web camera. Even in case of 480p the consumed bandwidth was 1.5-2.5
> Mbps. It got really huge in higher resolutions.

640x480-ish should use around max 1.3Mbps (now; it used to use 2Mbps - Chrome last I knew uses 2Mbps by default for all resolutions).  We adapt the bandwidth limits to the resolution and framerate - yes, it's quite high at high resolutions and framerates.  On the other hand, if you cut the resolution input to 320x200 or lower, the bitrates drop a lot lower.

We scale semi-linearly with framerate for framerates below 30fps.

The idea is to avoid the video being low quality or wasting bits on improvements no one can see.

We do expect to land code for controlling bitrates and resolutions very soon (without editing SDP) using RTPSender/etc.

> We are developing a broadcasting system based on WebRTC. And Firefox is
> currently the only suitable browser because Chrome cannot forward audio
> stream. Also Firefox for Android is doing his job well here.
> 
> So the only huge problem is that we cannot specify lower bandwidth for the
> outgoing stream. It would be great to support it.

How low?  What framerate, and what sort of quality are you expecting?

Comment 10

[Olexandr Vynnychenko]

Thank you for your reply, Randell!

I recently transferred video stream from an Android device with Firefox to a desktop Firefox. The resolution dropped down automatically from 1280x720 to 480x270, and the bitrate still was 2.0-2.8 Mpbs when I moved the camera over the scene filled with many objects. It probably depends on the device I use.

It would be great to be able to limit the bandwidth of 1280x720 to 2Mbps, for example. I don't have any specific numbers as we are only building a proof of concept.

I know that 2Mbps is too low for such resolution for dynamic video. But it would be fine for just a face on a plain background. We are trying to broadcast to many peers, so network bandwidth may be an issue. The limit is needed just to make sure we will not overload the network when the guy on the video suddenly moves, and the bitrate increases.

Olexandr

Comment 11

[90210]

Hi there,

I'm working on a webrtc implementation as well and as many others came across this bug. As I saw in mozilla central (https://hg.mozilla.org/mozilla-central/rev/340c59ea962d) the built in webrtc bitrate limits where implemented in June 2015. The limits certainly make sense, if no specific bandwidth requirement is given, but an explicitly given bandwidth limit should always come first, as the devs will probably know best, which requirements to be met. Chrome already handles the b:AS bandwidth limits very well. Any chance to get this implemented in Firefox soon?

Best regards
Greg

Comment 12

[tzaitsev]

Enabling P2P without b=AS: has proven to crush the majority of our users networks.  We're holding off from enabling P2P in Firefox until we can control the target bandwidth through this or (any) way in Firefox.

Comment 13

[Tom Brückner]

I am trying to communicate with an older iPad 3 device (using the eface2face Cordova plugin for iOS). With the standard settings, quality is very bad, the only solution at the moment is to limit the bandwith on getUserMedia(). But since I don't know if the user on the other side is using such an old device, I would have to reset the connection once I detect an old device, which causes FF to ask the user for permission again. Bandwidth limitation would be very helpful here. In Chrome, I managed to fix the problem using bandwith throttling during the negotiation phase.

Comment 14

[Randell Jesup [:jesup] (needinfo me)]

(In reply to Tom Brückner from comment #13)
> I am trying to communicate with an older iPad 3 device (using the eface2face
> Cordova plugin for iOS). With the standard settings, quality is very bad,
> the only solution at the moment is to limit the bandwith on getUserMedia().
> But since I don't know if the user on the other side is using such an old
> device, I would have to reset the connection once I detect an old device,
> which causes FF to ask the user for permission again. Bandwidth limitation
> would be very helpful here. In Chrome, I managed to fix the problem using
> bandwith throttling during the negotiation phase.

Why would it require a new permission?  Attach the existing stream to the new call (and don't call .stop() on the tracks).  Or is the issue you need to reduce the resolution of the capture to implicitly reduce the bitrate?  You could also set track.enabled = false and force it to black/low-frame-rate to save bits during setup (perhaps the easiest)

Comment 15

[Tom Brückner]

Yes, I want to reduce the resolution and the framerate if an iPad <= 3 is detected to these constraints:

{
  audio: false,
  video: {
    frameRate: 10,
    width: {
      max: 360
    }
  }
}

track.stop() won't help me in this case because this would disable video completely.

I also saw there is a MediaStreamTrack.applyConstraints() method, but this doesn't seem to be implemented yet. It only throws overconstrained errors, for every type of constraints I hand over to it.

Comment 16

[Randell Jesup [:jesup] (needinfo me)]

(In reply to Tom Brückner from comment #15)
> Yes, I want to reduce the resolution and the framerate if an iPad <= 3 is
> detected to these constraints:
> 
> {
>   audio: false,
>   video: {
>     frameRate: 10,
>     width: {
>       max: 360
>     }
>   }
> }
> 
> track.stop() won't help me in this case because this would disable video
> completely.

Right, that's what I said.  track.enabled=false would 'mute' it, which should reduce bandwidth (you implied the need was only temporary for setup).

> I also saw there is a MediaStreamTrack.applyConstraints() method, but this
> doesn't seem to be implemented yet. It only throws overconstrained errors,
> for every type of constraints I hand over to it.

From a test:
    .then(() => navigator.mediaDevices.getUserMedia({video: true, audio: true}))
      .then(stream => stream.getVideoTracks()[0].applyConstraints({ width: 320 })
      .then(() => stream.getAudioTracks()[0].applyConstraints({ })))
    .then(() => ok(true, "applyConstraints code exercised"))

Comment 17

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

track.applyConstraints was implemented in Firefox 43 (bug 912342) and works for me.

It is per-track, so it expects only the part after video:

Try it here https://jsfiddle.net/g5zvzcw3/

Comment 18

[Tom Brückner]

> you implied the need was only temporary for setup

Then I explained it wrong, probably. It is not temporary for setup. I want to limit bandwidth permanently if one of the parties involved in the call is using an old device. Unfortunately, the automatic stream degradation of WebRTC does not work. On my iPad 3, the streams freeze if I don't limit the bandwidth in getUserMedia, and after a while they disconnect.

> track.applyConstraints was implemented in Firefox 43

Strange. I will try again. When I tried it yesterday, all I got was an overconstrained error.

Comment 19

[Nils Ohlmeier [:drno]]

Might be worth trying to implement b=TIAS per RFC 3890 https://tools.ietf.org/html/rfc3890 right away.

Comment 20

[Nils Ohlmeier [:drno]]

Attachment: MozReview Request: Bug 976521: use b=TIAS for maxBitrate encoding constraints

Review commit: https://reviewboard.mozilla.org/r/54412/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/54412/

Comment 21

[Nils Ohlmeier [:drno]]

Note for all the people who have requested this feature: the plan right now is to land support b=TIAS in bug 1276368 for Firefox 49, which then allows you to restrict the sending bandwidth of Firefox video encoders.

Comment 23

[Bulk Bug Changes for mreavy's org]

Mass change P3->P4 to align with new Mozilla triage process.

Comment 24

[Nils Ohlmeier [:drno]]

I don't have the capacity any more to work on this any time soon.

Comment 25

[BugBot [:suhaib / :marco/ :calixte]]

The severity field for this bug is relatively low, S3. However, the bug has 19 votes.
:mjf, could you consider increasing the bug severity?

For more information, please visit auto_nag documentation.