1303279 - Resolution for outbound video stream drops to 352x288 on replacing video track during webrtc call.

Status: VERIFIED FIXED

(Core :: WebRTC: Audio/Video, defect, P1)

Description

[Prerak Jain]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36

Steps to reproduce:

1. Start a webrtc call on FF.
2. During the call change the video track on peer connection object using replaceTrack(). In this case I started a new capture on a new video device during the call and replaced the old video track with new one.
3. All the video stream are captured at 1280x720


Actual results:

The repalceTrack call was successful and FF started sending the video from the new source. But in Firefox v 49 and above, as soon the track is replaced the outbound video stream resolution drops to 352x288 and does not ramp up during the call. 

Note - The stream is captured at 1280x720 and there are no network losses.


Expected results:

The outbound video stream resolution should not have dropped to 352x288. It tried it in Firefox 48, the resolution does not drops on replacing the tracks. I have started seeing the problem Firefox 49 and onwards.

Comment 1

[Loic]

Do you have any testcase to reproduce the issue?

Comment 2

[Prerak Jain]

(In reply to Loic from comment #1)
> Do you have any testcase to reproduce the issue?

Steps to reproduce the issue.

1. Attach an external camera to laptop.
2. Join a webrtc call using the inbuilt camera.(start video capture only on in built camera only at 1280x720)
3. In between the call, start capturing the video from the external camera at 1280x720.
4. Once the capture is started, replace the inbuilt camera video track in the peer connection object with the external one using rtcRtpSender.replaceTrack()and stop/kill the in-built camera stream.
5. Once the track is replaced successfully, firefox starts sending the outbound video at 352x288. (I have verified that camera capture is done at 1280x720)
6. This issues is reproducible 90% of times. Keep switching the camera during the call if for the first time resolution is not dropped.

Comment 3

[Alex Chronopoulos [:achronop]]

A similar problem exists after ICE restart. You can repro by following the link and executing the example: https://webrtc.github.io/samples/src/content/peerconnection/restart-ice/

On the console before and after the re-negotiation you will see:
Before: Remote video size changed to 640x480
After: Remote video size changed to 352x288

SDPs before and after restart are the same.

Note that according to the spec replaceTrack might require re-negotiation as well.

Comment 4

[Andreas Pehrson [:pehrsons]]

Also see the URL for a simple way to reproduce this.

Comment 5

[Andreas Pehrson [:pehrsons]]

I can confirm that this doesn't happen in 48.0.2.

Comment 6

[Randell Jesup [:jesup] (needinfo me)]

Attachment: reset codec resolution to last frame when reconfiguring in webrtc

Comment 7

[Randell Jesup [:jesup] (needinfo me)]

This was caused by some change in the MediaPipelineFactory probably, perhaps to ensure we switch mode if you replace a video feed with a screenshare feed.

Comment 8

[Randell Jesup [:jesup] (needinfo me)]

byron - just NIing you as an FYI, since this was fallout of some change in 49 to how ReplaceTrack works in PCMedia It hink, or maybe to MediaPipelineFactory.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/3649c55173d2
reset codec resolution to last frame when reconfiguring in webrtc r=pkerr

Comment 10

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/3649c55173d2

Comment 11

[Prerak Jain]

Randell - Is this fix going to be available for firefox beta or it will only be part of mozilla52 only ?

Comment 12

[Randell Jesup [:jesup] (needinfo me)]

I plan to ask for uplift to 51 and 50

Comment 13

[Andreas Pehrson [:pehrsons]]

It would be nice, and fairly simple, to have a mochitest for this.

Comment 14

[Randell Jesup [:jesup] (needinfo me)]

Comment on attachment 8796784 [details] [diff] [review]
reset codec resolution to last frame when reconfiguring in webrtc

Approval Request Comment
[Feature/regressing bug #]: Signaling change landed in 49

[User impact if declined]: An important usage pattern for webrtc will result in degraded image quality (highly so in the case of screenshares or HD captures

[Describe test coverage new/current, TreeHerder]: Manual testing.  This could be tested; I'll look to add this to the ReplaceTrack test

[Risks and why]: no risk - just ensures that the channel's current resolution and framerate aren't ignored on replaceTrack.

[String/UUID change made/needed]: none

Comment 15

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Hi Prerak, could you please verify this issue is fixed as expected on a latest Nightly build? Thanks!

Comment 16

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Comment on attachment 8796784 [details] [diff] [review]
reset codec resolution to last frame when reconfiguring in webrtc

Fixes a recent regression, Aurora51+, Beta50+

Comment 17

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/releases/mozilla-aurora/rev/176950592d50

Comment 18

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/releases/mozilla-beta/rev/cab24df88364

Comment 19

[Prerak Jain]

Randell - I have observed the issue again if you try to replace the stream more than once in call, for the first time resolution is unchanged but second time when you do it, the resolution drops back to 352x288.I tested in on 52.0a1.

Check https://jsfiddle.net/erxqr3qp/3/, hit the replace button twice you will observe the issue again.

I observed similar behaviour while during re-ice, if you do re-ice twice during a call the resolution is dropped to 352x288 during the second re-ice attempt.

Comment 20

[Randell Jesup [:jesup] (needinfo me)]

Confirmed; the mSendingWidth is reset later in that call, so calling it again will assume it's a 'new' conduit.  Very simple fix, will file a followup

Comment 21

[Cornel Ionce [:noni] [Hubs QA]]

Reproduced the initial issue on Firefox 49, build ID 20160916101415.

Confirming this is fixed on:
* Fx 50 RC, build ID 20161101104304,
* Latest 51.0a2 Aurora, build ID 20161102004003,
* Latest 52.0a1 Nightly, build ID 20161102030205.