Closed Bug 1237630 Opened 4 years ago Closed 4 years ago

Image freezes and no sound available after ~2 minutes call in https://appear.in/

Categories

(Core :: WebRTC, defect, P1)

45 Branch
defect

Tracking

()

VERIFIED FIXED
mozilla45
Tracking Status
firefox45 --- fixed
firefox46 --- verified
Blocking Flags:

People

(Reporter: bogdan_maris, Assigned: pkerr)

References

Details

(Keywords: regression)

Attachments

(2 files, 1 obsolete file)

STR:
1. Start Firefox (we tested with latest Aurora on both devices PC and phone).
2. Visit https://appear.in/
3. Start a call between the two devices.

Expected: During the call there are no issues.

Actual results:
During a call between Asus P024 tablet (android version 5.0.2) and Ubuntu 14.04 x64 the image freezes and no sound for each guest (on tablet the person from Ubuntu is freezed and the other way around for desktop user). 

Same thing happens using a Nexus 5 with android 6.0.1 and Ubuntu 14.04 x64 and, Windows 7 x64; 

Notes:
- No issues were recorded while making a call between Nexus 5 with android 6.0.1 and Chromium (Ubuntu 14.04 x64)
- Will investigate further if this is a recent regression
I tried this a number of times with a Nexus 10 and Windows, both running Nightly.

Several times (but not all) I had Android->Windows video freeze; audio continued.   Once, after a long time with Android video frozen, video from Windows->android froze. Again, audio continued.

I've had only one other instance of video (only) freezing recently; during a talky.io call (from the same windows machine); the other person said my video froze after a few minutes.  I was runnging a second brwoser and the system was overloaded; I killed the second browser, reloaded the page and video and audio were fine for 45 minutes.

What we need is a log and maybe wireshark trace.  set NSPR_LOG_MODULES=signaling:4,webrtc_trace:65535 WEBRTC_TRACE_FILE=nspr NSPR_LOG_FILE=some temp file.   Try to record approximately when the freeze starts for correlating to the log timestamps.  I'll try this myself.

There might be a talky.io change, or there might be fallout from the webrtc.org 43 update. It might be a recent regression, though I can't think of any source for such a regression.   If so, we should be seeing it in Aurora/45 as well, and this report was initially against Aurora.  The initial report here was both audio and video failing, which I have not seen, so these may *not* be the same issue - though the logs needed are the same.
backlog: --- → webrtc/webaudio+
Rank: 10
Keywords: regression
Priority: -- → P1
If it is the 43 update, it's likely some busted assumption in the 43 error recovery code
Assignee: nobody → pkerr
I also see this happen - appear.in sessions last about 10 minutes max. It's happening with both Aurora and Nightly users (45 and 46). We've had to switch away from WebRTC meetings because of it.

I'll look at doing a wireshark trace.

I generally see all connections lost at the same time, with all of the connections timing out, though sometimes one user stays connected for a little while longer (it seemed to always be the only other Nightly / Aurora user)
Thanks, Thomas, for chiming in. pkerr is taking the lead on fixing this, and I hope to have a fix within the next couple of days -- depending on how easy this is to reproduce. If we have trouble repro'ing, we'll definitely reach out to you -- and we'll ask you to test once we have a patch that we believe fixes the problem.
I am able to reproduce between my Ubuntu machine and Android tablet via talky.io. Android side stops sending video. I am working on log collection and analysis.
Duplicate of this bug: 1237301
Status: NEW → ASSIGNED
It appears that there is a new check being used in the video_sender: _codecDataBase.MatchesCurrentResolution() that begins to fail after the quality manager makes changes in the spatial resolution. This method did not exist in the VCMCodecDataBase class in our previous webrtc.org import. Based on a TODO left in the VideoSender::AddVideoFrame method, I believe I will need to add code to synchronize the changes in frame size of the I420VideoFrames being generated with the width and height recorded in the _codecDataBase.
The change mentioned above was introduced into the webrtc.org codebase through issue https://webrtc-codereview.appspot.com/48379004. The same code exists in the branch48 repo.
Fix for video freeze
Log has changed in VCMCodecDataBase::SuportsRenderScheduling method and the existing error message is no longer valid.
Comment on attachment 8710487 [details] [diff] [review]
Part 1: Video freeze from WebRTC sender

I have tested this under cpu stress on OS X, Linux and Android to the point where the LoadManager triggers a frame size change. This would have triggered the frame drops in the past but now the correct adaptation takes place and video remains stable.
Attachment #8710487 - Flags: review?(rjesup)
Attachment #8710489 - Flags: review?(rjesup)
Comment on attachment 8710487 [details] [diff] [review]
Part 1: Video freeze from WebRTC sender

Review of attachment 8710487 [details] [diff] [review]:
-----------------------------------------------------------------

::: media/webrtc/trunk/webrtc/modules/video_coding/main/source/video_sender.cc
@@ -379,5 @@
> -  if (!_codecDataBase.MatchesCurrentResolution(videoFrame.width(),
> -                                               videoFrame.height())) {
> -    LOG(LS_ERROR) << "Incoming frame doesn't match set resolution. Dropping.";
> -    return VCM_PARAMETER_ERROR;
> -  }

Because upstream will continue to have this, I think I'd prefer to ifdef this code out for Mozilla.  This will make future merges a little easier
Attachment #8710487 - Flags: review?(rjesup) → review+
Attachment #8710489 - Flags: review?(rjesup) → review+
Target Milestone: --- → mozilla45
Version: unspecified → 45 Branch
Incorporate review comment. Carry forward r+.
Attachment #8710487 - Attachment is obsolete: true
Comment on attachment 8710644 [details] [diff] [review]
Part 1: Video freeze from WebRTC sender

Review of attachment 8710644 [details] [diff] [review]:
-----------------------------------------------------------------

Carry forward r=rjesup.
Attachment #8710644 - Flags: review+
Comment on attachment 8710644 [details] [diff] [review]
Part 1: Video freeze from WebRTC sender

Approval Request Comment
[Feature/regressing bug #]: 1237630
[User impact if declined]:
If the system load reaches the point that the LoadManager requests a change in the frame size the video encoder will begin to drop frames. The user's video will appear to freeze on the receiving end of the connection.
[Describe test coverage new/current, TreeHerder]:
There is no performance test currently that would trigger this condition. 
[Risks and why]: Low risk.
A new frame size verification test that was added in the newer webrtc.org release branch (43) is meant to handle a theoretical bug (see link in comment above) that may appear when using hardware codecs. Mozilla does not use hardware codecs outside of FxOS. The change made with this patch reverts the video encode path operation back to what has successfully be the model in previous releases of WebRTC from Mozilla: the codec is responsible for handling frame size changes. In addition, manual testing has been done on OS X, Linux and Android to first reliably trigger the video freeze and then, after applying the patch, preventing the freeze.
[String/UUID change made/needed]: none needed
Attachment #8710644 - Flags: approval-mozilla-aurora?
Comment on attachment 8710644 [details] [diff] [review]
Part 1: Video freeze from WebRTC sender

Fix a freeze, taking it.
Attachment #8710644 - Flags: approval-mozilla-aurora? → approval-mozilla-aurora+
Flags: qe-verify+
I did not reproduce this anymore on a appear.in call between Asus ZenPad 8 (Android 5.0.2) and Ubuntu 14.04 32-bit and Windows 7 64-bit using Firefox 46 beta 8.
Status: RESOLVED → VERIFIED
Flags: qe-verify+
You need to log in before you can comment on or make changes to this bug.