1675364 - Video conference *Zoom* with jumps

Status: RESOLVED INVALID

(Core :: WebRTC: Audio/Video, defect, P2)

Description

[Paul Menzel]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:84.0) Gecko/20100101 Firefox/84.0

Steps to reproduce:

Using Nightly, doing a video conference with Zoom.

Actual results:

The video is not played at constant speed, but seems to be a little slower and after ten seconds, it very fast to catch up.

Expected results:

The video should be played at constant speed.

I created a media profile 1. (During with the video and audio did not work at all.)

Comment 1

[Paul Menzel]

Attachment: Output of `about:support`

Comment 2

[Paul Menzel]

Audio also seems to be lagging and sometimes be slower and sometimes faster to catch up.

Comment 3

[Paul Adenot (:padenot)]

Paul, this profile contains almost nothing of interest. On Linux, the profiler unfortunately takes some time to start (that's a bug on our side). It might be better to start it while in a call where you see the problem, if that's an option for you?

It's also very likely that what you see is a behaviour of Zoom, that doesn't use any of the facilities of the browser, and reimplements everything in javascript (which can be quite slow sometimes compared to what native code can do). Was your machine doing anything else while this was happening? Is it recent, or on the older side?

Comment 4

[Paul Menzel]

(In reply to Paul Adenot (:padenot) from comment #3)

Paul, this profile contains almost nothing of interest. On Linux, the profiler unfortunately takes some time to start (that's a bug on our side). It might be better to start it while in a call where you see the problem, if that's an option for you?

The profile was generated while in in a call (and audio and video stopped working right away until the profiler was stopped).

It's also very likely that what you see is a behaviour of Zoom, that doesn't use any of the facilities of the browser, and reimplements everything in javascript (which can be quite slow sometimes compared to what native code can do).

Some vendor design decisions are very strange. (I wonder how much more energy is wasted that way for video calls and who pays the energy bills.)

Was your machine doing anything else while this was happening? Is it recent, or on the older side?

No, just the call (and several other tabs open). And yes, it’s a five year old Intel Broadwell Dell Latitude E7250, where the graphics device only supports H264 and VP8 decoding.

$ vainfo
libva info: VA-API version 1.9.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_9
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.9 (libva 2.9.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 20.3.0 ()
vainfo: Supported profile and entrypoints
      VAProfileNone                   :	VAEntrypointVideoProc
      VAProfileNone                   :	VAEntrypointStats
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Simple            :	VAEntrypointEncSlice
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointFEI
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointEncSlice
      VAProfileH264High               :	VAEntrypointFEI
      VAProfileVC1Simple              :	VAEntrypointVLD
      VAProfileVC1Main                :	VAEntrypointVLD
      VAProfileVC1Advanced            :	VAEntrypointVLD
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline:	VAEntrypointFEI
      VAProfileVP8Version0_3          :	VAEntrypointVLD

I have the acceleration enabled for Firefox

MOZ_X11_EGL=1 MOZ_LOG="PlatformDecoderModule:1" nightly

but I couldn’t find out, what codec is used in about:webrtc.

Comment 5

[Paul Adenot (:padenot)]

Well for starters you GPU will not be used for anything related to encoding/decoding off-loading, because Zoom is doing everything in js, including video and audio decoding, encoding, playback, audio processing such as echo cancellation, etc.

If you're using a service that uses the facilities the browser has (regular WebRTC APIs), then yes, we try to use the hardware if possible, but you see that this is not finished yet on Linux, need to flip preferences, etc. It's getting there, I don't know yet exactly how this work for WebRTC, but it's working increasingly well for playback on Linux.

In the grand scheme of things, this machine is perfectly capable of doing high quality calls I assume, but maybe not quite when everything is reimplemented in js.

It might be worth it to try to get a profile. I happen to have an old Linux laptop around that should be fairly equivalent to what you have (not exactly but close), I'll try to do a Zoom call and see if I can do a profile myself, trying to side step the issues we have on Linux, or generally assess the experience of Zoom Web in Firefox on those older machines.

Thanks for the report in any case, but I'm afraid that there isn't something quick we can do about this. I'll report back if/when I get a profile.

Comment 6

[Paul Adenot (:padenot)]

https://www.cpubenchmark.net/compare/Intel-i5-5300U-vs-Intel-i7-2620M/2459vs872 is the CPU I tested with vs. the CPU you have (from searching the web with the model name). Yours should be slightly faster, the machine I'm testing on has 8GB of RAM.

https://share.firefox.dev/2I3nxQv is a profile I could get of a call, it contains nothing for a good 20s, and then shows that Zoom uses lots of memory, and if I use htop, I see that it's using almost of all the machine's CPU (all cores at a rather high percentage). This was a 1:1 call between this linux machine and another machine where zoom web usually has no problem. How much memory does your computer have? Apparently the zoom web app is leaking lots of buffers that come from the buffer backing a canvas. By the end of the call, the process that was running the Zoom call was at about 4.5GB of resident memory usage, so I wonder if you're swapping? Mine was just a couple minutes of call.

While the profiler was running, the video was extremely slow, about a frame every two seconds, and because the profiler initialization is pausing the process, the audio is severely delayed by a few seconds, but then recovers, and is flawless. When the profiler was not running it's more or less fine (again, just a 1:1 call, I don't know if you were doing a scenario with more peers).

Comment 7

[Paul Menzel]

(In reply to Paul Adenot (:padenot) from comment #6)

[…]

Thank you for testing this so quickly. My machine has 16 GB of memory, but I’ll check next week, where the pressure comes from.

While the profiler was running, the video was extremely slow, about a frame every two seconds, and because the profiler initialization is pausing the process, the audio is severely delayed by a few seconds, but then recovers, and is flawless. When the profiler was not running it's more or less fine (again, just a 1:1 call, I don't know if you were doing a scenario with more peers).

Ok, I’ll wait longer next time I guess. It was a call with several participants, but I do not know how Zoom works, but thought I only get one stream from the Zoom server. Only the video of the actively speaking person was shown. I believe the Zoom Web client does not allow to view several speakers. Just to make sure, you tested with Microsoft Windows, right?

Lastly, I only noticed the non-constant speed, because, let’s say, the speaker had a metronom running.

Comment 8

[Paul Menzel]

I am also fine with closing the bug report, so more focus can be put on alternatives using open standards and FLOSS. Zoom can do the performance analysis und improve Firefox themselves.

Comment 9

[Paul Adenot (:padenot)]

Just to make sure, you tested with Microsoft Windows, right?

All my testing/measurement was done on an old Thinkpad X220 running Ubuntu 20.04 LTS and the latest Firefox Nightly build, since about:support shows us you're running Debian.

You're right regarding the limitations of Zoom Web, only one video stream, but I wonder if audio streams (cheaper to decode) can pile up, or something else is going on.

It's unclear that we can do anything here indeed, but it's good to have measurements at least to know where we stand.

Comment 10

[Paul Adenot (:padenot)]

(In reply to Paul Adenot (:padenot) from comment #6)

[...] and then shows that Zoom uses lots of memory, [...] Apparently the zoom web app is leaking lots of buffers that come from the buffer backing a canvas.

False alarm, I think. It's just that the profiler buffer counts itself in the memory it's measuring, weirdly. I had it set to a 1GB buffer, but it was using 4.7GB.

Comment 11

[Nils Ohlmeier [:drno]]

As Paul pointed out already as Zoom implements video transport and decoding in JS it is basically up to them to decide to skip over late content or play late content at a higher speed. That they prefer to play at higher speed might be an artifact from Zoom's old web client used TCP for media transport and therefore could not skip over missing content.

And yes you are right that the Zoom web client is limited to showing only the video of the active speaker for performance reasons AFAIK.

As it looks like this is not actionable from our side I'll close for now. Feel free to re-open this bug any time if new information points at Firefox problems.