1925024 - GMPVideoEncoder/Decoder shmem handling is bad

Status: RESOLVED FIXED

(Core :: WebRTC: Audio/Video, defect, P2)

Description

[Andreas Pehrson [:pehrsons]]

GMP's shmem handling has appeared to be a problem with the work in bug 1917191 and bug 1920687 to skip processing desktop capture frames without content updates and to use libwebrtc's zero hertz encoding to achieve good perceived video quality on the remote side when capture isn't generating new frames to encode.

While not super common it does show as a new test_peerConnection_basicScreenshare.html failure mode. When this fails we the remote side of the peerConnection in the test has not gotten any receive stats for 30 seconds. In the log there is both a problem with the GMP encoder dropping frames because it has no shmems:

E/GMP GMPVideoEncoderParent::Encode: Out of mem buffers. Frame Buffers:123 Max:120, Encoded Buffers: 29 Max: 40

and with libwebrtc's decoder integration dropping decoded frames because they're too old:

D/GMP GMP Decoded: 2162524988
W/webrtc_trace (generic_decoder.cc:125): Too many frames backed up in the decoder, dropping frame with timestamp 194627249

A pernosco recording I took locally doesn't cause a test failure but it does show the Out of mem buffers error several times. I did break down in that recording what GMP is up to that leads to these errors. I can summarize one particular case from the recording like this:

• GMPVideoEncoderParent::Encode -> 3x Alloc -> 3x PGMPContentParent::ShmemCreated + PGMPVideoEncoderParent::SendEncode (repeat ~15x=60 msgs)
• GMPVideoEncoderParent hits limit of frame buffers, stops sending messages
• GMPVideoEncoderChild sees a ~2.3s stall before receiving the next RecvEncode message
• During this stall the GMP process processes all the PGMPContentChild::ShmemCreated messages sent due to GMPVideoEncoderParent::Encode's Allocs (see first line).

Root problem here is that the send side sends (E = Encode, SC = ShmemCreated) [SC1.1, SC1.2, SC1.2, E1, SC2.1, SC2.2, SC2.3, E2] but the receive side processes them in the order of [SC1.1, SC1.2, SC1.3, SC2.1, SC2.2, SC2.3, E1, E2] which for a large number of messages causes significant delay!

We could also be a bit more proactive in the webrtc GMP codecs to drop frames before processing, to reduce the risk of reaching the shmem limit, overall load on GMP, and to avoid libwebrtc dropping decoded frames after they were decoded (throwing away real work).

Comment 1

[Andrew Osmond [:aosmond] (he/him)]

Attachment: Bug 1925024 - Part 2. Remove shmem usage from GMPVideoPlaneImpl.

Now that we don't use GMPVideoPlaneImpl inside of GMPVideoi420FrameImpl,
there are no uses of GMPVideoPlaneImpl inside of our GMP implementation.
It is exposed as part of the GMP API, so we must continue to supply it,
but there is no need for it to be IPDL aware/performant.

Comment 2

[Phabricator Automation]

Comment on attachment 9432109 [details]
Bug 1925024 - Part 2. Remove shmem usage from GMPVideoPlaneImpl.

Revision D226258 was moved to bug 1925808. Setting attachment 9432109 [details] to obsolete.

Comment 3

[Andrew Osmond [:aosmond] (he/him)]

Attachment: Bug 1925024 - Rework shmem management with GMP video encoder/decoder.

This patch makes it so that we pass along the necessary buffers as we go
along with encoding/decoding. This should increase shmem reuse, avoid
sync IPDL calls and reduce the overall complexity for buffer management.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/95d976ef43af
Rework shmem management with GMP video encoder/decoder. r=ipc-reviewers,media-playback-reviewers,nika,padenot

Comment 5

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/95d976ef43af