1920687 - Enable libwebrtc's ZeroHertzAdapterMode in FrameCadenceAdapter

Status: ASSIGNED

(Core :: WebRTC: Audio/Video, task)

Description

[Andreas Pehrson [:pehrsons]]

Without ZeroHertzAdapterMode we'll use PassthroughAdapterMode which does what it sounds like -- lets frames through 1 by 1.

With ZeroHertzAdapterMode on the other hand, which is only used for desktop capture, libwebrtc assumes the capture framerate is dynamic based on whether what is visible on screen has changed. When the capture framerate is lower than configured, for instance zero for some period of time, the ZeroHertzAdapterMode schedules re-encodings of the latest frame to ensure the receiving end receives something of good visual quality.

Imagine a bandwidth-constrained scenario. On the capture side the user changes slide. That's a lot of new information to encode. Bandwidth is low so the first encoded frame is of low visual quality. But no more frames are captured because the slide is static on the screen. This is where the ZeroHertzAdapterMode re-encodes the latest frame several times so that the visual quality gradually improves. It seems to use a heuristic based on the codec's quantization parameter to decide whether quality is good or not.

This is a prerequisite of bug 1917191.

Comment 1

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1920687 - Signal constraints changes through MediaStreamTrackSource sinks. r?#webrtc-reviewers!

Comment 2

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1920687 - Clean up MediaTrackConstraints includes. r?jib!

Comment 3

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1920687 - Mark MediaTrackConstraints' Range child classes final for good measure. r?jib!

Comment 4

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1920687 - Remove unused NormalizedConstraints::mBadConstraint. r?jib!

Comment 5

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1920687 - Remove unused aMemberPtr and aList args in Normalized/FlattenedConstraints. r?jib!

Comment 6

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1920687 - Fix some warnings in MediaTrackConstraints. r?jib!

Comment 7

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1920687 - To NormalizedConstraints and FlattenedConstraints, add default ctors. r?jib!

Comment 8

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1920687 - To Normalized/FlattenedConstraints add equality operators. r?jib!

This allows these classes to be used in StateMirroring.

Comment 9

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1920687 - In RTCRtpSender make the send track's constraints a canonical. r?#webrtc-reviewers!

Comment 10

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1920687 - Hook up send track fps constraints to video send stream sink. r?#webrtc-reviewers!

Passing frameRate:min == 0 and frameRate:max > 0 to the send stream sink enables
the ZeroHertz FrameCadenceAdapter mode if we're sending screen content.

Comment 11

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1920687 - Fix some includes and warnings. r?#webrtc-reviewers!

Comment 12

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1920687 - In VideoConduit trigger the watch functions on the initial state. r?#webrtc-reviewers!

Without this we can't trigger ZeroHertzAdapterMode unless some constraints were
explicitly requested.