1152317 - Investigate if we are burning CPU cycles when appending null or same frames to MediaStream::AppendToTrack

Status: NEW

(Core :: WebRTC: Audio/Video, defect, P5)

Description

[Andreas Pehrson [:pehrsons]]

See bug 1032848 comment 60.

jesup wrote:
> I think someone should double-check sometime that we're not wasting cycles here due to lack of optimizations for the same-image or null-image cases in VideoSegment/MediaSegment/etc (also when forcing black).

Comment 1

[Andreas Pehrson [:pehrsons]]

This was created on a misconception (I had a bug in an old patch revision, appending null frames instead of same-as-previous-frames). Still we could probably clarify the inline comments of MediaStream::AppendToTrack as part of this bug.

Comment 2

[Andreas Pehrson [:pehrsons]]

I looked into this a bit. AppendToTrack() does two things:

* Adds all chunks in |aSegment| to the track in question.
* Notifies direct listeners through |NotifyRealtimeData|.


It's up to each consumer to sanity check what's in the track (null or same frame). Some that I looked at:

* Local video element (See MediaStreamGraphImpl::PlayVideo):
Exits early if it sees the same frame twice.
>   if (!frame || *frame == aStream->mLastPlayedVideoFrame)
>     return;

* WebRTC (See MediaPipelineTransmit::PipelineListener::NewData):
Exits early if it sees the same frame twice.
>   // We get passed duplicate frames every ~10ms even if there's no frame change!
>   int32_t serial = img->GetSerial();
>   if (serial == last_img_) {
>     return;
>   }

* MediaRecorder (See VP8TrackEncoder::PrepareRawFrame):
Checks for null but not same-frame. Does some plane conversions on each frame. (also only handles ImageFormat::PLANAR_YCBCR :/)