Investigate if we are burning CPU cycles when appending null or same frames to MediaStream::AppendToTrack

NEW
Unassigned

Status

()

Core
WebRTC: Audio/Video
P5
normal
Rank:
59
3 years ago
3 years ago

People

(Reporter: pehrsons, Unassigned)

Tracking

Trunk
Points:
---

Firefox Tracking Flags

(firefox40 affected)

Details

(Reporter)

Description

3 years ago
See bug 1032848 comment 60.

jesup wrote:
> I think someone should double-check sometime that we're not wasting cycles here due to lack of optimizations for the same-image or null-image cases in VideoSegment/MediaSegment/etc (also when forcing black).
(Reporter)

Updated

3 years ago
Depends on: 1032848
(Reporter)

Comment 1

3 years ago
This was created on a misconception (I had a bug in an old patch revision, appending null frames instead of same-as-previous-frames). Still we could probably clarify the inline comments of MediaStream::AppendToTrack as part of this bug.
backlog: --- → webRTC+
Rank: 59
Priority: -- → P5
(Reporter)

Comment 2

3 years ago
I looked into this a bit. AppendToTrack() does two things:

* Adds all chunks in |aSegment| to the track in question.
* Notifies direct listeners through |NotifyRealtimeData|.


It's up to each consumer to sanity check what's in the track (null or same frame). Some that I looked at:

* Local video element (See MediaStreamGraphImpl::PlayVideo):
Exits early if it sees the same frame twice.
>   if (!frame || *frame == aStream->mLastPlayedVideoFrame)
>     return;

* WebRTC (See MediaPipelineTransmit::PipelineListener::NewData):
Exits early if it sees the same frame twice.
>   // We get passed duplicate frames every ~10ms even if there's no frame change!
>   int32_t serial = img->GetSerial();
>   if (serial == last_img_) {
>     return;
>   }

* MediaRecorder (See VP8TrackEncoder::PrepareRawFrame):
Checks for null but not same-frame. Does some plane conversions on each frame. (also only handles ImageFormat::PLANAR_YCBCR :/)
backlog: webRTC+ → ---
Priority: P5 → --
(Reporter)

Updated

3 years ago
Priority: -- → P5

Updated

3 years ago
backlog: --- → webRTC+
You need to log in before you can comment on or make changes to this bug.