Only process a DesktopFrame's dirty region
Categories
(Core :: WebRTC: Audio/Video, task)
Tracking
()
People
(Reporter: pehrsons, Unassigned)
References
(Depends on 1 open bug)
Details
With bug 1917191 we are skipping all DesktopFrames that are duplicates of the previous frame. Some backends will tell us in more detail exactly which regions had changed in a frame update. We could use this information to only process the dirty region when a frame is not a duplicate.
Screen capture frames can be very large and the most costly processing up front is a conversion from ARGB to I420. If we keep a copy of the previous frame we would be able to save some cpu time by only converting the dirty parts. This is low-hanging fruit.
Higher-hanging fruit would be to propagate this information through IPC to clients. We could for instance keep the shmem buffer and DeliverFrame protocol as they are today, but tag along updated_region info and only copy the updated region in and out of the shmem, to avoid reallocations. The downscaling that happens in clients could then also apply only to the updated region.
Like bug 1917191 this wouldn't help at all with worst-case scenarios (think capturing a screen of a full-screen 60fps video). But it would lower system load in most other scenarios.
Description
•