Mac screen capture: Capture frames in sRGB color space
Categories
(Core :: WebRTC: Audio/Video, task, P3)
Tracking
()
People
(Reporter: pehrsons, Assigned: pehrsons)
References
Details
The libwebrtc screen capturer currently captures frames in the display colorspace, see CGDisplayStreamCreate properties and its kCGDisplayStreamColorSpace key.
This is why we've had issues with test_getUserMedia_basicScreenshare.html.
Outlined:
- Screen is captured in BGRA format (equivalent to libyuv's ARGB) in display colorspace, for instance Apple's Display P3.
- We convert with libyuv to I420 where I420 is in the BT.601 limited range colorspace and I assume the ARGB is in the sRGB colorspace, libyuv doesn't mention this.
- We pack this I420 frame in a proper gfx::Image format and pass it through gecko.
For proper colorspace handling screen_capturer_mac.mm needs to set up the display stream to capture frames in the sRGB colorspace. At least that is the simplest change that would give us accurate colors.
|
Assignee | |
Comment 1•1 year ago
|
||
This needs to be fixed upstream.
|
|
Assignee | |
Updated•1 year ago
|
|
|
Assignee | |
Comment 2•1 year ago
|
||
Filed upstream: https://bugs.chromium.org/p/webrtc/issues/detail?id=15116
Patch upstream: https://webrtc-review.googlesource.com/c/src/+/302960
I'll amend a bit though: currently upstream has a bug where icc profile data doesn't propagate for screen capture. with that fixed we could handle the frame being in the display colorspace on our end, though I imagine performance is better if get the frames directly in sRGB and avoid the conversion on our end.
This applies to screen capture. The same problem exists for window capture, where there is no flag to pass to the API to specify color space. Window capture frames are CGImages which we don't seem to have a layers::Image type for. For screen capture we can choose to get IOSurfaces as well, and there we have a layers::Image type which seems to be able to handle colorspace.
If we were to migrate to ScreenCaptureKit (we'll have to, long term) we can specify colorspace in the backend for both screen and window.
Description
•