2071520 - WebGPU: Color space conversion is entirely skipped for single-plane external textures

Status: NEW

(Core :: Graphics: WebGPU, defect, P3)

Description

[Andy Leiserson [:aleiserson]]

Naga calls write_convert_yuv_to_rgb_and_return when the external texture is 2 or 3 planes, but not when it is a single plane.

As I understand it there are at least four things that can actually be happening within "convert yuv to rgb":

• YUV to RGB conversion (obviously), but this part doesn't matter because our single-plane formats are currently all RGB
• Channel permutation. Currently Firefox incorrectly assumes GBR channel order (claude claims; I haven't verified at all) and naga ignores the permutation. So we're correct if the source is RGB order, otherwise wrong (and fixing one side but not the other would break differently)
• Gamut conversion. Matters if the application wants display-P3 output. We also don't support display-p3 canvas colorspace, so probably not immediately important, but it's possible an application would transfer display-p3 data somewhere besides a visible canvas.
• Transfer function. Firefox passes a Rec709 -> sRGB conversion, which is incorrect. Naga ignores it, so it works out okay, but like the channel permutation is a possible source of future problems.

Everything below here is Claude analysis, it seems plausible but I'm not a color space expert and I haven't cross-checked with the code.

What "single plane" means in Firefox

num_planes == 1 is exactly ExternalTextureFormat::Rgba, and in Firefox that is produced only by MapFormat in dom/webgpu/ExternalTexture.cpp for B8G8R8A8/B8G8R8X8/R8G8B8A8/R8G8B8X8. So a single-plane external texture is always one interleaved RGB(A) plane — never a packed YUV format like YUY2/UYVY/AYUV, because those surface formats aren't handled at all (MOZ_CRASH("Unexpected SurfaceFormat")).

Possible colour spaces

Destination is whatever GPUExternalTextureDescriptor.colorSpace says: "srgb" or "display-p3" (dom/webidl/WebGPU.webidl). Both reach the single-plane path — ExternalTextureCache even keys its cache on the colour space.

Source is ExternalTextureSourceHost::mColorSpace, a gfx::YUVRangedColorSpace, set differently per construction path:

path	single-plane source colour space
CreateFromBufferDesc, RGBDescriptor	hardcoded GbrIdentity, with // TODO: support HLG and PQ (bug 2024870) — the source's real primaries/transfer are discarded
CreateFromMacIOSurfaceTextureHost	ToYUVRangedColorSpace(ioSurface->GetYUVColorSpace(), ...); MacIOSurface::mColorSpace defaults to Identity, and the only setter (AppleVTDecoder) only ever tags biplanar NV12/P010 buffers → GbrIdentity in practice
CreateFromDXGITextureHost	ToYUVColorSpace(descriptor.colorSpace()); the only BGRA producers are SharedSurface_ANGLEShareHandle / SharedSurface_D3D11Interop (WebGL, tagged Display/SRGB/DISPLAY_P3) and dom/webgpu/SharedTextureD3D11.cpp (hardcoded SRGB). The zero-copy video path asserts NV12/P010/P016, so it never lands here → GbrIdentity in practice

Verdict on naga's assumption

Skipping the YUV matrix is safe — and load-bearing. Nothing in Firefox today can hand a single-plane texture a real YUV colour space, and worse, the matrix Firefox does send for GbrIdentity is built for a G/B/R plane order. srcRgbTfFromSrc works out to R = Y + 2(Cr-0.5), G = Y, B = Y + 2(Cb-0.5), so feeding it plane0.x as Y and plane0.yz as CbCr would produce garbage. If you ever make naga apply the matrix unconditionally, ExternalTexture.cpp must be fixed to emit identity for Rgba at the same time.

But naga skips more than the YUV matrix, and that part is not valid. write_wrapped_image_load_function / write_wrapped_image_sample_function (back/hlsl/help.rs:365, :483; back/msl/writer.rs:6408, :6515) return plane0 raw, bypassing gamut_conversion_matrix, src_tf and dst_tf as well. ExternalTextureParams in wgpu-core/src/device/resource.rs:98 only documents yuv_conversion_matrix as ignored for one plane, and Firefox populates the other three non-trivially:

• GbrIdentity hardcodes TransferFunctionDesc::Rec709() (and, unlike every other branch, ignores gfx.color_management.rec709_gamma_as_srgb), while the destination gets TransferFunctionDesc::Srgb(). ColorspaceTransform::Create always sets srcTf/dstTf, so single-plane always ships a non-identity gamma round-trip that naga drops.
• For colorSpace: "display-p3", Chromaticities::DisplayP3() != Rec709(), so dstRgbLinFromSrcRgbLin is a genuine gamut matrix — also dropped. This is a real, user-visible bug: importExternalTexture({source: someRgbVideoFrame, colorSpace: 'display-p3'}) returns unconverted sRGB-gamut values. (For "srgb" the gamut matrix is identity, since Chromaticities::Srgb() == Rec709().)

So the two sides disagree about who owns the conversion. My read is that naga's pass-through is the more defensible behaviour for the transfer function — an RGBA SourceSurfaceImage really is sRGB-encoded, and Firefox declaring it Rec.709 would wrongly shift gamma if applied — which means the GbrIdentity branch should use TransferFunctionDesc::Srgb() rather than Rec709(). The gamut matrix is the opposite: naga has to stop short-circuiting it, because no descriptor fix can make sRGB→Display-P3 an identity.

Two latent hazards worth noting while you're in here:

• ToYUVColorSpace (gfx/2d/Types.h:743) MOZ_CRASHes on DISPLAY_P3 and on UNKNOWN/Display (they're the same enumerator). The DXGI single-plane path feeds it surface metadata directly, and WebGL surfaces are tagged Display by default. It's unreachable today only because VideoFrame-from-canvas and canvas capture both snapshot to SourceSurfaceImage.
• ToYUVRangedColorSpace asserts range == ColorRange::FULL for Identity, so a limited-range-tagged RGBA surface would assert in debug builds.