2034249 - Support HDR transfer function and metadata in PlanarYCbCrImage for software-decoded video

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, task, P2)

Description

[Alastor Wu [:alwu]]

Software-decoded video produces PlanarYCbCrImage frames without HDR transfer functions or metadata. As a result, the WebRender shader has no way to apply the correct EOTF, causing all software-decoded HDR content to be rendered as SDR, regardless of whether the source is PQ or HLG.

In addition, this bug won't make software-decoded HDR work immediately, there are more works to do.

Comment 1

[Alastor Wu [:alwu]]

Attachment: Bug 2034249 - Part 2: Propagate transfer function and HDR metadata in FFmpeg shmem buffer path.

FFmpegVideoDecoder::AllocateTextureClientForImage builds PlanarYCbCrData
directly and allocates a shmem image buffer for FFmpeg to decode into.
Because this path uses VideoData::CreateFromImage rather than
CreateAndCopyData, it bypasses SetVideoDataToImage and the transfer
function and HDR metadata from mInfo were never written to the image.

Set mTransferFunction and mHDRMetadata from mInfo at the point where
the PlanarYCbCrData is constructed, so the shmem-backed image carries
correct color metadata from the outset.

Comment 2

[Alastor Wu [:alwu]]

Attachment: Bug 2034249 - Part 3: Propagate transfer function and HDR metadata in AppleVTDecoder.

AppleVTDecoder stores color metadata (mColorSpace, mColorPrimaries,
mTransferFunction) as class members initialized from VideoInfo at
construction time. The local VideoInfo built for CreateAndCopyData in
the SW fallback path only had mDisplay set, so mTransferFunction and
mHDRMetadata were not propagated to PlanarYCbCrData.

Add mHDRMetadata as a class member (initialized from aConfig.mHDRMetadata),
and populate both info.mTransferFunction and info.mHDRMetadata on the local
VideoInfo before passing it to CreateAndCopyData.

Comment 3

[Alastor Wu [:alwu]]

Attachment: Bug 2034249 - Part 1: Add mHDRMetadata to PlanarYCbCrData and wire through YCbCrDescriptor IPC.

Add Maybe<gfx::HDRMetadata> mHDRMetadata to PlanarYCbCrData and the
corresponding HDRMetadata? hdrMetadata field to the YCbCrDescriptor IPC
struct. Wire it through SetVideoDataToImage (copying from VideoInfo),
CreateForYCbCr, and the SharedPlanarYCbCrImage path in ImageContainer.

This is the media-side plumbing prerequisite that allows HDR metadata
to reach WebRender for software-decoded video paths.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/cb4af0018bc8
https://hg.mozilla.org/integration/autoland/rev/53599349fe0c
Part 1: Add mHDRMetadata to PlanarYCbCrData and wire through YCbCrDescriptor IPC. r=ahale,padenot
https://github.com/mozilla-firefox/firefox/commit/a36f94977348
https://hg.mozilla.org/integration/autoland/rev/536151d34d16
Part 2: Propagate transfer function and HDR metadata in FFmpeg shmem buffer path. r=media-playback-reviewers,padenot
https://github.com/mozilla-firefox/firefox/commit/b6aa43294202
https://hg.mozilla.org/integration/autoland/rev/d25e33f3e70f
Part 3: Propagate transfer function and HDR metadata in AppleVTDecoder. r=media-playback-reviewers,padenot

Comment 5

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/53599349fe0c
https://hg.mozilla.org/mozilla-central/rev/536151d34d16
https://hg.mozilla.org/mozilla-central/rev/d25e33f3e70f