1539686 - Retrieve color and transfer function information from webm container

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, enhancement, P2)

Description

[Jean-Yves Avenard [:jya]]

For VP9 streams, we can only determine if the video is HDR by getting information from the container as the bytestream only contains colorspace data.

BT.2100 standardized two transfer functions:
Dolby PQ and HLG, only the presence of that information can tell us if the content is HDR

nestegg needs to provide a way to retrieve that info.

Comment 1

[Matthew Gregan [:kinetik]]

Which subelements of the Color element do we need to parse? I can add all of them, but prefer to only add what we're going to use for simplicity.

Comment 2

[Jean-Yves Avenard [:jya]]

(In reply to Matthew Gregan [:kinetik] from comment #1)

Which subelements of the Color element do we need to parse? I can add all of them, but prefer to only add what we're going to use for simplicity.

I'd say we only care about vp9 and av1 in webm. And vp9 bytestreams contains info about colorspace, bits per channel, coefficient, chroma subsampling and range.

TransferCharacteristics is the primary one.

I'm thinking we should integrate all those found in YT videos like
https://jyavenard.github.io/htmltests/mediatest/webm/hdr/The%20World%20in%20HDR.webm
As if they integrated it, surely there must be a reason for it :)

The Videolan folks stated that you can't rely on information such as MaxCLL or Luminance Info or any of the mastering display metadata as it's often wrong; so instead they scan every single frame to determine that information and generate a shader for each frame. So they rely on mastering display metadata as a fallback only.

Comment 3

[Matthew Gregan [:kinetik]]

WIP here: https://github.com/kinetiknz/nestegg/pull/62

This seems to work with the "The World In HDR.webm" and does the expected thing for a WebM without Colour/MasteringMetadata present.

nestegg_video_params has been extended to expose the new fields, so this means an ABI change for libnestegg. Another option is to expose these via a new getter function to avoid ABI changes, but I don't think it's worth it.

I considered exposing the x/y pairs as a 2 element array, but I'm not sure the end result is clearer.

The direct subelements of Colour all have sensible defaults from the Matroska spec. MasteringMetadata and its subelements don't, so any elements that aren't present result in the corresponding field in nestegg_video_params being initialized to NaN.

Comment 4

[Matthew Gregan [:kinetik]]

PR62 updated with tests and ready for review.

Comment 5

[Matthew Gregan [:kinetik]]

Attachment: GitHub Pull Request

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/3648f6355f7d
Update libnestegg to 90521e7f. r=bryce

Comment 7

[Alexandru Michis [:malexandru]]

https://hg.mozilla.org/mozilla-central/rev/3648f6355f7d