Closed Bug 1493898 Opened 6 years ago Closed 5 years ago

Add support for BT2020/2100 color space

Categories

(Core :: Graphics: Color Management, enhancement, P3)

Product:
Core
Component:
Graphics: Color Management
Type:
enhancement

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla68
Tracking Flags:
Tracking Status
firefox68 --- fixed

People

(Reporter: jya, Assigned: jya)

References

(Blocks 1 open bug,
URL
)

Details

Attachments

(6 files, 2 obsolete files)

Bug 1493898 - P1. Add preliminary BT2020 colorspace support. r?mattwoodrow
47 bytes, text/x-phabricator-request
Details | Review
Bug 1493898 - P2. Add BT2020 support to decoders. r?bryce
47 bytes, text/x-phabricator-request
Details | Review
Bug 1493898 - P3. Don't downsample 10/12 bits AOM images. r?bryce
47 bytes, text/x-phabricator-request
Details | Review
Bug 1493898 - P4. Add BT2020 YUV->RGB conversion. r?mattwoodrow
47 bytes, text/x-phabricator-request
Details | Review
Bug 1493898 - P5. Add Colorspace and YUV range data to VideoInfo. r?bryce
47 bytes, text/x-phabricator-request
Details | Review
Bug 1493898 - P6. Move YUVColorSpace definition in the gfx namespace. r?mattwoodrow.
47 bytes, text/x-phabricator-request
Details | Review 
HDR 10/12 bits image are typically used with BT.2100 colorspace (identical to the one in BT.2020)

We should support that in addition to BT.601 and BT.709
Blocks: wr-video-color
just a clarification, bt2100 is exactly the same as bt2020 plus of the HDR curves.

It would be nice to support plain bt2020 NCL as well, since it should be (mostly) a matter of coefficients and primaries (plain bt2020 curve is the same as 709). I am not sure if bt2020 CL is worth supporting but it would require a more invasive change.
Priority: -- → P3
Assignee: nobody → jyavenard

Vittorio, in your conversion tables, for video range YUV, you have plain coefficients, while most literature I find has something like:

G = Y - 0.34414 (Cb-128) - 0.71414 (Cr-128)

How could this translate into one another?

Flags: needinfo?(vitto.giova)

that formula computes the green value for full (aka pc, not video) range for bt601

The bt.* specifications always point out the full range coeffients for yuv->rgb, and leave the rest is left as an exercise to the reader. For converting full to limited (aka video) range, you need to rescale the coefficient matrix depending on the number of levels available for the bit depth of the pixel representation (8 bit or 10 or 12 and so on).

Finally you invert the matrix obtained for rgb->yuv and use the scaled coefficients with an updated offset for the ycbcr elements. So, in 8bit bt601 case the limited/video range green would be computed as:

G = 1.164383562 * (Y + 16) - 0.3917622901 * (Cb - 120) - 0.8129676472 * (Cr - 120)

Flags: needinfo?(vitto.giova)

(In reply to Vittorio Giovara from comment #3)

Finally you invert the matrix obtained for rgb->yuv and use the scaled coefficients with an updated offset for the ycbcr elements. So, in 8bit bt601 case the limited/video range green would be computed as:

G = 1.164383562 * (Y + 16) - 0.3917622901 * (Cb - 120) - 0.8129676472 * (Cr - 120)

But I would assume that the limited YUV range (16-235 for Y, and 16-240 for CbCr) gets converted into the full RGB range: [0-255] (or [0-1] if float)

In which case we also need to normalize that output too

Which ends up giving us:
G = 1.164383562 * (Y + 16) * 255/235 - 0.3917622901 * (Cb - 120) * 255/240 - 0.8129676472 * (Cr - 120) * 255/240

Is this assumption correct?

Also, from other sites (like wikipedia), are you sure it's not: Y-16 instead of Y+16?
G = 1.164383562 * (Y - 16) - 0.3917622901 * (Cb - 120) - 0.8129676472 * (Cr - 120)

Flags: needinfo?(vitto.giova)

(In reply to Jean-Yves Avenard [:jya] from comment #4)

(In reply to Vittorio Giovara from comment #3)

Finally you invert the matrix obtained for rgb->yuv and use the scaled coefficients with an updated offset for the ycbcr elements. So, in 8bit bt601 case the limited/video range green would be computed as:

G = 1.164383562 * (Y + 16) - 0.3917622901 * (Cb - 120) - 0.8129676472 * (Cr - 120)

But I would assume that the limited YUV range (16-235 for Y, and 16-240 for CbCr) gets converted into the full RGB range: [0-255] (or [0-1] if float)

RGB is always considered full range, the different coefficients and the rescaling of the terms take this into consideration.

In which case we also need to normalize that output too

Which ends up giving us:
G = 1.164383562 * (Y + 16) * 255/235 - 0.3917622901 * (Cb - 120) * 255/240 - 0.8129676472 * (Cr - 120) * 255/240

Is this assumption correct?

No, the formula I provided converts limited YUV to full range RGB, the formula you wrote converts limited YUV to something incorrect (similar to limited RGB, it should not be used). The coefficients (1.164383562, 0.3917622901, 0.8129676472) are already rescaled, they don't need a further 255/240 step.

Also, from other sites (like wikipedia), are you sure it's not: Y-16 instead of Y+16?
G = 1.164383562 * (Y - 16) - 0.3917622901 * (Cb - 120) - 0.8129676472 * (Cr - 120)

Yes sorry it's (Y-16), the other one is when you convert from RGB to YUV.

Flags: needinfo?(vitto.giova)

This is the tool I made for working with these:
https://jdashg.github.io/misc/colors/from-coeffs.html

I believe it does all rescaling properly except if "full" needs to be restricted to "Quantization level assignment" video data range [1,254], to reserve 0 and 255 for timing data.

If so, I can update that calculator, but I believe it is otherwise correct. PRs welcome, of course. :)

FWIW, these are equivalent:

G = 1.16438*(Y-16) - 0.39176*(U-128) - 0.81297*(V-128)
  = 1.16438*Y-18.63008 - 0.39176*U+50.14528 - 0.81297*V+104.06016
  = 1.16438*Y-18.63008 - 0.39176*U+50.14528 - 0.81297*V+104.06016
  = 1.16438*Y - 0.39176*U - 0.81297*V + 135.57536
G = 1.16438*Y - 0.39176*U - 0.81297*V + 0.531668*255

(In reply to Vittorio Giovara from comment #5)

Also, from other sites (like wikipedia), are you sure it's not: Y-16 instead of Y+16?
G = 1.164383562 * (Y - 16) - 0.3917622901 * (Cb - 120) - 0.8129676472 * (Cr - 120)

Yes sorry it's (Y-16), the other one is when you convert from RGB to YUV.

Also, the CbCr limited range is [16-240], so 128 +/- 112.

So we're back to:
G = 1.164383562 * (Y - 16) - 0.3917622901 * (Cb - 128) - 0.8129676472 * (Cr - 128) as Jeff used in his calculation.
(that's for rec601)

(In reply to Jeff Gilbert [:jgilbert] from comment #6)

This is the tool I made for working with these:
https://jdashg.github.io/misc/colors/from-coeffs.html

I can reach the same values as this tool calculate for limited range.
But not for full range.

Example:
Y=100, U=100, V=100.

The coefficient for rec601, full range are:
1 0 1.402
1 -0.3441362862 -0.7141362862
1 1.772 0

Which give:
R = 1Y + 1.402(V-128) = 60.744
G = 1Y-0.344136286(U-128)-0.7141362862*(V-128) = 129.63
B = 1Y+1.772(U-128) = 50.384

your matrix gives 61.445, 129.1025, 51.27
it's about off by 1 for all values

I believe it does all rescaling properly except if "full" needs to be restricted to "Quantization level assignment" video data range [1,254], to reserve 0 and 255 for timing data.

I don't think it matters nor should we want that.

I updated my calculator to use CbCr: [1,255] (0.0 at 128) for full-range, which I think was the only outstanding issue.

jya pointed out to me that rec2100 does specify quantization of full-range:

Y: D = Round((2^n-1) * E) | E = [0.0, 1.0]
Cb,Cr: D = Round((2^n-1) * E + 2^(n-1)) | E = [-0.5, 0.5]

I, uh, really don't like that, but they're the boss(es).

I was previously using Dcb,Dcr = Round(2^n*E + 2^(n-1)), which is 128 +/- 127, whereas the spec says it's 128 +/- 127.5, which is bizarre, since Ecb=(Dcb-128)/255, min=-127/255, max=127/255, thus it's impossible to dequantize Ecb to -0.5 or 0.5! (unless you include a rounding step on dequantization??)

(In reply to Jeff Gilbert [:jgilbert] from comment #12)

jya pointed out to me that rec2100 does specify quantization of full-range:

Y: D = Round((2^n-1) * E) | E = [0.0, 1.0]
Cb,Cr: D = Round((2^n-1) * E + 2^(n-1)) | E = [-0.5, 0.5]

I, uh, really don't like that, but they're the boss(es).

I was previously using Dcb,Dcr = Round(2^n*E + 2^(n-1)), which is 128 +/- 127, whereas the spec says it's 128 +/- 127.5, which is bizarre, since Ecb=(Dcb-128)/255, min=-127/255, max=127/255, thus it's impossible to dequantize Ecb to -0.5 or 0.5! (unless you include a rounding step on dequantization??)

NB: 2100 specifies quantization of full range only for 10 bit, 8 bit is still left to the implementation, of course it makes sense to apply the same logic for 8 bit too. At least for the 8bit case, I think the quantization error introduced in the conversion is negligible.

btw your tool is great, I wish I had it around when I was studying this topic

(In reply to Jean-Yves Avenard [:jya] from comment #8)

Also, the CbCr limited range is [16-240], so 128 +/- 112.

Correct, assuming symmetric, I forgot about that, your formula is right.

That's really sad, but I updated my calculator and now rec601 full-range YCbCr(100, 100, 100) converts to RGB(60.744, 129.63163, 50.384).

Bad spec is bad.

I'm glad it's been helpful!

Only active with webrender and AL.

All compositors support 10/12 bits images now.
Additionally, add BT2020 support to AOM decoder.

Depends on D25343

This is used by the basic compositor.
Re-using existing logic, however as with other conversion it only handles limited 8 bits ranges (16-235) and to make things worse is rounded aggressively as the focus is on speed.

Depends on D25344

And read the info from the VP9 bytestream.

Depends on D25345

YUVColorSpace is inseparable from the bit depth as the matrix coefficients to be calculated need the bit depth information.

So let's put the two types together. gfx namespace also makes more sense as that's where we find IntRect, IntSize and other.

The extent of the changes highlight how much similar data structures are duplicated across the code, to the point it's scary.

Depends on D25346

And read the info from the VP9 bytestream.

YUVColorSpace is inseparable from the bit depth as the matrix coefficients to be calculated need the bit depth information.

So let's put the two types together. gfx namespace also makes more sense as that's where we find IntRect, IntSize and other.

The extent of the changes highlight how much similar data structures are duplicated across the code, to the point it's scary.

Depends on D25369

Attachment #9054439 - Attachment is obsolete: true
Attachment #9054438 - Attachment is obsolete: true
Pushed by jyavenard@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/529a9a8efae1
P1. Add preliminary BT2020 colorspace support. r=mattwoodrow
https://hg.mozilla.org/integration/autoland/rev/a0f4ced9b57c
P2. Add BT2020 support to decoders. r=bryce
https://hg.mozilla.org/integration/autoland/rev/a6a750656333
P3. Don't downsample 10/12 bits AOM images. r=bryce
https://hg.mozilla.org/integration/autoland/rev/629f949df541
P4. Add BT2020 YUV->RGB conversion. r=jgilbert
https://hg.mozilla.org/integration/autoland/rev/81d979cbbca2
P5. Add Colorspace and YUV range data to VideoInfo. r=bryce
https://hg.mozilla.org/integration/autoland/rev/f2eda1d5abb8
P6. Move YUVColorSpace definition in the gfx namespace. r=mattwoodrow.
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: