1634741 - AVIF (AV1 Image File Format): proper color space support

Status: RESOLVED FIXED

(Core :: ImageLib, enhancement, P2)

Description

[Jon Bauman [:jbauman:]]

Add support for reading the ICC profile and providing the data to the graphics surface.

Edit: also consider the CICP/NCLX values. In summary, properly support the ColourInformation (colr) box from ISOBMFF (ISO 14496-12:2020) § 12.1.5.

In particular, see https://github.com/AOMediaCodec/av1-avif/issues/84

Comment 1

[chris]

Since AVIF already has information about the primaries, transfer function etc then it is possible, indeed likely, that a given AVIF will not contain an ICC profile (but will be fully specified in terms of colorimetry). So it would be inappropriate, for example, so fall back to sRGB in the absence of an explicit ICC profile.

Also, if the info in the ICC profile conflicts with that in the AVIF metadata, which has precedence?

In other words, please broaden this bug slightly to "displaying AVIF with the correct colors" which may be from ICC but may also not be.

Comment 2

[Jon Bauman [:jbauman:]]

Fair point, chris. I've updated the title of the bug

Comment 3

[martin]

Hi. I've noticed colours are more saturated and there's more contrast with AVIF images in FF. Is that something already covered by this bug?

Comment 4

[Jon Bauman [:jbauman:]]

Maybe. It depends whether the images are using a color space other than sRGB. If you have specific examples, that would help.

Comment 5

[Nick Doyle]

Attachment: SpeedyAndroid.avif

I'm also running in to oversaturation in Firefox, I'm attaching some images for comparison.

Comment 6

[Nick Doyle]

Attachment: SpeedyAndroid.bmp

Comment 7

[Nick Doyle]

Attachment: SpeedyAndroidScreenshot.png

Comment 8

[Nick Doyle]

I expect this oversaturation happens because my display is P3 and this isn't being taken in to account when displaying the image. It happens for me in both macOS and Linux X11.

Comment 9

[chris]

(In reply to Jon Bauman [:jbauman:] from comment #4)

Maybe. It depends whether the images are using a color space other than sRGB. If you have specific examples, that would help.

Or whether the display is wider gamut than sRGB. For example, regular sRGB Web content is oversaturated in Firefox on my laptop (Dell XPS 9650, Windows 10) which has a calibrated P3 screen, compared to Chrome or new-chromium-Edge. The display color profile is ignored and the raw RGB data is simply sent to the screen, producing the wrong colors.

I have observed the same on a 2016 Macbook, in Firefox compared to Chrome or Safari.

It seems natural that this will affect AVIF display as well. Which likely means it will be fixed when Firefox switches to doing color management for all Web content, instead of color management for ICC-tagged images only.

Comment 10

[Nick Doyle]

Chris is correct here. I didn't notice this because all of my image processing includes a shouldn't-be-necessary ICC profile on JPEG and WebP images to work around this bug. My new AVIF processing didn't include this ICC profile because all of the other major browsers fixed this years ago and I assumed this bug was long fixed in Firefox too.

Firefox is an outlier in this behaviour. Ironically, Firefox has had the ability to handle correct colour for a long time but just hasn't set the correct default.

gfx.color_management.mode needs to be defaulted to 1 for accurate colour but is currently defaulted to 2. Considering almost all new displays now are wider than sRGB, the current default for this parameter means that Firefox has dramatically inaccurate colour in most cases.

Unfortunately, muxing in an ICC profile in to an AVIF file is a lot more involved than it is in JPEG and WebP because of the nature of the ISOBMFF/HEIF container and the way it uses pointers between boxes. As a CDN image processor, this means supporting AVIF in Firefox will need to be delayed until this default is updated or until I can handle all the extra muxing to include an extra 0.5KiB noop ICC profile just to coax Firefox in to the correct colour code path.

For reference, here's an acid-like test page demonstrating the issue in general. https://kornel.ski/en/color

Comment 11

[martin]

My tests were saved from Photoshop (Save for Web, Convert to sRGB) as JPEGs then converted to AVIF using https://squoosh.app/ so I'm not sure if that strips/changes the colour profile.

But you can see the difference with the examples on this page: https://jakearchibald.com/2020/avif-has-landed/ (particularly "Illustration with gradients").

(As an aside the "Heavy SVG" example also shows FF not handling transparency with AVIF images)

Firefox 85.0.1
Firefox DE 86.0b8
Windows 10 v2004

Comment 12

[chris]

Attachment: kornel color profile test in Firefox, Chrome and Edge

Comment 13

[chris]

Testing just now on Windows 10, XPS 9560 with 4k P3 screen. Top to bottom: Firefox Nightly 86.0a1, Canary Version 90.0.4414.0 (Official Build) canary (64-bit), and chromium-based Edge (which seems to have removed both the help menu and about: so what version IDK)

Comment 14

[chris]

Firefox is the outlier in terms of inconsistency; Edge is also an outlier (consistent among subtests, but also consistently wrong I think.

Comment 15

[Yuan Tong]

(In reply to martin from comment #3)

Hi. I've noticed colours are more saturated and there's more contrast with AVIF images in FF. Is that something already covered by this bug?

(In reply to Nick Doyle from comment #8)

I expect this oversaturation happens because my display is P3 and this isn't being taken in to account when displaying the image. It happens for me in both macOS and Linux X11.

Most oversaturation cases (includes attachment 9202460 [details]) should related to bug 1654461. This is the info dumped using avifdec:

avifdec --info  SpeedyAndroid.avif
Image decoded:  SpeedyAndroid.avif
 * Resolution     : 500x513
 * Bit Depth      : 8
 * Format         : YUV420
 * Alpha          : Absent
 * Range          : Full
 * Color Primaries: 2
 * Transfer Char. : 2
 * Matrix Coeffs. : 6
 * ICC Profile    : Absent (0 bytes)
 * XMP Metadata   : Absent (0 bytes)
 * EXIF Metadata  : Absent (0 bytes)
 * Transformations: None
 * 1 timescales per second, 1.00 seconds (1 timescales), 1 frame
 * Frames:
   * Decoded frame [0] [pts 0.00 (0 timescales)] [duration 1.00 (1 timescales)]

This is a full range AVIF, but firefox decoded it as limited range. If you are using avifenc, add --range limited will produce AVIF that displays correctly in current Firefox. squoosh.app always produces full range AVIF and can't be changed.

Comment 16

[db]

I'm noting the oversaturated images on Firefox as well.

My notes on the subject:

1. I'm encoding test cases from a PNG file with no ICC profile
2. First test is with cavif which uses the rav1e encoder
3. Second test is with heifenc which uses the AOM encoder
4. Both images were displayed in Google Chrome and Firefox

Results:

1. Google chrome displays both encoded avif files correctly.
2. Firefox displays the cavif/rave1e encoded avif file correctly.
3. Firefox oversaturates the libheif/AOM encoded avif file.

I'm not too familiar with the internals, so I don't know if this is a bug in Firefox reading the AOM encoded version, or a problem with how it is encoded. However Chrome has no issue displaying it correctly. None of these files have ICC profiles.

I'll attach images showing the differences.

My test files are all here:

https://www.dropbox.com/sh/qlyv7xy7v9jxhjg/AADH2_0DMnTJny_0MV8Su7ypa?dl=0

Comment 17

[db]

Attachment: comparison of cavif/libheif created AVIF files on Firefox and Chrome

Comment 18

[db]

Attachment: Difference layer in Photoshop of incorrect rendering libheif image over correct rendering rav1e image

Comment 19

[db]

Attachment: animation of over-saturated libheif avif image alternated with correctly rendered rav1e avif image

Comment 20

[db]

Attachment: AVIF image generated by cavif/rav1e encoder ... should display correctly in Chrome and Firefox

Comment 21

[db]

Attachment: AVIF example libheif/AOM encoded, displays correctly in Chrome but oversaturated in Firefox

Comment 22

[db]

Images were generated via:

cavif \
    --quality 58 --overwrite --quiet test.png -o cavif.avif 

heif-enc \
    -A -p min-q=20 -p max-q=24 -p threads=2 test.png -o heifenc.avif

Comment 23

[db]

I've now built heif-enc using the rav1e encoder and the images from it have the same issues.

I've verified the above posted workaround for avifenc, when I use this:

avifenc \
    --min 26 --max 30 -j 2 test.png --range limited -o avifenc.avif

Firefox displays things correctly.

Comment 24

[Yuan Tong]

(In reply to db from comment #22)

Images were generated via:

cavif \
    --quality 58 --overwrite --quiet test.png -o cavif.avif 

heif-enc \
    -A -p min-q=20 -p max-q=24 -p threads=2 test.png -o heifenc.avif

Yes your test confirmed the issue is about color range.
cavif-rs hard coded a limited range formula, so Firefox displays it correctly.
libheif defaults to full range, and it seems --full_range_flag 0 is ignored, so libheif always produces full range images.

Comment 25

[Vittorio Giovara]

can you please add [platform-rel-Vimeo] tag as it's relevant to a product launch we're interested in?

Comment 26

[sh2be]

Attachment: AVIF image sample with color primaries set to BT.2020 (CICP value 9)

Firefox doesn't handle information about color space when it's signaled via CICP values (ITU-T H.273) instead of ICC profile. The attached AVIF image example has a CICP value of 9 for color primaries (BT.2020 primaries) but it's interprated as being sRGB thus colors are wrong.

Comment 27

[sh2be]

Attachment: Comparison between Firefox 89 and Chrome 91 of the rendered AVIF BT.2020 sample on sRGB display.jpg

Here is how the previous image is rendered on sRGB display by Firefox 89 compared to Chrome 91 which does it correctly.

Comment 28

[sh2be]

I'm not sure how this affects the plan to make AVIF enabled by default.
We might want to open an new bug to specifically treat the CICP handling.

Comment 29

[chris]

The bug isn't called "enable AVIF by default" it is "proper color space support". So, since most AVIF in practice use H.273 CICP values, rather than ICC profiles, to indicate the colorspace then it seems entirely appropriate here. And treating BT.2020 values as sRGB certainly counts as improper color space support.

Comment 30

[Jon Bauman [:jbauman:]]

Thanks for the report, sh2be. As chris says, this is the bug for addressing that issue, but I've updated the description to make that clearer.

(In reply to sh2be from comment #28)

I'm not sure how this affects the plan to make AVIF enabled by default.

This is one of the issues we are planning to resolve before enabling AVIF by default in the release version of Firefox. That's why it's a blocking bug for that issue. We do plan on enabling AVIF by default before implementing 100% of the feature set, so the remaining issues are tracked under the more general AVIF metabug. I hope that clarifies things.

Comment 31

[Jon Bauman [:jbauman:]]

Attachment: Bug 1634741 - AVIF (AV1 Image File Format): proper color space support. r=jrmuizel,aosmond

Comment 32

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2782c77c1950
AVIF (AV1 Image File Format): proper color space support. r=aosmond

Comment 33

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/2782c77c1950