Open Bug 1443863 (AVIF) Opened 3 years ago Updated 2 days ago

Implement support for AV1 Still Image File Format (AVIF)

Categories

(Core :: ImageLib, enhancement, P3)

enhancement

Tracking

()

ASSIGNED

People

(Reporter: Virtual, Assigned: jbauman)

References

(Depends on 11 open bugs, Blocks 1 open bug, )

Details

(Keywords: dev-doc-needed, feature, nightly-community, Whiteboard: [gfx-noted])

Attachments

(2 files)

Has Regression Range: --- → irrelevant
Has STR: --- → irrelevant
Whiteboard: [gfx-noted]
Some more: https://github.com/AOMediaCodec/av1-avif/tree/master/testFiles/Microsoft

Especially the tiled one (Summer_in_Tomsk_720p_5x4_grid.avif) is tricky.
See Also: → PIK

Chromium also opened an issue to implement AVIF.

Aside from using libaom (or dav1d) directly for encoding (or decoding) and muxing into an .avif file, libavif could also be considered. Here is an implementation in Colorist as example.

The Windows 10 May 2019 Update is now rolling out, which contains AVIF support (including File Explorer and Paint).

Hello! I'm the author of libavif, and similarly to the Chromium issue cited above, I thought I'd jump in here and see if there was anything I could do to help with supporting this potential feature. Chromium uses libdav1d already (supported by libavif) for decoding and (now) has a pipeline to draw HDR YUV+A, but I don't have have any idea what Firefox's pipeline and library usage is.

I'm available for libavif support or general questions, and I'd love to see AVIF support in Firefox someday even if libavif isn't used. The more adoption, the better! I'd just like to get the conversation (re-)started.

Assignee: nobody → jbauman

I've implemented support for importing and exporting AVIF files in darktable [1] using libavif with the help of Joe Drago (thanks again). It will be available in version 3.0.1 soon. In case you need more pictures than just AVIF example files mentioned above.

[1] https://darktable.org/ - raw image developer

See Also: → AV1

That's great, Andreas! I'll definitely be checking it out as soon as 3.0.1 is released.

I created a test page to see which browser support AVIF without any JavaScript polyfill code:
http://188.121.162.14/avif/
Konqueror with patched khtml is able (the screenshot in the following article):
https://www.reddit.com/r/AV1/comments/faayy9/avif_image_browser_test/
I'd love to see AVIF support in Firefox too.

That's a cool test, dnovomesky. One thing that might make it even better is using a test image URL that gets served with the image/avif content type. The one you're using gets served up as binary/octet-stream since that server isn't configured to be aware of the AVIF MIME type.

That's a good remark Jon! I will try to change my test page in the future so the returned .avif will have proper mimetype image/avif .

Depends on: 1625363
Depends on: 1634741
Depends on: 1634751
Blocks: 1621548

Setting dev-doc-needed so we track documenting this. If there winds up being a separate "enable for release" bug, please add dev-doc-needed to that one too. Thanks!

Keywords: dev-doc-needed
See Also: → 1635296

I see that there is no registered media type (MIME type) for AVIF https://www.iana.org/assignments/media-types/media-types.xhtml#image
I can help with registerig a type if needed.

That would be great, Chris. I'm a bit surprised that hasn't already happened given what the spec says. Probably the spec editors are the best point of contact for synchronizing around that work.

Its now common to develop a media type registration in parallel with the rest of spec development. The trick is judging when to send it to IANA for comment. Too early and things aren't stable enough; too late and it is hard to change things that are wrong, due to existing deployment.

Depends on: 1641208
See Also: → 1639409
Depends on: 1553289

1553289 blocks this because libavif's lossless mode uses the identity matrix. https://github.com/AOMediaCodec/libavif/issues/243

Depends on: 1654461
Depends on: 1654462
Depends on: 1656099
Depends on: 1657075
Depends on: 1657300
Depends on: 1657346
Depends on: 1658008
See Also: → 1644950
Depends on: 1658906

https://aomediacodec.github.io/av1-avif/#change-list

Changes since v1.0.0 release
Remove the definition of the image/avif-sequence MIME type.
Adding bitdepth constraint for alpha and master images. This is a non-backwards compatible change.

FYI if you try to encode an AVIF image with squoosh.app, and you have image.avif.enabled turned on, it will crash. This works fine in Chrome 85 which has AVIF support enabled by default.

Let me know if you think i should file a bug for this.

(In reply to atjn from comment #19)

FYI if you try to encode an AVIF image with squoosh.app, and you have image.avif.enabled turned on, it will crash. This works fine in Chrome 85 which has AVIF support enabled by default.

Let me know if you think i should file a bug for this.

You should file a bug. FWIW, I couldn't reproduce that.

Attached image squoosh.app.avif

Please do file a bug that blocks this one, atjn. I generated an AVIF with squoosh just now (attached) and it opened successfully for me in both nightly and beta.

Flags: needinfo?(dev)

Arh, sorry for causing a stir. My beta and nightly browsers were both on version 80.x 🤦‍♂️
This doesn't currently work in stable, but i can confirm it is fixed in Firefox 81+

Flags: needinfo?(dev)

Yeah, image.avif.enabled isn't turned on by default in stable yet. Though with 80.0, I'm still able to view that image if I turn it on. You said you were getting a crash. Can you reproduce that? If so, I'm interested in figuring out how.

Though with 80.0, I'm still able to view that image if I turn it on. You said you were getting a crash. Can you reproduce that? If so, I'm interested in figuring out how.

Oh, interesting. In that case, here is a bug for it.

Depends on: 1661335
Depends on: 1661347
Depends on: 1661368
Depends on: 1662198
Depends on: 1662200
Depends on: 1663767

I'm just a user, not a dev, so I didn't want to make a new bug for this until I had commented here to verify it.

Currently (80.0.1), Firefox is not applying color management to AVIF images.

I have image.avif.enabled and image.avif.use-dav1d enabled. I also have color management enabled for all images via gfx.color_management.mode=1 and a monitor profile set.

Comparing two identical images, one AVIF, one PNG, it's immediately obvious that the AVIF one is not color managed. This is obviously an issue because it means that a JPEG or PNG image replaced with an identical AVIF image will look completely different. For example, this screws up my image codec comparison.

I attempted to add a sRGB profile to an AVIF image using MP4Box and the icc_path= option. Even with the color profile explicitly added, Firefox ignored it and did no color corrections. I can't be certain that the profile was added correctly, though, as the tooling around AVIF is still very janky.

If you'd like to see if an ICC profile is present in an AVIF, you can just use:

avifdec --info image.avif

One of the lines it'll print will be something like:

  • ICC Profile : Absent (0 bytes)

If it says "Present" and has a byte count, and that count is identical to the .icc file you supplied, the ICC profile probably made it intact.

If you'd like to extract the ICC profile from the AVIF, this should work:

colorist convert image.avif image.icc

Err, as a note on the colorist line:

If you do that statement, it will emit something as an ICC profile even if there isn't one present, but if there wasn't one, the output will say something like:

[   decode]     No embedded ICC profile, using SRGB

If you see that, no ICC profile is in the file.

(In reply to adam.m.fontenot from comment #25)

I have image.avif.enabled and image.avif.use-dav1d enabled. I also have color management enabled for all images via gfx.color_management.mode=1 and a monitor profile set.

Please note that the full color management mode isn't supported by default yet. See bug 455077. So if there is a problem with AVIF only please file a new bug and mark it blocking the former bug. Thanks.

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+2] from comment #28)

Please note that the full color management mode isn't supported by default yet. See bug 455077. So if there is a problem with AVIF only please file a new bug and mark it blocking the former bug. Thanks.

(adam.m.fontenot from comment #25)

… I also have color management enabled for all images via gfx.color_management.mode=1 and a monitor profile set.

I think there is already an AVIF-ICC bug filed at https://bugzilla.mozilla.org/show_bug.cgi?id=1634741.

avifdec --info image.avif

Thank you for this, I was actually unaware of avifenc / avifdec and had been using aomenc to output an IVF frame and then muxing that to AVIF with MP4Box. I will at least start testing avifenc and switch to using that if it gives equivalent quality to aomenc (as I expect it should).

In any case I can confirm that my embedded ICC profile did show up with avifdec. So Firefox is ignoring the profile, but as mentioned above there's already bug 1634741 for that.

So if there is a problem with AVIF only please file a new bug and mark it blocking the former bug. Thanks.

The specific issue I'm having is that when in full color management mode, AVIF images (whether untagged or tagged with sRGB) aren't treated as sRGB gamut images, so they look very wrong on high quality modern screens. PNG, JPEG, WEBP, etc etc all seem to work fine. You want the bug marked as blocking bug 455077 and not this bug? I'll try to file it later today.

adam.m.fontenot: since I think this is AVIF-specific, I'm happy to make bug 1634741 more of a general-purpose AVIF color management bug and save you the trouble of filing a new one.

(In reply to Jon Bauman [:jbauman:] from comment #31)

adam.m.fontenot: since I think this is AVIF-specific, I'm happy to make bug 1634741 more of a general-purpose AVIF color management bug and save you the trouble of filing a new one.

Works for me, thanks!

Depends on: 1669033

For the MDN docs I started populating a section about AVIF in the Image file type and format guide > AVIF.

The table at the end is missing information about maximum dimensions and supported color modes. It would be good to get parity with the other formats like JPG where we have info on resolution and supported colour modes. However all the information I can find on the codec appears relevant to video and doesn't cover this sort of thing.

This is therefore best probably done by someone who knows something about both image formats and this particular code.

Can you please suggest:

  • What you would expect to see there - modes etc.
  • Where the information might be obtained, or who best to ask.

If not we can leave it empty, but I'd prefer to add useful information if you guys have it.

Flags: needinfo?(jbauman)

AVIF doesn't really have max width or height limits defined. It depends on the encoder you use. The limit is probably UINT32_MAX x UINT32_MAX. As that's the datatype the implmentations use.

  • color modes: YUV444, YUV422, YUV420
  • greyscale support: yes (YUV400)
  • bits: 8/10/12 bit
  • alpha support: yes
  • ICC profile support
  • NCLX supports: sRGB, linear sRGB, linear Rec2020, PQ Rec2020, HLG Rec2020, PQ P3, HLG P3 (there are more but those are the interesting ones)
  • Compression: lossless and lossy
  • Tiling support

Does that help?

As far as maximum dimensions, AVIF is based on the generic image format HEIF (ISO/IEC 23008-12:2017), which requires an Image spatial extents (ispe) property. That property defines each dimension as a unsigned 32-bit value. In practice libaom returns this metadata on the decoded image as unsigned int and dav1d uses int. So, definitely no larger than UINT32_MAX, but I wouldn't expect uniform behavior of implementations beyond the maximum positive value of a platform int.

For everything else, Andreas's comment looks perfect to me.

Flags: needinfo?(jbauman)

Disclaimer: Everything Andreas and Jon said is correct.

As a note, Wan-Teh Chang and I put in a semi-artificial ceiling in libavif right now which blocks images whose total area is greater than 16384^2. I'm sure we'll reassess when people come up with legitimate usage for such image sizes in AVIF, but for now, this feels like a reasonable protection as otherwise a super tiny AVIF could advertise a monstrous file size and we'd have a 42.zip on our hands.

From avif/internal.h:

// A maximum image size to avoid out-of-memory errors or integer overflow in
// (32-bit) int or unsigned int arithmetic operations.
#define AVIF_MAX_IMAGE_SIZE (16384 * 16384)

It is up to you all to come up with your own level of tolerance for massive AVIFs. :-)

(In reply to joedrago from comment #36)

when people come up with legitimate usage for such image sizes in AVIF

Oh! 16k feels positively like 1990s in my humblest of opinions. People came up with legitimate uses for much more ages ago; first example that comes to mind:
https://commons.wikimedia.org/wiki/Category:Gigapixel_images_from_the_Google_Art_Project

I play with the software I use on JPEGs over an artificial 32k limit imposed by some writers… and when it barfs, I file a bug ;-) They are 8bit, I know, but still. Artificially limiting the new format at such a low value by such a high-profile project risks setting a very dangerous example. For example: for Firefox.

Sorry, I didn't mean to be dismissive with that choice of words. Nothing in libavif (other than that explicit define) assumes any other limits like this. In theory, you could set that number as high as you want (on a x64 build) and I imagine libavif would handle it just fine.

I'm sure someone will send me a PR someday which adds a CMake option to dodge these kinds of limitations. I still think it is a sensible default for now.

For the record, Firefox imposes no such limit currently and uses dav1d for decoding by default. I haven't dug into the dav1d sources enough to know whether there are limits lower than the maximum positive int on the platform in dav1d, but if you want to create a super large AVIF and file a bug should it break in Firefox, please feel free.

dav1d has a configurable setting dav1dSettings.frame_size_limit, which I believe defaults to 0 unless dav1d detects it is a 32bit build and clamps to 8192^2. So yes, I think if you have an x64 build and don't manually set frame_size_limit, you can decode any size you can fit.

Phew! It is me who is sorry: I misunderstood that this is just a config default tweakable by the lib implementer. Thank you both for alleviating my fears! :-)

A few comments on the MDN article
https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Image_types#AVIF

(e.g. lossy AVIF images are around 50% smaller than JPEG images). Good compression relative to WebP, which compresses the same images to around 30% of the JPEG size.

That is slightly confusing (one uses % smaller than, one uses % of) and seems to say that WebP does better than AVIF. Is that true?

MIME type image/avif

No such MIME type has been registered, see https://www.iana.org/assignments/media-types/media-types.xhtml#image

I can push on the authors of the AVIF format to register it, and help them if needed.

The color modes supported are given in section 6.4.2. Color config semantics, of the AV1 Bitstream & Decoding Process Specification

In practice though, BT.2020 primaries (which are also used for HDR with BT.2100) will be the most widely used.

AVIF is a subtype of HEIF, and there is a MIME registration for image/heif
https://www.iana.org/assignments/media-types/image/heif

but it is oddly constrained to one particular subtype of HEIF, so AVIF would not be able to use they same MIME type.

OK, I pushed on the AVIF authors to register sooner rather then later
https://github.com/AOMediaCodec/av1-avif/issues/119

Chris, Andreas, Jon et al. Thank you. I think this is more than sufficient.

That is slightly confusing (one uses % smaller than, one uses % of) and seems to say that WebP does better than AVIF. Is that true?

Thanks for pointing that out. Generally AVIF does better, though it is going to depend on the particular setup. I have reworded this and linked to the source doc I got the comparison from
Generally AVIF has better compression than WebP — median 50% vs 30% compression for the same JPG set (source: AVIF WebP Comparision (CTRL Blog)).

No such MIME type has been registered, see https://www.iana.org/assignments/media-types/media-types.xhtml#image

Thanks, I have clarified that the MIME type is what Chrome and Firefox use, and that at time of writing not yet registered.

Re maximum size

I have put in 2,147,483,647×2,147,483,647 pixels. I take the view that we talk about what the spec offers, not particular implementation limitations.

Re colour modes.

I copied in Andreas' info from above, and also mentioned Chris's point to the spec where this is defined. I didn't mention that "In practice though, BT.2020 primaries (which are also used for HDR with BT.2100) will be the most widely used." because to my mind that is beyond what the broad audience will need to know or will find useful.

Anyway, feel free to edit/clarify further in the docs.

Rumour has it that this feature will be enabled by default in FF83 next week. Can anyone confirm or deny?

Flags: needinfo?(jbauman)

It's not even enabled in nightly so that is highly highly unlikely.

Flags: needinfo?(jbauman)

When enabling AVIF support, images are not requested with HTTP Accept containing image/avif, like this is done for image/webp. Is that already handled or should I open a bug about this?

Note to future documenter, most of the work needed when this is accepted into an official release has been done and is described here: https://github.com/mdn/sprints/issues/3804#issuecomment-711544431 (PS, thanks Timothy for note above).

(In reply to Vincent Bernat from comment #48)

When enabling AVIF support, images are not requested with HTTP Accept containing image/avif, like this is done for image/webp. Is that already handled or should I open a bug about this?

This should've been fixed ~5 months ago with bug 1641208. Is your version of Firefox 79 or newer? If so, can you please provide your repro steps so we can see if there's something additional to fix?

One thing to note: if you have a value set for image.http.accept in about:config, that will take precedence and be used regardless of whether image.avif.enabled is turned on or not. To ensure dynamic behavior of the Accept header according to whether AVIF (and WebP) are enabled, make sure image.http.accept is set to the empty string (which is the default value).

Flags: needinfo?(bernat)

(In reply to Jon Bauman [:jbauman:] from comment #50)

(In reply to Vincent Bernat from comment #48)

When enabling AVIF support, images are not requested with HTTP Accept containing image/avif, like this is done for image/webp. Is that already handled or should I open a bug about this?

This should've been fixed ~5 months ago with bug 1641208. Is your version of Firefox 79 or newer? If so, can you please provide your repro steps so we can see if there's something additional to fix?

One thing to note: if you have a value set for image.http.accept in about:config, that will take precedence and be used regardless of whether image.avif.enabled is turned on or not. To ensure dynamic behavior of the Accept header according to whether AVIF (and WebP) are enabled, make sure image.http.accept is set to the empty string (which is the default value).

Very sorry about that. I have just tried again, and image/avif is present. I have also tried in an existing tab and it's here too.

Flags: needinfo?(bernat)

(In reply to Vincent Bernat from comment #51)

Very sorry about that. I have just tried again, and image/avif is present. I have also tried in an existing tab and it's here too.

OK, it's not here when requesting an image directly, while image/webp is. For example, browse https://www.mozilla.org/media/contentcards/img/home-2020/card_1/card1-high-res.1726940bba0b.png and look at the "Network" tab in devtools. So, this is pretty minor.

OK, it's not here when requesting an image directly, while image/webp is. For example, browse https://www.mozilla.org/media/contentcards/img/home-2020/card_1/card1-high-res.1726940bba0b.png and look at the "Network" tab in devtools. So, this is pretty minor.

Ah! That's because when you request an image directly (as opposed to in the context of an <img> element), the request is considered to be in the "document" context rather than the "image" context and the code for generating the Accept header is different. Fortunately, that's about to be addressed via bug 1658008 which is nearly ready to merge thanks to sagudev.

Depends on: 1681816
Depends on: 1682662
Depends on: 1682677
Depends on: 1682995
Depends on: 1684688
Attached image GenevaDrive.avif

Is there any plan on supporting animated AVIF? Currently it doesn't seem to work despite this codec being extremely performant with animation.

Joining a simple animation as an example.

Depends on: 1686338

Yes, we'd like to support animations and image sequences. Nobody had filed a bug on it yet, so I went ahead and added bug 1686338. Thanks for the reminder!

Depends on: 1687443

I'm not sure if this should go in a new bug, but there seems to be some caching issues with avif.

On my website, quackack.com, on first loading, the avif images load. But on second loading, they don't. However, they will load if you reload with control+F5.

I choose the images programatically in javascript through react when the page loads. My website is a single page web app hosted on S3.

There seems to be something wrong with the caching logic for avif images, and chrome suffers from a similar issue.

(In reply to quackackjac from comment #56)

I'm not sure if this should go in a new bug, but there seems to be some caching issues with avif.

You can file a bug that blocks this one. The point of this meta bug is to collect all AVIF-related issues.

However, it seems unlikely to me that there would be anything wrong on the browser side related to AVIF-specific caching since nothing about Firefox's caching behavior is AVIF specific. The fact that you're seeing similar behavior in Chrome makes me wonder if the issue could something else entirely. In the bug you file, please include details for reproducing the behavior, including your OS and browser versions (the details for Chrome as well would be helpful).

(In reply to Jon Bauman [:jbauman:] from comment #57)

I filed a bug for this at https://bugzilla.mozilla.org/show_bug.cgi?id=1690001

I think you are right that its not actually an AVIF issue at all, but something dumb in the default way the react service worker handles avif files. My website is a very vanilla react website with very minimal configuration, so I am having trouble figuring out where exactly the issue is. I'll need to continue investigating this. I suppose that is what I get for deciding to use a framework at all.

(In reply to quackackjac from comment #58)

(In reply to Jon Bauman [:jbauman:] from comment #57)
I filed a bug for this at https://bugzilla.mozilla.org/show_bug.cgi?id=1690001

It was an old, cached service worker on my laptop from when I tried to add AVIF support late 2019. I should have tested this on another machine. Sorry for the noise.

No worries! Thanks for following up and thanks for giving AVIF a try with Firefox

Adding this just in case:

In the following website comparing AV1 to other formats, the lossy AVIF images seem to lose a lot of colour information under section comparing Flat illustrations : https://jakearchibald.com/2020/avif-has-landed/

Screencapture video: https://www.youtube.com/watch?v=uQ1QOztHHes

Firefox Beta 86.0
Build ID: 20210217195321
OS: Linux 5.10.16-arch1-1 #1 SMP PREEMPT Sat, 13 Feb 2021 20:50:18 +0000
All av1 flags set to true in config.

As I mentioned in bug 1634741. I'm concerned Firefox is going to enable AVIF by default before relevant colour bugs are fixed. As things stand now, it seems like AVIF colour is broken on non-sRGB displays. It also seems like this colour inaccuracy may go beyond the typical colour inaccuracy around Firefox's continued mishandling of untagged colour on some displays and operating systems (as demonstrated here: https://kornel.ski/en/color).

My particular concern around shipping AVIF on-by-default-but-buggy is that this would mean that services like CDN image optimizers will not be able to trust the Accept header for working AVIF support. If the colour can't be trusted to be correct, image optimization using AVIF will be incorrect. Not being able to trust the Accept header adds a lot of complication that nobody wants. Similar issues with the Accept header and WebP support in Chrome meant that the header wasn't trustable for years (who knows how many buggy and out of date installations were around) and I would hate to see the same thing happen again with AVIF. In the past, this meant determining image support using the User-Agent but that is definitely not a desirable solution and, with new privacy measures being taken, will be impossible in the future.

I would say that it is very important that either these colour issues are fixed before enabling AVIF by default. I would delay enabling it by default if necessary.

Sorry to push so hard on this but Firefox 86 is going to be released in two days and AVIF support is not ready enough to ship. Please see my comments on bug 1682995.

I agree with Nick that Firefox shouldn't ship AVIF enabled by default without fixing AVIF critical bugs. Can we please postpone enabling AVIF by default to subsequent versions until colour issues are fixed?

I agree with Nick that shipping with critical bugs will have an overall chilling effect on being able to depend on AVIF support.

https://github.com/AOMediaCodec/libavif/releases

libavif 0.9.0 released, Firefox Nightly should be updated to the latest version.

Here Firefox 86.0, can't see the picture in this url: https://petapixel.com/2021/02/20/android-12-will-support-avif-photos/

Firefox 86 does not have avif afterall, it was disabled in 86 at the last minute.

You need to log in before you can comment on or make changes to this bug.