1625363 - AVIF (AV1 Image File Format): experimental support

Status: RESOLVED FIXED

(Core :: Graphics: ImageLib, enhancement, P1)

Description

[Jon Bauman [:jbauman:]]

This bug is to track the release of an AVIF prototype behind the image.avif.enabled preference (disabled by default).

Comment 1

[Jon Bauman [:jbauman:]]

Attachment: Bug 1625363 - AVIF (AV1 Image File Format): experimental support. r=aosmond,tnikkel

There are many limitations currently, but this prototype should successfully
render most basic AVIF images. Known limitations:

• No support for any derived image items (crop, rotate, etc.)
• The primary image item must be an av01 (no grid support)
• HDR images aren't tone-mapped

Comment 2

[Jan Beich]

Comment on attachment 9136207 [details]
Bug 1625363 - AVIF (AV1 Image File Format): experimental support. r=aosmond,tnikkel

const bool hasAlpha = img->fmt & AOM_IMG_FMT_HAS_ALPHA;

Future versions of libaom won't support alpha, see https://bugs.chromium.org/p/aomedia/issues/detail?id=2199

Comment 3

[Laurence "GreenReaper" Parry]

Awesome to see this is being worked on! We've looked at WebP to replace JPG and PNG, but it still doesn't have 100% support, or 4:4:4 lossy for art.

For my community art site, alpha as specified in https://aomediacodec.github.io/av1-avif/#alpha-images would be needed to fully-replace PNG; however, the majority of works or reduced-size copies are JPG or PNG without an alpha channel, so we could just enable it for those. They're currently 8-bit per channel; at some point we may want to go further, but right now few even have screens supporting the full sRGB range.

Regarding mem pressure: art can be large - we support up to 16K / 36MB; but most are 1-4K / 1-10MB - and many users have moderate CPU freq but little RAM/cache. In case it matters: we scale using regular img containers with width/max-width set to viewport width, and height: auto, and for those on low-bandwidth connections, being able to see the image streaming in can be useful. [We also use CSS drop-shadow on it, but I guess that's irrelevant; it can be disabled if it's a problem, already is for mobile per bug 1273010.]

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9804951497f9
AVIF (AV1 Image File Format): experimental support. r=aosmond,necko-reviewers,valentin

Comment 5

[Bogdan Tara[:bogdan_tara | bogdant]]

Backed out changeset 9804951497f9 (bug 1625363) for bustages complaining about nsAVIFDecoder.cpp

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&fromchange=b54df3b14d13beaa0c7c3341796c5602a5e28fb2&searchStr=build&tochange=987060b94b516700bee056edbd97b7646b02ef97&selectedTaskRun=Vl26OWhcT1C1j1nu5TI_lQ-0

Backout link: https://hg.mozilla.org/integration/autoland/rev/987060b94b516700bee056edbd97b7646b02ef97

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=300020030&repo=autoland&lineNumber=24154

[task 2020-04-29T16:42:17.839Z] 16:42:17     INFO -  make[4]: Entering directory '/builds/worker/workspace/obj-build/image/decoders'
[task 2020-04-29T16:42:17.839Z] 16:42:17     INFO -  /builds/worker/fetches/sccache/sccache /builds/worker/fetches/clang/bin/clang++ -std=gnu++17 -o Unified_cpp_image_decoders0.o -c  -I/builds/worker/workspace/obj-build/dist/stl_wrappers -I/builds/worker/workspace/obj-build/dist/system_wrappers -include /builds/worker/checkouts/gecko/config/gcc_hidden.h -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -DNDEBUG=1 -DTRIMMED=1 -DOS_POSIX=1 -DOS_LINUX=1 -DMOZ_HAS_MOZGLUE -DMOZILLA_INTERNAL_API -DIMPL_LIBXUL -DSTATIC_EXPORTABLE_JS_API -I/builds/worker/checkouts/gecko/image/decoders -I/builds/worker/workspace/obj-build/image/decoders -I/builds/worker/workspace/obj-build/ipc/ipdl/_ipdlheaders -I/builds/worker/checkouts/gecko/ipc/chromium/src -I/builds/worker/checkouts/gecko/ipc/glue -I/builds/worker/checkouts/gecko/gfx/2d -I/builds/worker/checkouts/gecko/image -I/builds/worker/checkouts/gecko/gfx/skia -I/builds/worker/checkouts/gecko/gfx/skia/skia -I/builds/worker/workspace/obj-build/dist/include -I/builds/worker/workspace/obj-build/dist/include/nspr -I/builds/worker/workspace/obj-build/dist/include/nss -fPIC -DMOZILLA_CLIENT -include /builds/worker/workspace/obj-build/mozilla-config.h -Qunused-arguments -Qunused-arguments -Wall -Wbitfield-enum-conversion -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wshadow-field-in-constructor-modified -Wsign-compare -Wtype-limits -Wunreachable-code -Wunreachable-code-return -Wwrite-strings -Wno-invalid-offsetof -Wclass-varargs -Wempty-init-stmt -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wloop-analysis -Wc++2a-compat -Wcomma -Wimplicit-fallthrough -Wunused-function -Wunused-variable -Werror=non-literal-null-conversion -Wstring-conversion -Wtautological-overlap-compare -Wtautological-unsigned-enum-zero-compare -Wtautological-unsigned-zero-compare -Wno-error=tautological-type-limit-compare -Wno-inline-new-delete -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=backend-plugin -Wno-error=return-std-move -Wno-error=atomic-alignment -Wformat -Wformat-security -Wno-gnu-zero-variadic-macro-arguments -Wno-unknown-warning-option -D_GLIBCXX_USE_CXX11_ABI=0 -fno-sized-deallocation -fno-aligned-new -fcrash-diagnostics-dir=/builds/worker/artifacts -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread -pipe -g -Xclang -load -Xclang /builds/worker/workspace/obj-build/build/clang-plugin/libclang-plugin.so -Xclang -add-plugin -Xclang moz-check -O2 -fno-omit-frame-pointer -funwind-tables -Werror -Wno-error=shadow -fexperimental-new-pass-manager  -MD -MP -MF .deps/Unified_cpp_image_decoders0.o.pp   Unified_cpp_image_decoders0.cpp
[task 2020-04-29T16:42:17.839Z] 16:42:17     INFO -  In file included from Unified_cpp_image_decoders0.cpp:11:
[task 2020-04-29T16:42:17.839Z] 16:42:17    ERROR -  /builds/worker/checkouts/gecko/image/decoders/nsAVIFDecoder.cpp:324:16: error: unused variable 'next_img' [-Werror,-Wunused-variable]
[task 2020-04-29T16:42:17.839Z] 16:42:17     INFO -    aom_image_t* next_img = aom_codec_get_frame(mCodecContext.ptr(), &iter);
[task 2020-04-29T16:42:17.839Z] 16:42:17     INFO -                 ^
[task 2020-04-29T16:42:17.839Z] 16:42:17    ERROR -  /builds/worker/checkouts/gecko/image/decoders/nsAVIFDecoder.cpp:268:37: error: cannot pass an arithmetic expression of built-in types to 'CheckedInt<int>'
[task 2020-04-29T16:42:17.839Z] 16:42:17     INFO -    const CheckedInt<int> rgbStride = rgbSize.width * bytesPerPixel;
[task 2020-04-29T16:42:17.839Z] 16:42:17     INFO -                                      ^
[task 2020-04-29T16:42:17.839Z] 16:42:17     INFO -  2 errors generated.
[task 2020-04-29T16:42:17.839Z] 16:42:17     INFO -  /builds/worker/checkouts/gecko/config/rules.mk:750: recipe for target 'Unified_cpp_image_decoders0.o' failed
[task 2020-04-29T16:42:17.839Z] 16:42:17    ERROR -  make[4]: *** [Unified_cpp_image_decoders0.o] Error 1
[task 2020-04-29T16:42:17.839Z] 16:42:17     INFO -  make[4]: Leaving directory '/builds/worker/workspace/obj-build/image/decoders'
[task 2020-04-29T16:42:17.839Z] 16:42:17     INFO -  /builds/worker/checkouts/gecko/config/recurse.mk:74: recipe for target 'image/decoders/target-objects' failed
[task 2020-04-29T16:42:17.839Z] 16:42:17    ERROR -  make[3]: *** [image/decoders/target-objects] Error 2
[task 2020-04-29T16:42:17.839Z] 16:42:17     INFO -  make[3]: *** Waiting for unfinished jobs....
[task 2020-04-29T16:42:17.855Z] 16:42:17     INFO -  make[4]: Entering directory '/builds/worker/workspace/obj-build/dom/base'

Comment 6

[Jon Bauman [:jbauman:]]

Revision updated with fixes

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8cb47e0e268d
AVIF (AV1 Image File Format): experimental support. r=aosmond,necko-reviewers,valentin

Comment 8

[Razvan Maries]

Backed out for causing gtest failures at ImageDecoders.AVIFSingleChunk.

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedTaskRun=S-yXTfe9SUSNFRxH9BqyTw-0&revision=f9c7506b95af0b3cb46688a5761368b8ce411ecf

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=300139976&repo=autoland&lineNumber=24854

Backout: https://hg.mozilla.org/integration/autoland/rev/83beb87d9f6945ccddefe62341b5536398eb72a0

Comment 9

[Jon Bauman [:jbauman:]]

🤦‍♂️

Thanks for the heads-up. Investigating this now.

Comment 10

[Jon Bauman [:jbauman:]]

Looks like the issue was a midair collision between the addition of GTests for AVIF and this change to the decoder gtests generally. I was unaware of the other change, but I should've re-run the GTests locally to ensure they were still working.

I'm trying a fix which looks to be working and will update the revision shortly so landing can be retried.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/aaf93463fe1a
AVIF (AV1 Image File Format): experimental support. r=aosmond,necko-reviewers,valentin

Comment 12

[Alexandru Michis [:malexandru]]

https://hg.mozilla.org/mozilla-central/rev/aaf93463fe1a

Comment 13

[Marco Sousa]

Firefox Nightly 77.0a1 (2020-05-03) show AVIF images with an problem.

What's doesn't work as expected. When I open directly a avif file URL should open the image and not try to download it:
http://download.opencontent.netflix.com.s3.amazonaws.com/AV1/Chimera/AVIF/Chimera-AV1-10bit-1280x720-2380kbps-100.avif

Like when open a jpg file:
https://www.cleverfiles.com/howto/wp-content/uploads/2018/03/minion.jpg
It open the file and doesn't ask to download it.

Comment 14

[Marco Sousa]

**Firefox Nightly 77.0a1 (2020-05-03) show AVIF images just fine in my system, but directly url.

Comment 15

[Jon Bauman [:jbauman:]]

(In reply to Marco Sousa from comment #13)

What's doesn't work as expected. When I open directly a avif file URL should open the image and not try to download it:
http://download.opencontent.netflix.com.s3.amazonaws.com/AV1/Chimera/AVIF/Chimera-AV1-10bit-1280x720-2380kbps-100.avif

This is the correct behavior since this server is returning a generic content type for that request:

$ curl -LI "http://download.opencontent.netflix.com.s3.amazonaws.com/AV1/Chimera/AVIF/Chimera-AV1-10bit-1280x720-2380kbps-100.avif" | grep Content-Type
Content-Type: binary/octet-stream

The same issue has been reported with Chrome.

Here is an example from Link-U that correctly serves the content as image/avif and that renders correctly for me (assuming image.avif.enabled is true in about:config).

Comment 16

[Jon Bauman [:jbauman:]]

Release Note Request (optional, but appreciated)
[Why is this notable]: A new image format with excellent compression and no patent restrictions developed by the Alliance for Open Media
[Affects Firefox for Android]: No
[Suggested wording]: Experimental AVIF support can be enabled by turning on image.avif.enabled in about:config. Images accessed via a file:// handler will be rendered in-browser, but remote URLs require the server to be configured to return the image/avif Content-Type header. https://resources.link-u.co.jp/avif/ contains a large number of properly-served sample images.
[Links (documentation, blog post, etc)]: https://github.com/AOMediaCodec/av1-avif contains the specification and additional test files

Comment 17

[Kagami Rosylight [:saschanaz] (they/them)]

Here is an example from Link-U that correctly serves the content as image/avif and that renders correctly for me (assuming image.avif.enabled is true in about:config).

I have that flag disabled and Firefox does not try downloading. I guess this is a regression as it should download it.

Comment 18

[Jon Bauman [:jbauman:]]

(In reply to Kagami :saschanaz from comment #17)

Here is an example from Link-U that correctly serves the content as image/avif and that renders correctly for me (assuming image.avif.enabled is true in about:config).

I have that flag disabled and Firefox does not try downloading. I guess this is a regression as it should download it.

Is the behavior any different from accessing https://www.gstatic.com/webp/gallery/1.webp or https://developers.google.com/speed/webp/gallery1 with image.webp.enabled turned off?

Comment 19

[Kagami Rosylight [:saschanaz] (they/them)]

Is the behavior any different from accessing https://www.gstatic.com/webp/gallery/1.webp or https://developers.google.com/speed/webp/gallery1 with image.webp.enabled turned off?

No, and I think that's a bug too. Will file an issue for webp thing.

Comment 20

[joedrago]

Hi all, I just caught up on the conversation, and I wanted to make a quick comment about this:

Future versions of libaom won't support alpha

This is certainly true at the libaom/AV1 layer, but this isn't how AVIF provides alpha. AVIF (and HEIF) pack color into one AV1(/HEVC) payload, and then use an "auxilliary" item which contains a separate AV1(/HEVC) payload, in which the Y channel is to be treated as alpha for the primary item.

I didn't want there to be confusion in this context that "AVIF doesn't have alpha because of libaom". AVIF (and HEIF) as a format solves alpha a different way, and that should definitely be implemented in Firefox eventually (please).

Comment 21

[Jon Bauman [:jbauman:]]

Alpha support will definitely be coming. There are a number of additional pieces of data that need to be parsed with mp4parse-rust, but they should mostly be straightforward. The goal with the initial experimental support was just do to simplest thing to get simple images rendering in the browser so folks could play with it.

Comment 22

[tt_1]

Is there a way to disable avif support at compile time? I'd rather build with system-avif or not compile it at all.

Comment 23

[Darkspirit]

No. Firefox is just reusing its AV1 code, there is no huge AVIF library. This prototype uses libaom which was used previously for AV1 playback, but is still part of Firefox. Libaom will be removed from Firefox once the AVIF code is switched to dav1d as well.

Comment 24

[tt_1]

ok, thanks for the info. Can't wait to see libaom being cleaned from the source code tree.

Mind you that the --disable-av1 switch now causes a compile error with firefox-77.0_beta3, as of writing this I couldn't find evidence if that's an official thing or introduced by a user patch. Should I report a bug with all the logs?

Comment 25

[Julien Cristau [:jcristau]]

We don't normally include disabled features in release notes.

Comment 26

[Jon Bauman [:jbauman:]]

I'll defer to whatever our policies are, but I would say that adding a new image format is a pretty significant development. The earlier web developers are aware of the support and can play with it, the better the chances are of broad adoption.

Additionally, the more use this gets early, the more feedback and bug reports we can get, which will help ensure a robust implementation when it is shipped on by default.

Comment 27

[Kingsley]

I am just coming across this thread now and i am glad to affirm that this bug has been fixed in the version that i am using V90.0.2
I download here https://www.bravotecharena.com/2018/07/update-ios12-available-for-download-on.html and a window popped up to confirm if i prefer to open file with or save to directory.

Comment 28

[arifitworld]

Thanks for the heads-up. Investigating this now. I download it here https://theispfamily.com/ and a window popped up to confirm if I prefer to open the file or save it to the directory.

Comment 29

[etransfermoney1]

What's doesn't work as expected. When I open directly a avif file URL should open the image and not try to download it: https://etransfermoney.com/

Comment 30

[Julien Cristau [:jcristau]]

Restricting comments to stop spam on this old, resolved bug.

Comment 31

[Hamish Willee]

Removing "dev-doc-needed". This was added to docs in https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Image_types#avif_image in FF90 (ish) when the feature shipped.