1739210 - AVIF (AV1 Image File Format): <picture> fallback for 'avis' major brand image sequences

Status: NEW

(Core :: Graphics: ImageLib, task, P3)

Description

[Jon Bauman [:jbauman:]]

Since an AVIF can contain either a "primary item" still image, an image sequence or both, the determination of the default rendering should be determined by the major brand in the ftyp box. See the specification for ISOBMFF (ISO/IEC 14496-12:202) § 4.3.1:

It identifies which specification is the ‘best use’ of the file, and a minor version of that specification; and also a set of other specifications to which the file complies

Currently, Firefox lacks AVIF image sequence support (see bug 1686338), so in the case that the major brand is avis, it's appropriate to consider the file unsupported for the purposes of <picture> element fallback processing.

Note also that Firefox already supports AV1 video within a <video> element that can serve the same purposes as most AVIF image sequences.

Comment 1

[sahil.ms]

Regarding using AV1 video, if there is a way to encode and render the AV1 videos with transparency, can you please share it?

Also, in my testing, rendering an AVIF image sequence with alpha inside <video> element works in Chrome, but it'll replace the alpha with white background. Same trick won't work in Firefox. So I guess even if we are able to create an AV1 video with alpha, the video element will just reject the alpha.

Comment 2

[Jon Bauman [:jbauman:]]

Regarding using AV1 video, if there is a way to encode and render the AV1 videos with transparency, can you please share it?

The way it would work would be basically identical to an AVIF image sequence since AVIF image sequences are basically just videos (that is a moov box) in the same container. I'd look at https://github.com/AOMediaCodec/libavif/pull/765 for some context.

Also, in my testing, rendering an AVIF image sequence with alpha inside <video> element works in Chrome

Can you share this example?

Comment 3

[sahil.ms]

Attachment: index.zip

I have attached the zip files, containing the source gif file, AVIF image sequence and the index.html

If AVIF image sequence is put inside the video element, the alpha is replaced with white colour in Chrome.
FIrefox won't render it.

Comment 4

[sahil.ms]

(In reply to Jon Bauman [:jbauman:] from comment #2)

Can you share this example?

Shared the example above ^

Comment 5

[arthur]

It would be better if Firefox just add support for Avif sequences, and they would not have to deal with this fallback issue.
I have also thought about using AV1 transparency, but it seems that ffmpeg does not support it, I think it should show YUVA420P, YUVA422P and YUVA444P in this file if they had support for it:

https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/libaomenc.c

In the meanwhile I'm just using:

<script>
  let videofallback = async(e) => {
    e.parentElement.innerHTML = `<picture>
        <source srcset='ship.webp' type='image/webp'>
        <source srcset='ship.apng' type='image/apng'>
        <img src='ship.gif' alt='ship'>
      </picture>`;
  }
</script><div>
  <video autoplay muted loop>
    <source src="ship.webm" type='video/webm; codecs="vp9"'
     onerror="videofallback(parentNode)" />
  </video>
</div>

As firefox supports VP9 it will probably not fall to the <picture> element, so I could probably just use Avif there.