2055634 - Firefox shows an error message for any AVIF image file that has a 'lsel' property (besides 0xffff)

Status: NEW

(Core :: Graphics: ImageLib, defect)

Description

[Wan-Teh Chang]

Attachment: lsel-test-files.zip

I originally reported this issue in https://github.com/AOMediaCodec/av1-avif/issues/333.

The attached lsel-test-files.zip archive contains the following AVIF image files:

lsel-test-files/photo_progressive_60_s6gd_aom3141_lsel0.avif
lsel-test-files/photo_progressive_60_s6gd_aom3141_lsel1.avif
lsel-test-files/photo_progressive_60_s6gd_aom3141_lsel2.avif
lsel-test-files/photo_progressive_60_s6gd_aom3141_lsel3.avif
lsel-test-files/photo_progressive_60_s6gd_aom3141_lsel4.avif
lsel-test-files/photo_progressive_60_s6gd_aom3141_lsel0xffff.avif
lsel-test-files/photo_progressive_60_s6gd_aom3141.avif

These AVIF files all contain an elementary AV1 bitstream consisting of two spatial layers (of spatial layer ID 0 and 1). They differ in whether they contain a 'lsel' property and the value of the 'layer_id' field in the 'lsel' property. I encode that information at the end of the filename. For example, lsel-test-files/photo_progressive_60_s6gd_aom3141.avif does not have a 'lsel' property, lsel-test-files/photo_progressive_60_s6gd_aom3141_lsel0.avif has a 'lsel' property with 'layer_id' equal to 0, and so on.

Since there are only two spatial layers, the _lsel2, _lsel3, _lsel4 files should be invalid.

Firefox shows an error message for any file that has a 'lsel' property.

I think the correct behavior for the files with a 'lsel' property is as follows:

• lsel-test-files/photo_progressive_60_s6gd_aom3141_lsel0.avif: show layer 0
• lsel-test-files/photo_progressive_60_s6gd_aom3141_lsel1.avif: show layer 1
• lsel-test-files/photo_progressive_60_s6gd_aom3141_lsel2.avif: error (invalid file)
• lsel-test-files/photo_progressive_60_s6gd_aom3141_lsel3.avif: error (invalid file)
• lsel-test-files/photo_progressive_60_s6gd_aom3141_lsel4.avif: error (invalid file)
• lsel-test-files/photo_progressive_60_s6gd_aom3141_lsel0xffff.avif: either show layer 1 only; or first show layer 0 and then show layer 1 (progressive rendering)

A simple first step would be to allow a 'lsel' property with 'layer_id' equal to 0xffff. It should be treated as if the 'lsel' property were absent.

Comment 1

[Timothy Nikkel (:tnikkel)]

Thanks for the heads up. I'll get the 'lsel' property with 'layer_id' equal to 0xffff fix done quickly at least so we don't break for that case.

Comment 2

[Wan-Teh Chang]

Thank you, Timothy. The layer_id=0xffff case is described in the last paragraph of https://aomediacodec.github.io/av1-avif/v1.2.0.html#layered-items-overview:

[AV1] expects the renderer to display only one frame within the selected Operating Point, which should be the highest spatial layer that is both within the Operating Point and present within the temporal unit, but [AV1] leaves the option for other applications to set their own policy about which frames are output, as defined in the general output process. AVIF sets a different policy, and defines how the 'lsel' property (mandated by [HEIF] for layered images) is used to control which layer is rendered. According to [HEIF], the interpretation of the layer_id field in the 'lsel' property is codec specific. In this specification, the value 0xFFFF is reserved for a special meaning. If a 'lsel' property is associated with an AV1 Image Item but its layer_id value is set to 0xFFFF, the renderer is free to render either only the output image of the highest spatial layer, or to render all output images of all the intermediate layers and the highest spatial layer, resulting in a form of progressive decoding. If a 'lsel' property is associated with an AV1 Image Item and the value of layer_id is not 0xFFFF, the renderer is expected to render only the output image for that layer.

According to the AV1 spec, progressive decoding requires a 'lsel' property with layer_id equal to 0xffff, but libavif also allows progressive decoding when the 'lsel' property is absent. Because of this Firefox bug, libavif cannot add a 'lsel' property with layer_id equal to 0xffff when encoding a layered image (see libavif PR #2429 and PR #2430).

Comment 3

[Timothy Nikkel (:tnikkel)]

Pull request is up in bug 2055813 /https://github.com/mozilla/mp4parse-rust/pull/453
Whenever that gets review I'll land it, should be soon.