1878852 - Consider reporting a console warning when image decoding fails due to unsupported AVIF features

Status: NEW

(Core :: Graphics: ImageLib, defect)

Description

[Claus Andersen]

Attachment: 1920x908 blank correct test avif file which Firefox currently cannot render

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:122.0) Gecko/20100101 Firefox/122.0

Steps to reproduce:

Having a srcset with images Firefox cannot parse does not give an error.

<img src="img/radio1920x908.png"
     alt=""
     srcset="img/radio5604x2650.avif 5604w,
             img/radio5120x2421.avif 5120w,
             img/radio4096x1937.avif 4096w,
             img/radio3840x1816.avif 3840w,
             img/radio2560x1211.avif 2560w,
             img/radio2048x968.avif  2048w,
             img/radio1920x908.avif  1920w,
             img/radio1920x908.png   1920w,
             img/radio1280x605.avif  1280w,
             img/radio1280x605.png   1280w,
             img/radio640x303.avif    640w,
             img/radio640x303.png     640w,
             img/radio320x151.avif    320w,
             img/radio320x151.png     320w"
     sizes="100dvw">

Due to bug 1696090 (AVIF grid-based image support) Firefox cannot display all avif files.
This problem compounds as the console does not display any errors when an image within a srcset should be displayed.

I have attached a tiny 1920x908 avif file shich can be used to easily reproduce as part of a srcset. The behaviour is the same for the HTML tag img srcset and when doing CSS background-image: image-set.

Actual results:

This works as expected in a certain Blink based browser. An avif image is fetched and displayed for all resolutions when changing the viewport. Have not tested Webkit. With Goanna it does pick up the srcset but tries the avif which they do not support. The png fallback does not work as hoped.

It is the same image resized to different resolutions and then saved using the same settings using medium profile to get the optimal filesize.

In Firefox you will see the image for 640w, 1280w, 2560w, 4096w easily on a 4K monitor. And by using mobile responsive view you can verify 320w and 5120w.

But for 1920w, 2048w, 3840w and 5604w it shows nothing and blanks the image!

Expected results:

Expected behaviour to me:
A) Prominent display of error in the developer tools. Preferably the console log.
B) A display of the broken image icon when rendering fails.
C) This is debateable: But I expected a fallback to the next when multiple formats are supplied for the same width in the srcset.

As I have done image stacks before (especially for webp) I spent a lot of time looking into this. I had a very clear expectation that any failures would be logged to the console or somehow be prominently displayed in developer tools. Maybe I need to manage my expectations but I consider this a bug in how errorhandling is done with srcset. We can spin it and call it "sub-optimal". But cursing was involved :-D.

In Network Tools we see GET with imageset as the Initiator. Bytes are transferred as expected when we resize. If we have been at a resolution before we see "Cached" as expected. If we are too fast we see "$bytes (raced)". All with a status of "200 OK". If we are much too fast then we get "NS-BINDING_ABORTED" with a failed status. All very good and as expected.

No errors anywhere. Nothing in the console. Only headscratching.

With this new after the fact knowledge I found another route. It is a smarter way but I think it is buried too deep (said the village idiot :-)).

Go to the Firefox Profiler. You will not see it in the default view for "Call Tree". But if you select "All frames" or "Native" for the "Call Tree" you will now see an additional entry after "imgLoader::LoadImage". A little gray "mozilla::image::RasterImage::DoError". DoError Does not Do what (I think) it should do.

If you venture further into the profiler we can look at the "Marker Table". You will see offending images with a "Load" and think things are OK. But you need to look for what you cannot see. There are no "Image Paint" for the offending images.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'DevTools::Console' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Julian Descottes [:jdescottes]]

The error should probably be created when handling the image tag. Moving to Core DOM.

Comment 3

[Claus Andersen]

Clarification 1: The attached test file is just a very small demonstration of a spec compliant file which cannot render in Firefox. As it is just a blank canvas understand that this file is not used to verify if the images in the srcset are rendered as blank!

Here are 3 tiled/grid images from the official AVIF test suite which does not work in Firefox:
Summer_in_Tomsk_720p_5x4_grid.avif
animals_00_multilayer_grid_a1lx.avif
animals_00_multilayer_grid_lsel.avif

Clarification 2: The previous srcset example does not w3 validate as it has multiple identical w-values. This gives the same results and validates:

<picture>
  <source type="image/avif"
    srcset="img/radio5604x2650.avif 5604w,
            img/radio5120x2421.avif 5120w,
            img/radio4096x1937.avif 4096w,
            img/radio3840x1816.avif 3840w,
            img/radio2560x1211.avif 2560w,
            img/radio2048x968.avif  2048w,
            img/radio1920x908.avif  1920w,
            img/radio1280x605.avif  1280w,
            img/radio640x303.avif    640w,
            img/radio320x151.avif    320w"
    sizes="100dvw">
   <source type="image/png"
     srcset="img/radio1920x908.png   1920w,
             img/radio1280x605.png   1280w,
             img/radio640x303.png     640w,
             img/radio320x151.png     320w"
     sizes="100dvw">
   <img src="img/radio1920x908.png" alt="">   <!-- Fallback -->
</picture>

Clarification 3: A) I am expecting better error reporting, yes but B) If the browser fails to render the image it choose poorly in the srcset. I do not think it would be unreasonable to expect it to pick another candidate as fallback. Or C) Use the actual img src fall back.

And 3A - Better error handling is my primary concern as proper AVIF decoding seems to be low priority.

Comment 4

[Henri Sivonen (:hsivonen)]

emilio, it seems that you've edited the image DOM code quite a bit lately. What are your thought on the way forward here?

Comment 5

[Emilio Cobos Álvarez [:emilio]]

So unless I'm missing something this has nothing to do with srcset in particular. Same happens with <picture><source> and with <img src> etc.

The broken image icon isn't displayed because you're using alt="", and that is intended, see bug 1616537 and related. If you remove that, you see the broken image icon, right? Other than that, it seems like the behavior is the same as every other broken image. The real bug here is bug 1696090.

That said, if we know that some AVIF images are valid but we fail to decode them properly, we could send a console message after image decoding (moving this appropriately).

Changing the fallback mechanism only for AVIF seems bad, and changing it for all images seems risky and would need a specification change at least. Probably not worth going that route, specially since we wouldn't want authors to do that: Falling back only after decoding is terrible for resource usage and performance.

Ideally this use case would be covered by <source type="...">, but unfortunately it isn't because the mime type for avif that we don't support and avif that we do support is the same... :'(

Comment 6

[Claus Andersen]

Correct. It is not the srcset as such. But it is when the web developer is commonly using srcset it will bite the hardest. And I am so old that bug 1616537 was a surprise to me - I did not expect that.

You are technically correct regarding the alt tag but with a huge caveat. According to MDN:

The alt attribute is officially mandatory; it's meant to always be specified. If the image doesn't require a fallback (such as for an image which is decorative or an advisory icon of minimal importance), you may specify an empty string (""). For compatibility reasons, browsers generally will accept an image without an alt attribute, but you should try to get into the habit of using it.

I agree that a fallback for only avif is bad. Everything is relative (contingent on messy code!) but I am not sure I see the big risk in changing fallback behaviour for all. If an image fail to decode I would find it natural to try another candidate rather than doing nothing. Make a best effort to actually show something from the suppplied srcset. It was supplied for a reason! I cannot imagine anyone depending on the current behaviour when an image fail to decode. I have not verified but I think blink does the right thing(TM). And I think that there is a general understanding among web developers that it is completely opaque how the browser chooses within a srcset. The resource usage and performance is not really a concern here as this will never happen on the happy path. If the web developer is stupid and chooses to supply a broken image then we spend the extra resources to make the end user happy. A decent web developer will catch this easily (especially if an img.decode error is logged!!). But more importantly when the browser implodes on img.decode as is the case here a small penalty is worthwhile in the attempt to render the intended content. Proper error handling seems even more on point when bug 1696090 has been known for one month shy of 4 years! It is important to note that I only expect the penalty to occur when we have the img.decode error condition. Proper and compliant sites will render and look good in Chrome but looks weird in Firefox. Hence any risk could then be considered minimal.

But the reason for this report is that I have the (maybe unreasonable) expectation that the browser should log an error if an img.decode DOMException is raised during normal rendering. We are able to catch this via Javascript so why not when the browser does the rendering?

Maybe it was just the wording "we could send". I would say we should always send a console message after image decoding. If img.decode fails for any reason that should be cause for notification.

Last, I strongly disagree that this use case could be covered by source type. No different mime types for png either. The hard truth is that Firefox only has partial avif support and missing important parts used in the wild. This leads us back to agreement that bug 1696090 is the root cause. I do however feel that it also exposes lacking error handling and logging. Today avif, tomorrow bmp2000.

Examples which can be found on codepen as well.

A) img src<br>
<img src="https://bug1878852.bmoattachments.org/attachment.cgi?id=9378482"><br>
B) picture/source/img<br>
<picture>
    <source type="image/avif"
            srcset="https://bug1878852.bmoattachments.org/attachment.cgi?id=9378482 1920w"
            sizes="100dvw">
    <source type="image/png"
            srcset="https://upload.wikimedia.org/wikipedia/commons/6/6a/PNG_Test.png 1920w"
            sizes="100dvw">
    <img src="https://upload.wikimedia.org/wikipedia/commons/6/6a/PNG_Test.png">   <!-- Fallback -->
</picture>
<br>
C) img srcset src<br>
<img srcset="https://bug1878852.bmoattachments.org/attachment.cgi?id=9378482 1x" src="https://upload.wikimedia.org/wikipedia/commons/6/6a/PNG_Test.png">
<br>
D) img with onerror fallback<br>
<img src="https://bug1878852.bmoattachments.org/attachment.cgi?id=9378482" onerror="this.onerror=null; this.src='http://libpng.org/pub/png/img_png/pngnow.png'">
<br>
E) img.decode</a> does raise "EncodingError" DOMException.<br>
<script>
    const img = new Image();
    img.src = "https://bug1878852.bmoattachments.org/attachment.cgi?id=9378482";
    img.decode().then(() => {
        document.body.appendChild(img);
    }).catch(() => {
        document.body.appendChild(new Text(" ERROR: Could not decode the avif for example E"));
    });
</script>

• As the decoding process actually raises an error in the process as shown in D & E I think it is reasonable to expect that the browser logs an error in A - C. You only see the "broken image" if you fail to supply the alt tag.
• It would be reasonable to expect B to fall back to either source or at least img.
• It would be reasonable to expect C to fall back to src

Comment 7

[Emilio Cobos Álvarez [:emilio]]

As the decoding process actually raises an error in the process as shown in D & E I think it is reasonable to expect that the browser logs an error in A - C. You only see the "broken image" if you fail to supply the alt tag.

Sure, that's what I wrote in comment 5.

It would be reasonable to expect B to fall back to either source or at least img.
It would be reasonable to expect C to fall back to src

That's not what the spec says, nor how any other broken image works, so I don't think it would be reasonable. Also that'd give you an error event, then a load event, or multiple error events. That seems bad too?