1256646 - animated image set flickers with big rescaled images

Status: NEW

(Core :: Graphics: ImageLib, defect, P3)

Description

[rostislav.kouznetsov]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0
Build ID: 20160304114926

Steps to reproduce:

Go to
http://silam.fmi.fi/AQ_v5_4/aqforecast.html?parameter=O3&region=Europe&height=0
and wait until the animation starts.


Actual results:

Animation blinks until all images had been shown. After the first loop the animation goes smoothly. I guess, the reason is that FF removes previous image before displaying a new one, and caches the scaled image afterwards. 
The problem is reproducible with few versions of FF both in Windows and Linux.


Expected results:

All recent IE, Safari and Chromium i have seen show the animation smoothly from the very beginning.

Comment 1

[Loic]

Regression range:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=cac6192956ab&tochange=369a8f14ccf8

njn, I CCed you as you appear to work on ImageLib now. :)

Comment 2

[Nicholas Nethercote [inactive]]

I think this one might be beyond my meagre ImageLib abilities. tnikkel, can you take a look?

Comment 3

[Timothy Nikkel (:tnikkel)]

The site uses a series of img elements for animation. It has them all display:none, except the current frame. The images are about 30kb. When painting an image that isn't fully decoded we will sync decode it if it is under the size specified by the pref image.mem.decode_bytes_at_a_time, which is 16kb.

https://hg.mozilla.org/mozilla-central/rev/d5462de77e9f introduced this mechanism for deciding when to sync decode. Before that changeset we used to always do some sync decoding, limited by how much time we should spend decoding the image. We likely finished decoding in this time window before.

For animations that work by changing the src of a single img element we have a bug on file to show the previous image until the new one is decoded (don't have bug number handy). But for animations like this, with one img element per frame, I'm not aware of a plan.

Comment 4

[Mason Chang [Inactive] [:mchang]]

Do you know what bug Tim is referencing in comment 3?

Comment 5

[Seth Fowler [:seth] [:s2h]]

(In reply to Mason Chang [:mchang] from comment #4)
> Do you know what bug Tim is referencing in comment 3?

Bug 1177587 is the bug he references about changing the |src| of a single element. We cannot use that approach here. The second bug he references is bug 1253995, but we cannot use the approach discussed in that bug either, since different <img> elements are used in this case.

This is not easy to fix, because this approach to animation is frankly extremely bad. Here are the constraints that any fix would have to work within:

(1) <img> elements are displayed progressively, which means that we will paint them before the whole image is decoded. That inherently means that we'll get flicker if we display an <img> element for which the image is not decoded.

(2) We will not start sync decoding all images on paint, as that'd hurt performance massively.

(3) We will not start decoding all images, even if they're nonvisible, as that would hurt performance and more importantly memory usage massively.

So what remains is that we could try to heuristically detect this case and sync decode when we think it's happening. It's nontrivial to do so *selectively* because we have to consider all <img> elements on the page and not just a single one, as in the case of bug 1177587. I'd love to hear proposals for how we might implement a clever heuristic for this case.

A non-clever heuristic would be to just consider images as visible even if the frame subtree they're in is display:none. I'm not sure I want to go down that road, but it would certainly solve this problem.

It would be helpful to know if this is affecting more sites in the wild than just the single example given in comment 0.

Comment 6

[Nicholas Nethercote [inactive]]

When I viewed the site in question I only got a small amount of flickering on the first loop. It was little enough that I wouldn't have thought to complain. Does the flickering get worse in certain cases -- slower connection, slower computer, something else?

Comment 7

[Timothy Nikkel (:tnikkel)]

Slower computer likely makes it worse. There are actually three maps that are all part of the same image, the top map never flickered for me. The bottom two did.

Comment 8

[rostislav.kouznetsov]

Thank you all for taking care about the case. 

I am not aware about any other site that shows the problem. Though others do not attempt to make a high-resolution animation with controls (no animated GIFs) and scalable images (some people might want to see the animation on half-screen- window).

The flickering gets really strong if window is resized to about original image size or less, so image gets scaled.

OFFTOPIC: Indeed, I would appreciate if someone could tell me (may be by e-mail) how to implement the animation properly so it is smooth and demands less cpu/graphics etc., if it is faster than fixing Mozilla...
Some time ago I have somewhat reduced the problem in previous versions of Mozilla by turning visibility off only when animation is over, or previous image has to be shown . In normal animation it just turned visibility on, so new image appears over the old one http://silam.fmi.fi/aqforecast.html?parameter=O3&region=Europe&height=0. It helped for smaller images (Europe), but did not help much for larger images...
Would it be faster to swap image rather than swapping the visibility? Tank you once again.

Comment 9

[Mason Chang [Inactive] [:mchang]]

See comment 8.

Comment 10

[Seth Fowler [:seth] [:s2h]]

(In reply to rostislav.kouznetsov from comment #8)
> OFFTOPIC: Indeed, I would appreciate if someone could tell me (may be by
> e-mail) how to implement the animation properly so it is smooth and demands
> less cpu/graphics etc., if it is faster than fixing Mozilla...

Sure.

> Would it be faster to swap image rather than swapping the visibility?

Yes, it's drastically better, because the browser has a much easier time understanding what you're doing.

You should wait for the onload event to fire for all the images (which you're probably already doing). That event tells you that the images have been downloaded; none of these techniques will work unless you wait for that.

Then you can use one of these techniques to implement the animation. Any of these will work:

(A) Use a single <img> element and change the 'src' attribute when you want to advance to the next frame.

(B) Use a <canvas> element and draw a new Image into it when you want to advance to the next frame.

(C) Use multiple <img> elements, positioned over each other, and change their *opacity* when you want to advance to the next frame. This approach means that we still see the <img> elements as visible (as opposed to "display: none", which tells the browser that the images are not visible, and so we won't necessarily decode the image). You can do this using CSS Animations and your animation will run totally off the main thread. An example is here: http://people.mozilla.org/~bbirtles/bugs/1223658/wavey-davey.html

I should note that if you do (C) with CSS Animations, you may see some flicker due to a bug we're currently working on with *those*. =) For reference, that work is happening in bug 1223658.

I'd recommend (A) or (B) for now, but in the long term CSS Animations will be the best choice, as that will eventually provide the best performance. There is a proposal to allow animating "background-image" directly using CSS Animations, which would make what you want to do even easier, but unfortunately that feature is not available yet.