Last Comment Bug 692573 - Incorrect rendering of resized animated gif images
: Incorrect rendering of resized animated gif images
Status: RESOLVED FIXED
: regression
Product: Core
Classification: Components
Component: ImageLib (show other bugs)
: Trunk
: All All
: -- normal (vote)
: ---
Assigned To: Scott Johnson (:jwir3)
:
Mentors:
http://www.furaffinity.net/journal/22...
Depends on:
Blocks: 694374 666446
  Show dependency treegraph
 
Reported: 2011-10-06 13:40 PDT by Scott Johnson (:jwir3)
Modified: 2012-01-05 13:21 PST (History)
8 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---
-


Attachments
Patch to fix this bug (1.02 KB, patch)
2011-10-10 12:25 PDT, Scott Johnson (:jwir3)
joe: review-
Details | Diff | Splinter Review
Test Case (31.44 KB, text/html)
2011-10-10 12:37 PDT, Scott Johnson (:jwir3)
no flags Details
Patch (v2) (1.21 KB, patch)
2011-10-13 11:55 PDT, Scott Johnson (:jwir3)
no flags Details | Diff | Splinter Review
Patch (v3) (3.92 KB, patch)
2011-10-14 14:51 PDT, Scott Johnson (:jwir3)
joe: review-
Details | Diff | Splinter Review
Patch (v4) (3.07 KB, patch)
2011-10-15 16:45 PDT, Scott Johnson (:jwir3)
joe: review+
Details | Diff | Splinter Review

Description Scott Johnson (:jwir3) 2011-10-06 13:40:11 PDT
Some animated gifs, such as the hyena in: http://www.furaffinity.net/journal/2263796/ appear to be animating incorrectly. This could be related to bug 666446.
Comment 1 Scott Johnson (:jwir3) 2011-10-06 13:41:33 PDT
See also bug 666446 comment 376.
Comment 2 Scott Johnson (:jwir3) 2011-10-06 14:01:23 PDT
So it looks like this is a result of the image frames coming out of order. When I view the image in GIMP, it looks like there is a 0ms time per frame. I wonder if maybe changes that were made in bug 666446 might be causing a frame to be skipped, since the refresh driver is now coalescing paint events. 

The correct display of the image is actually dependent on the ordering of images, as some of the frames aren't reproductions of the whole image, simply sprite animations.
Comment 3 Scott Johnson (:jwir3) 2011-10-06 14:02:28 PDT
(In reply to Scott Johnson (:jwir3) from comment #2)
> The correct display of the image is actually dependent on the ordering of
> images, as some of the frames aren't reproductions of the whole image,
> simply sprite animations.

I meant 'frames' when I said 'images' in:  '... dependent on the ordering of images...'.
Comment 4 Scott Johnson (:jwir3) 2011-10-10 12:25:48 PDT
Created attachment 565990 [details] [diff] [review]
Patch to fix this bug

So, it looks like the problem here was that the first frame of the animation wasn't being composited correctly. Instead of being composited against the last frame of the animation after it had looped once, the logic of AdvanceFrame() was detecting that it was the first frame of the animation, and thus not correctly calling DoComposite. I modified this to only skip DoComposite (that is, redraw everything from scratch) only if we're on the first frame and first iteration of the animated image.

This is going to be integrated into bug 666446's patches for landing, but I wanted to get it reviewed on its own before integrating it. As long as things look ok, I'm going to add this to bug 666446 and close this bug.

Note that if you intend to apply this patch, it has as a prerequisite that the patches from bug 666446 be applied first.
Comment 5 Scott Johnson (:jwir3) 2011-10-10 12:37:22 PDT
Created attachment 565994 [details]
Test Case

Posting a more concise test case. This fails after patches for bug 666446 is pushed (visible problems), but looks fine after the patch for this bug is pushed.
Comment 6 Joe Drew (not getting mail) 2011-10-11 15:28:35 PDT
Comment on attachment 565990 [details] [diff] [review]
Patch to fix this bug

Review of attachment 565990 [details] [diff] [review]:
-----------------------------------------------------------------

::: modules/libpr0n/src/RasterImage.cpp
@@ +388,5 @@
>    imgFrame *frameToUse = nsnull;
>  
> +  // We need to treat the first frame as special (no compositing), but only if
> +  // it's the first time through the animation. (See bug 692573).
> +  if (nextFrameIndex == 0 && mLoopCount == 0) {

mLoopCount == 0 is true precisely on the *last* frame of an animation, not the first, because we *decrement* mLoopCount in AdvanceFrame.
Comment 7 Daniel Holbert [:dholbert] 2011-10-11 15:33:47 PDT
(In reply to Joe Drew (:JOEDREW!) from comment #6)
> mLoopCount == 0 is true precisely on the *last* frame of an animation, not
> the first, because we *decrement* mLoopCount in AdvanceFrame.
(s/frame/loop/)
Comment 8 Scott Johnson (:jwir3) 2011-10-11 16:30:52 PDT
Hmmm.. yes, you're right of course. It must be that its now compositing EVERY frame, so that's why this patch works. Basically, what it seems to me is that the first frame isn't getting composited correctly. I'll try it again with the correct logic, and see if it's still working.
Comment 9 Scott Johnson (:jwir3) 2011-10-11 19:43:16 PDT
So, why are we treating the first frame as special anyway? Wouldn't it make sense to call DoComposite() on every frame, regardless of whether it's the first frame of the animation or not? 

I suppose there is probably a performance gain by not calling DoComposite() if we don't have to - i.e. if the dirty region is such that we're going to be repainting the entire area of the image anyway, but in that case it seems like we should be detecting that, rather than this method of trying to detect if we're on the absolute first frame of the animation.

Even for a single-looped image, our performance gains are going to decrease multiplicatively with the number of frames in the image, so it doesn't seem like a large performance gain in that case. 

Or perhaps I'm missing some subtle detail here?
Comment 10 Joe Drew (not getting mail) 2011-10-12 13:57:15 PDT
(In reply to Scott Johnson (:jwir3) from comment #9)
> So, why are we treating the first frame as special anyway? Wouldn't it make
> sense to call DoComposite() on every frame, regardless of whether it's the
> first frame of the animation or not?

The first frame of a GIF is always stored in 32-bit colour, so we don't need to do any composition to draw it (except, I guess, in this case!!) It's also a little complex to define the "previous frame" in general for the first frame, especially for the very first time we're drawing it. Also, the first frame has to be refreshed to clear all the changes that have been made while drawing the animation.

But aside from those caveats, I am supportive of removing special cases!

> Even for a single-looped image, our performance gains are going to decrease
> multiplicatively with the number of frames in the image, so it doesn't seem
> like a large performance gain in that case. 

I don't really understand what this means.
Comment 11 Scott Johnson (:jwir3) 2011-10-12 14:11:35 PDT
(In reply to Joe Drew (:JOEDREW!) from comment #10)
> 
> The first frame of a GIF is always stored in 32-bit colour, so we don't need
> to do any composition to draw it (except, I guess, in this case!!) It's also
> a little complex to define the "previous frame" in general for the first
> frame, especially for the very first time we're drawing it. Also, the first
> frame has to be refreshed to clear all the changes that have been made while
> drawing the animation.
> 
> But aside from those caveats, I am supportive of removing special cases!

Ok, I'll see what I can do to remove this special case. I am optimistic that it can be removed. :)

> I don't really understand what this means.

Yeah, you probably don't need to worry about it. It was actually more me just thinking out loud toward an argument against it being a performance gain. What I was trying to say is that if we have an animated image with 10 frames, that loops 100 times, then the probability that we are able to take advantage of any performance gain by not compositing the _very_ first frame is 1 / 10 * 100 = 1/1000, and that this decreases by a factor of 100 for each frame we add to the animation. I guess it was more just to convince myself that there probably wasn't a performance gain in effect here. :)
Comment 12 Scott Johnson (:jwir3) 2011-10-13 11:55:15 PDT
Created attachment 566896 [details] [diff] [review]
Patch (v2)
Comment 13 Scott Johnson (:jwir3) 2011-10-14 09:31:43 PDT
After some debugging, I've noticed that RasterImage::AdvanceFrame() isn't actually called prior to painting the first frame of an animated image. That is to say, it appears to only be called when we want to actually ADVANCE to the next frame. It is called, of course, when currentFrame = numFrames -1, and nextFrame = 0, but this is included in the general case where we want to call DoComposite().

Thus, my suggestion is to simply remove the code for the special case (the if statement) and always call DoComposite(), because AFAICT, we never actually use the special case code at all (or, rather, we're using it right now every time we want to paint frame 0, but this is incorrect).

I've been performing some tests on the effects of removing this if statement, and it appears that all of our current unit tests are passing, as well as all of my interactive test cases for bug 666446. I'm running the entire test suite right now, just to verify I didn't miss anything.

Joe originally recommended that we move frameChanged() back into AdvanceFrame(), rather than have it in RequestRefresh(), but I don't especially like this idea for two reasons - 

1) if we are updating our image at a slower pace (e.g. if the tab with animations is in the background, and the refresh driver is throttled back), then we're going to be calling frameChanged() for every frame advancement we perform, even though we really only need to do it for the final resulting frame. 

2) I recall that we originally did this when developing the architecture for bug 666446, and I know there was a reason we choose to go with the frameChanged() being in the requestRefresh(), call, rather than in AdvanceFrame(). I can't remember the exact details, but I know that there is a problem waiting to hit us in the face. (Yeah, this argument doesn't hold as much water as I'd like, but I just have this gut feeling...)

At any rate, I'd like to discuss this and see what interested parties think about it.
Comment 14 Scott Johnson (:jwir3) 2011-10-14 14:51:22 PDT
Created attachment 567189 [details] [diff] [review]
Patch (v3)

After speaking with Joe in IRC, we came to the conclusion that it would probably be better if we could just accumulate the dirty rectangle during consecutive AdvanceFrame() calls, and then use this for the dirty rectangle that's passed to FrameChanged() at the end of RequestRefresh(). This method seems to work well, so it's what I'm posting for review.
Comment 15 Joe Drew (not getting mail) 2011-10-14 18:06:12 PDT
Comment on attachment 567189 [details] [diff] [review]
Patch (v3)

Review of attachment 567189 [details] [diff] [review]:
-----------------------------------------------------------------

::: modules/libpr0n/src/RasterImage.cpp
@@ +416,5 @@
>  
>    // Set currentAnimationFrameIndex at the last possible moment
>    mAnim->currentAnimationFrameIndex = nextFrameIndex;
>    mAnim->currentAnimationFrameTime = aTime;
> +  *aDirtyRect = aDirtyRect->Union(dirtyRectCopy);

I think the Union should be external to AdvanceFrame(). Since AdvanceFrame() only ever advances one frame, there's no reason it should care about external state. RequestRefresh can accumulate the dirty rect.

I think this'll also result in cleaner code.

@@ +471,2 @@
>  #ifdef DEBUG
>    mFramesNotified++;

FWIW, this should be indented correctly in patch 9 in bug 666446.
Comment 16 Scott Johnson (:jwir3) 2011-10-15 16:45:43 PDT
Created attachment 567309 [details] [diff] [review]
Patch (v4)

(In reply to Joe Drew (:JOEDREW!) from comment #15)
> I think the Union should be external to AdvanceFrame(). Since AdvanceFrame()
> only ever advances one frame, there's no reason it should care about
> external state. RequestRefresh can accumulate the dirty rect.
> 
> I think this'll also result in cleaner code.
> 
Agreed. I made that change.

> FWIW, this should be indented correctly in patch 9 in bug 666446.
I corrected this, but the entire patch is going to be integrated into PATCH9 of bug 666446. I'm just pulling it out in this bug so that it's easier to review & you don't have to look through the entire PATCH9 every time there's a simple change to this guy. ;)
Comment 17 Joe Drew (not getting mail) 2011-10-16 07:32:01 PDT
Comment on attachment 567309 [details] [diff] [review]
Patch (v4)

Review of attachment 567309 [details] [diff] [review]:
-----------------------------------------------------------------

Hooray! \o/

::: modules/libpr0n/src/RasterImage.cpp
@@ -388,5 @@
>    imgFrame *frameToUse = nsnull;
>  
>    if (nextFrameIndex == 0) {
>      frameToUse = nextFrame;
> -    aDirtyRect = &(mAnim->firstFrameRefreshArea);

Oh man - this is something I definitely should have caught in review!

@@ +466,5 @@
>      }
>  
> +    // Notify listeners that our frame has actually changed, but do this only
> +    // once for all frames that we've now passed (if AdvanceFrame() was called
> +    // more than once.

Close parenthesis missing.
Comment 18 Scott Johnson (:jwir3) 2011-10-17 09:07:59 PDT
> Oh man - this is something I definitely should have caught in review!

No worries. I caught it. You can't be expected to catch everything in review. Two sets of eyes are better than one. :)

> Close parenthesis missing.

Fixed.

I've integrated this into PATCH9 for bug 666446. Should I leave this bug open until 666446 lands, or just close it now that it's actually been integrated into bug 666446's patches?
Comment 19 Joe Drew (not getting mail) 2011-10-17 09:31:54 PDT
(In reply to Scott Johnson (:jwir3) from comment #18)
> I've integrated this into PATCH9 for bug 666446. Should I leave this bug
> open until 666446 lands, or just close it now that it's actually been
> integrated into bug 666446's patches?

Well, this bug actually doesn't exist on trunk anymore, so this was fixed by the backout, and we won't have a similar bug when we reland bug 666446.

Note You need to log in before you can comment on or make changes to this bug.