699150 - html with two animated PNGs hangs the browser

Status: RESOLVED DUPLICATE of bug 523950

(Core :: Graphics: ImageLib, defect)

Description

[Chris Kingsley]

I have an html that uses two animated PNGs.  The first few frames are displayed, then the browser goes non-responsive.  The three files are tar / gzip to be able to attach them.  6 MB, sorry.

Comment 1

[Chris Kingsley]

Well now, it seems that simply loading either of the APNGs alone will also hang the browser.

The .png files were created using libpng, version 1.5.5, patched with libpng-1.5.5-apng.patch

Comment 2

[Josh Matthews [:jdm]]

Chris, can you provide a link or attachment with the failing APNG? We can't really do much if we can't reproduce the problem.

Comment 3

[Chris Kingsley]

Attachment: a gzip'd apng file that makes it hang.

I tried to attach this when first submitting the bug.  Somehow I missed the error message that it won't take attachments over 4 MB...  (Thanks, Josh.)

Comment 4

[Chris Kingsley]

Attachment: a second gzip'd apng file that makes it hang.

Comment 5

[Chris Kingsley]

Attachment: an html that loads the two APNGs

Comment 6

[Josh Matthews [:jdm]]

Chris, can you give more information about your setup? I can't reproduce this problem on OSX 10.6.8 nightlies.

Comment 7

[Chris Kingsley]

I'm running Firefox 7.0.1 (on the release update channel).  Running on Microsoft Windows XP, Version 2002, Service Pack 3.  On a Dell, Intel Core i7 CPU, 3.18 GB of RAM installed.

Comment 8

[Josh Matthews [:jdm]]

Do you see the same symptoms in trunk builds? http://nightly.mozilla.org/

Comment 9

[Chris Kingsley]

I don't have such a setup.  I'll give it a try on my home computer tonight.

Comment 10

[Chris Kingsley]

The task manager showed that the memory usage of the computer increased by about 1.5 GB when I tried to load the html with both images.  When loading one image alone it needed about 1.2 GB to render the first frame, another .1 GB to get another frame, and so on, until the process hung, having used up about 1.5 or 1.6 GB.

Could it be trying to cache fully rendered frames?  The image is 800w x 300h.  It only uses 7 colors, but they're 24 bits in the palette.  Nah.  Even that is less than 1 MB per frame, and it's using 100 times that much memory.  (Well, the images should go for 4000 frames, but it dies after rendering fewer than 10.)

(This was also on Windows XP SP3, Firefox 7.0.1)

Comment 11

[Chris Kingsley]

I just tried it on my larger computer.  Windows Vista SP 2, AMD 8450 triple core processor, 4.0 GB physical memory, 64 bit operating system, Firefox 7.01 (though running a 32-bit version of firefox)

It had the same gigantic memory hoggage, and almost brought down the computer.  I was eventually able to kill the tab with the .png, and the computer gradually came back to life.

Comment 12

[Josh Matthews [:jdm]]

Is the behaviour any different with a nightly build from the link in comment 8?

Comment 13

[Max Stepin]

4000 frames? 800x300 each? 
Well, of course it will use a lot of memory.

I would say it's not a bug.

Comment 14

[Chris Kingsley]

Of course it's a bug for a loaded file to hang a program.

It should not take 1500 MB to render 10 frames that can be stored in 1 MB each.

Comment 15

[Chris Kingsley]

As a bug fix, it should detect when it can't get enough memory and report an error.

As an enhancement -- I have observed that the first time it renders my animation the memory usage peaks at about 1.34 MB / frame.  After going through the first cycle it drops to about half that usage.  It would be good if it would not have that peak usage, but only the half.  Even better would be if it would do a proper area / time trade-off, and if it would be too much memory to cache the rendered frames, then just re-render them each time.

Now that I understand what's going on, I can work around, and get something useful.

Comment 16

[Max Stepin]

I'm not sure where the "10 frames 1 MB each" comment comes from... these animations are 4000 frames, not 10 frames.

It might be possible to predict how much memory will be needed for APNG, but it's not possible for GIF, because GIF does not store "number of frames" in the header.

Comment 17

[Chris Kingsley]

The "10 frames" comes from that the program hung after rendering only 10 frames.  It must have tried to pre-allocate the memory for the whole animation.

In this particular case, the rendering is so easy that the CPU utilization never went above about 8% (my machine, of course).  It's unfortunate that the program chose to tilt the speed vs memory trade off to the speed when it didn't need to.

Comment 18

[:aceman]

There really is something wrong. I loaded just the first png from local drive and it freezed FF7 after about 10 frames, using 1.5GB of virtual memory (as seen in Windows task manager). I had to kill FF.

On FF10, it managed to draw more frames and slowing down quickly, also taking so much memory. But while it was still a bit responsive I managed to open about:memory. But I couldn't copy it as rightclicking on the page made FF crash.
Crashreports:
bp-21215c6b-9cfe-49c9-9654-a31cb2111104
bp-a6cdb974-ce83-4ce8-944d-4d3692111104

Then, after starting FF10 again, it offered to restore the session (with the png and about:memory, both of them listed as thumbnails (favicons)). Even after I clicked Start new sssions, it consumed 1,7GB of RAM, this time also in the Used memory column of Task manager.

Comment 19

[Boris Zbarsky [:bzbarsky]]

There is an existing bug on not decoding all frames of an animation.  Sounds like this is a duplicate of that.

Chris, it's not just speed vs memory; there are correctness issues involved too.

Comment 20

[Emanuel Hoogeveen [:ehoogeveen]]

(In reply to Boris Zbarsky (:bz) from comment #19)
> There is an existing bug on not decoding all frames of an animation.  Sounds
> like this is a duplicate of that.
Bug 677044? That should be fixed by backout of bug 609499.

Comment 21

[Chris Kingsley]

Boris, I don't understand what you mean by correctness issues.  I thought that to render frame N you could only ever need the completely resolved frame N-1.  You shouldn't need to store frames 0 to N-2.  Why isn't that right?

Comment 22

[Max Stepin]

In theory, it's possible to correctly decode the whole animation without storing all frames in memory. I think the actual problem is about invoking inflate() or DoLzw() continuously.

Comment 23

[:aceman]

Boris wanted to say that Firefox is aggressively decoding all frames even before displaying them. Each 800x300 frame takes about 1MB therefore the whole png really wants to allocate 4GB of RAM.

This is more likely a dupe of bug 661304 or some other dependent of bug 683284.

But I am not sure it explains the crashes in comment 18.

Comment 24

[Max Stepin]

For bug 629606 I created a test page that by my calculations should use 512MB of RAM to store uncompressed animation frames. That's exactly what I see in about:memory and there are no leaks after I close the tab.

No matter what you do, it will always be possible to create a very heavy web page, and cause memory problems as the result. 

It doesn't even have to be animation. Someone could just stick 10MB of extremely compressed (1:200 or 1:300) static PNGs on the webpage, and browser will ask for 2GB-3GB of RAM to store uncompressed images. 

The only way to solve this is not to create such heavy web pages.

Comment 25

[:aceman]

Yes, but for animations this can be mitigated by not allocating memory for frames not currently visible. In extreme case, only the memory for 1 frame will be always allocated so in case of sane resolutions (like here 800x300) it would work nicely. Yes, it will be slower and use CPU to always re-decode frames when they become visible. So it is a tradeoff.

Comment 26

[Max Stepin]

Such tradeoff will result in better memory usage for 0.01% users running unrealistic extreme tests, and higher CPU usage for all others.

Comment 27

[:aceman]

Yes, FF developers need to find the balance.

But it seems other browsers do not have the problem, otherwise people would not create such sites.

Reporter (Chris) can you say which other browsers work on the testcase fine? I have tried IE 8, but it does not seem to support APNG. Can't test others on Win XP.

Comment 28

[Hugh Nougher [:Hughman]]

Attachment: about:memory while viewing apng2.png from desktop

Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0a1) Gecko/20111106 Firefox/10.0a1

> Explicit Allocations
> 2,841,821,791 B (100.0%) -- explicit
> ├──2,779,097,983 B (97.79%) -- heap-unclassified
> ├─────40,503,184 B (01.43%) -- images

Why is there so much heap-unclassified? My only guess is that it is temporary usage while decoding the images since bug 629606 has almost no heap-unclassified.

I ran it a few times (getting crashes now and again). Sometimes images can use a few hundred MB of memory and show lots more strange red/green frames.

For comment 27, I just ran it in the latest Opera release.
The APNG was still a lot of strange red and green lines but it appeared to display a lot more frames and used only 600MB of RAM.

Comment 29

[Hugh Nougher [:Hughman]]

(In reply to Hugh Nougher [:Hughman] from comment #28)
> temporary usage while decoding the images since bug 629606 has almost no
> heap-unclassified.

I better clarify my wording a little better here. The test was done using attachment 551282 [details] in bug 629606.

Comment 30

[Max Stepin]

Attachment: apng2_400.png

I cut the second example to include only first 400 frames, and by my calculations all frames together are 238MB uncompressed, it matches nicely to what I see in about:memory

270.56 MB (100.0%) -- explicit
├──239.76 MB (88.61%) -- images
│  ├──239.41 MB (88.49%) -- content
│  │  ├──239.41 MB (88.49%) -- used
│  │  │  ├──238.81 MB (88.26%) -- uncompressed-nonheap

Comment 31

[Hugh Nougher [:Hughman]]

With attachment 572433 [details] I ran more tests. I'm sorry if this is taking the bug off topic.

First of all, if I disable hardware acceleration I seem to get almost exactly the same results as comment 30 (except occasionally it is double like bellow, this doubling seems to be random).
> 478.99 MB (94.01%) -- images

Another thing I noticed with non acceleration method is Process Explorer graphs two jumps in memory usage, the second happens as soon as the animation starts the second time.
> 239.90 MB -- gfx-surface-win32
and after
> 478.70 MB -- gfx-surface-win32

Now with acceleration.
> 239.59 MB (88.45%) -- images
> 13.27 MB (04.90%) -- heap-unclassified
> 239.91 MB -- gfx-surface-image
> 315.97 MB -- private
and after a second rise which happens sometime during the second loop of animation
> 254.62 MB (49.70%) -- heap-unclassified
> 239.59 MB (46.77%) -- images
> 478.72 MB -- gfx-surface-image
> 559.52 MB -- private

So given the above information it seems that RAM usage of these images is doubled.
I can always reproduce these results.

Now its past 1am. Zzzz.

Comment 32

[Chris Kingsley]

Would it be hard to put a wrapper around the render/display a frame code?  One wrapper would be used when the image would cycle and could be expected to fit in memory.  The other would be used when it wouldn't cycle or didn't fit in memory.  For png, the memory size of the rendered images can be known early on, leading to a correct choice of wrapper.  Shouldn't be too much code, it would run fast for the common cases of small images cycling, and it wouldn't crash the browser for the big data displays.

Comment 33

[Max Stepin]

Is it worth the trouble, how often people post 3 min animations as GIF or APNG?

Comment 34

[:aceman]

And what would be a lossless alternative?

Comment 35

[Max Stepin]

Lossless formats aren't really for the web. But in this particular case, considering 5 colors palette, it might help convert it into GIF.

I think uncompressed APNG frames are always 32bpp, while GIF frames are 8bpp.

Comment 36

[Hugh Nougher [:Hughman]]

For the record, my findings in comment 31 will be handled in bug 700514 since I believe its the same problem.

Comment 37

[Chris Kingsley]

I wasn't using it for the web.  I was using it as a data presentation method.  If it works, it can be used for other things.

Comment 38

[Max Stepin]

In that case, I would use one of the lossless video codecs, like Lagarith.
And use a video player, instead of a browser.
That way you will get a benefit of having playback controls.

Comment 40

[Stefan Endrullis]

I'm a bit disappointed that this ticket is open since 7 years and Firefox is still not able to handle APNGs memory efficiently.  My current Firefox version 59.0.2 (64-Bit) is still consuming 2.5 GB RAM to open a single web site (https://stackoverflow.com/questions/29248188/is-there-a-way-to-create-custom-postfix-completions-in-intellij), while Chromium needs only 50 MB RAM for the same site.  Or in other words:  Firefox needs 50 times the RAM of Chromium.  And since more and more tools now support the generation of APNGs I expect them to replace the animated GIFs peace by peace.

Comment 41

[Andrew Osmond [:aosmond] (he/him)]

Stefan, I hope you will be pleased to learn this has been fixed by bug 523950 :). It is currently in the 60 beta, so that is why you still hit the problem in 59.

I confirmed that nightly memory usage is reasonable for both the test page attached to this bug and the stackoverflow link you provided in comment 40. If you continue to see this problem after 60 is released (or you try the beta/nightly builds), please reach out and I will ensure it is addressed!