514841 - A 10MB GIF file should not be automatically rendered if it will consume over a gigabyte of memory and cause Firefox to become unresponsive and/or crash

Status: RESOLVED INCOMPLETE

(Core Graveyard :: Image: Painting, defect)

Description

[Brandon Mitchell]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2

A 10MB GIF file should not be automatically rendered if it will consume over a gigabyte of memory and causes Firefox to become unresponsive and/or crash.

If you are opening a lot of tabs in the background it is difficult to pinpoint and shut down the the tab with the 10MB GIF on it, especially if the hard disk is thrashing and Firefox and the whole system are becoming extremely unresponsive. (The STOP button is not global; it affects only the current tab.)


Reproducible: Always

Steps to Reproduce:
Go to the URL and view the version of File:Harmonograph_trace.gif uploaded on 14 August 2009 (the current version as of 9/5/2009).

Actual Results:  
Firefox uses at least a gigabyte of memory and, depending on how much RAM is installed on your computer, may become unresponsive or crash (presumably because new memory cannot be allocated after the system's RAM and page file are maxed out).


Expected Results:  
At some point Firefox should identify the GIF animation as problematic, and the image should stop loading and be replaced with a warning of some sort: "Your computer has 256MB of RAM. Attempting to render this GIF animation on a computer with less than a gigabyte of memory could cause Windows to resize the page file, and may cause Firefox to become unresponsive and/or crash. Click here to have Firefox attempt to render the image anyway." Or maybe: "An image on this page is consuming a lot of memory and causing Firefox to become unresponsive. Do you want to continue loading the image? [Yes] or [No]".

Comment 1

[Brandon Mitchell]

Attachment: page file graph

Comment 2

[Ria Klaassen (not reading all bugmail)]

Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.3a1pre) Gecko/20090905 Minefield/3.7a1pre

The display of the image causes no problem here.
Also with latest Shiretoko no problem.

Comment 3

[Brandon Mitchell]

> The display of the image causes no problem here.

I only have 256 MB of RAM. On a computer with a few gigabytes of RAM the GIF file probably would not cause problems. However, you should still be able to confirm that the page at the above URL uses a relatively large amount of memory by typing "about:cache" in the location bar.

My point is that Firefox should be able to identify situations in which it is not possible to render an animated GIF with the available system resources (256 MB of RAM in my case). Even if a 10 MB GIF file does not causes problems on a newer computer with a lot of memory, a 100 MB GIF file might.

But if your about:cache page says that the file is using less than, say, a few hundred megabytes of memory, then I guess the rendering of the GIF file is not a problem. (When I access the file, I seem to get a least of a gigabyte of memory usage according to Windows Task Manager, but I can't access about:cache due to the unresponsiveness of Firefox.)

Comment 4

[Jo Hermans]

Actually, the images aren't visible in about:cache anymore in Firefox 3.5.

Your example is the worst case - luckily it's not very common.
- since Firefox 3.0, decoded images are always stored with 4 bytes per pixel, even though a GIF only needs 1 per pixel. This simplified the code a lot.
- there's no optimization for animated GIFs. I think that each frame occupies the same amount of memory
- images are removed from memory when they're not needs anymore (30 seconds after their display or something). But there was a problem with animated frames : frames could not removed from memory after they were used. Animated frames stays in memory until the entire page is replaced (caused by bug 414259). Bug 500402 was filed for a solution.

Comment 5

[Brandon Mitchell]

I've previously had problems with large GIF files when browsing Category thumbnail galleries on Wikimedia Commons. The Wikimedia software apparently doesn't create server-side thumbnails for GIF images, so the original GIF files are downloaded and scaled to thumbnail size on the client side by the browser - this can use a lot of memory when viewing a Category containing hundreds of thumbnails of large GIF images.

If the image at the above URL doesn't cause problems, I've attempted to create a proof of concept with a 9X-looped extended version, made with VirtualDub. I intended it to be 9*10MB = 90MB, but it ended up only being 18.2 MB for some reason. Still, the 9,000 frames occupy 25 GB of memory when uncompressed (versus 2.8 GB for the original file), according to VirtualDub at least. Perhaps this version will cause problems with people who have 4 GB of RAM; I personally haven't tried loading it into Firefox because I'm afraid of what might happen.

http://rapidshare.com/files/276166227/harmonograph-trace-ext.gif
(18.2 MB)

(The original GIF image was published on Wikimedia Commons with the author opting to release it into public domain, so uploading a derivative work on RapidShare shouldn't pose a legal problem. See "URL" field above for the original image.)

Comment 6

[Jo Hermans]

I repeat : this is the worst case scenario. In order to show the animated image, Firefox needs to decode all frames, and it does it in 4-bytes per pixel. Your 9000 frames animation will ALWAYS occupy an enormous large amount of memory. The same thing with large images, that need to be decoded as a whole before they can be resized.

It might be possible to optimize these algorithms, or use less memory (bug 500402), but I don't know if these are planned. These types of images are very rare anyway, exactly because they can cause such problems (in some browsers more than others). I know one website where they're used regularly, but even then it's only around 80 MB or so, and I haven't heard a complaint yet. The image is neatly removed form memory when you go to another page.

If you really have such a large animation, you should consider to use video anyway, beauase that's what this data is. Animated images is a hack from the early 90's.

Comment 7

[Brandon Mitchell]

Even if such images are worst-case and relatively rare, I do unfortunately encounter them in the wild, especially when browsing Wikipedia and Wikimedia Commons.

Another example of a resource-intensive GIF (not as bad as the first):
http://en.wikipedia.org/wiki/File:Golly_Constructor_layout.gif
at the page
http://en.wikipedia.org/wiki/Von_Neumann_cellular_automata

Fixing Bug 224452 and adding the ability to selectively "play" disabled GIF animations with a left-click or context-menu item would help a lot. Then I could set image.animation_mode = none, and the GIF would be frozen by default and only animate after I explicitly execute the "play" command.

Comment 8

[Brandon Mitchell]

What about showing only the first frame with a message like "This image could not be animated due to memory constraints," if the following is true:

imgWidth * imgHeight * imgFrameCount > maxAnimatedGifArea,

where maxAnimatedGifArea is a function of an about:config pref that specifies a percentage of the number of bytes of installed RAM? Instead of choking and dying, Firefox would refuse to swallow animated GIFs that are too large.

Apparently even the Wikimedia thumbnail generator has had problems with GIF files, as I got the above inequality here:
Bug 19089 – animated gif brought down wiki - 
https://bugzilla.wikimedia.org/show_bug.cgi?id=19089

More Bugzilla@Wikimedia links about problems with Giant GIFs:
https://bugzilla.wikimedia.org/show_bug.cgi?id=16451
https://bugzilla.wikimedia.org/show_bug.cgi?id=20312

http://commons.wikimedia.org/wiki/Category%3ANeue_Gedichte_%28Heine_1852%29
(category on Commons that uses 2+ GB of memory)

http://leuksman.com/misc/giant.gif
(a GIF that caused problems with the Wikimedia thumbnail generator but loads OK in Firefox, albeit slowly, with lots of page file usage)

Comment 9

[Brandon Mitchell]

See also Bug 455100 – A tiny, malicious SVG file causes Firefox to stall and use up the system's memory

That bug is about a relatively small SVG file causing Firefox to become unresponsive; this bug is about a relatively small GIF file causing Firefox to become unresponsive.

The reporter of the other bug suggests:
"Rendering should not block menus/other tabs, if rendering takes too long, a message like the one for slow javascript programs should ask the user whether to continue."

Comment 10

[timeless]

since these files aren't well supported, one would hope the people producing them would be told to stop. tell wikimedia to start rejecting them. if they stopped hosting them a significant portion of your problem would go away (and their headaches to boot).

adding lots of overhead and a terrible user experience for this edge case isn't a particularly good idea.

eventually we will change our handling so that this image would be hopefully decoded in a process for the tab in which it's rendered, which depending on how stupid your os is would mean you could more easily use the rest of your tabs and the browser itself, and you'd be able to see that the site you're visiting is stupid and vote w/ your feet by not visiting such a site, or using whatever comment and auditing system the provider has to discourage hosting of such objects.

Comment 11

[Douglas Held]

Comment 10 displays a terrible attitude toward application security.  Malicious
input is the fault of the user and the operating system, so it is OK
for Firefox to just crash or hang when it encounters input data the
developers weren't accounting for?

It's OK if we don't know how to solve the problem, and we simply say so.  But blaming a denial of service vulnerability on the end user is not a reasonable replacement.

Comment 12

[timeless]

we have thousands of crashes throughout our product. your crash is rare. there are many more valid cases to accept and render 10mb gif files.

we do have a plan to solve these problems in general by moving rendering to another process. we also hope to use a reduced privilege model for that process so if we do screw up, less damage is done.

however, those are projects, they might fix a given bug, but their development and updates do not belong in this bug.

your bug is mostly useless as it doesn't actually have any crash reports.

the unresponsive bit is unfortunate, the gif decoder could and perhaps should be improved so that it lets other things breathe, but again, rewriting decoding is not something which belongs in a bug such as this.

thanks for your time. marking as incomplete as this bug won't be fixed based on your input.