+++ This bug was initially created as a clone of Bug #1130968 +++ See the STR in bug 1130968 comment 1, and further discussion about the details. In short, we hit the image cache limit, because of a number of animated images, and things start going bad at that point. Bug 1130968 covers the eviction of animated images from the image cache - this bugs covers increasing the 1GB limit where appropriate.
The size of the surface cache is calculated here https://dxr.mozilla.org/mozilla-central/rev/c9a70b64f2faa264296f0cc90d68a2ee2bac6ac5/image/SurfaceCache.cpp#937
And the 1GB limit is currently a preference image.mem.surfacecache.max_size_kb, so we'll need to do some tweaking. Probably change the default value to -1 to mean "automatic", limit it to 1GB on 32-bit OS, otherwise follow the algorithm, and see what we get. It'd be good to list out some scenarios for "I have this much RAM, I will get this much image cache" so that we can see if a modified heuristic is required.
If "PR_GetPhysicalMemorySize" function/method truly worked I would never see images stopping to appear, right? I have 32 GB of RAM; it is by far not the biggest value nowadays, but by formulae in the code resulting cache size I should be getting must be good enough. Also, even if the cache is limited, why can not older images be purged from there, and re-downloaded, if necessary (e.g. if I would jump to the top of the page)? There is no such mechanism in the ImageLib software component?
The 32GB of RAM is irrelevant because we have a preference set that says "do not go beyond 1GB". What would be great is if you could change the preference (in about:config) image.mem.surfacecache.max_size_kb from the default 1048576 to, say, 10GB instead (e.g., 10485760) and tell us how Firefox behaves. I imagine you could still get it in trouble, it would take much longer, but I'm not sure if we otherwise limit things that would preclude you from getting the full 10GB image cache. And, yes, absolutely, the purge of animated images is bug 1130968 / bug 686905.
Thanks for the explanation, Milan. I doubt that I have the time to try the 10 GB limit; even 1 GB limit takes time. Theoretically, I could automate this by using AutoHotKey/AutoIt scripting languages, but even this would take some time since it is already years since I last time programmed such things (by the way, does not Mozilla already have internal tools for checking stability/repeatable UI actions?). I hope that fixing underlying issues should work even without such test, though.