1130968 - Remove animated images from image cache

Status: RESOLVED FIXED

(Core :: Graphics, defect)

Description

[User Dderss]

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:38.0) Gecko/20100101 Firefox/38.0
Build ID: 20150208030206

Steps to reproduce:

In a 64-bit FireFox 38 I opened a Tumblr page with huge quantity of content and viewed it for quite some time (by going to the bottom of the page so that more photos/gifs/videos would be loaded).


Actual results:

After page has many hundreds of photo/gif/video nodes, FireFox starts to significantly slow down at first, and then ceases to render the photos and is even unable to show GUI elements such as "About Nightly" window; the browser is absolutely useless by that time. You have to restart; there is just no way to look at older photos in normal view in FireFox, users have to choose Archive view, which only presents small thumbnails of content -- which is unacceptable.


Expected results:

Page should be rendered and smoothly scrollable as long as there is memory -- and with 64-bit version of FireFox, it is almost forever. I have tried to do exactly the same in Google Chrome 64-bit, and it was perfect: it never slowed down and disintegrated like FireFox did. Chrome went to smoothly handle page with well over thousand of image/gif/video nodes, and it is a shame that supposedly prosumer-friendly FireFox fails in such fundamental low level function as rendering pages and can not do that.

Comment 1

[Timothy Nikkel (:tnikkel)]

Could you provide the example page(s) that you are using for testing?

Comment 2

[User Dderss]

Attachment: Nightly 64-bit renderer fails on big pages -- browser unable to properly render even simple text.png

I can not name specific account since it is a pron (:)), but any account that has thousands of entries in it and endless full-screen tile filling (not paginated) will do the trick. For example:
1) http://hundredsandthousandsofphotos.tumblr.com
2) or http://babyanimalposts.tumblr.com.

I just tried to browse for pictures in the second Tumblr account -- but not to the absolute uselessness of the browser, but to borderline state of it. In the attachment is what you get after browser's renderer starts to fail.

Comment 3

[User Dderss]

Attachment: Nightly 64-bit renderer fails on big pages -- unable to show elements of page.png

Previous screen shot shows "About Nightly" window, and how it gets corrupted. And this attachment shows the page itself already corrupted, with graphics objects not being rendered.

Both of those screen shots were made immediately one after another.

Comment 4

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/reply/question/etc.)]

Can you still reproduce the issue with a clean fresh new profile without any addons and plugins?

Comment 5

[User Dderss]

Attachment: Nightly 38 renderer dies on big pages -- new profile.png

There was no need for that, because I intentionally attached screenshot of corrupted "About Nightly" window -- on which none of plug-ins and extensions can have effect -- to show that it is low level issue, but I just repeated the experiment in absolutely new profile with the same result.

Comment 6

[User Dderss]

This time I got a bit further than previously, so, as you can see, even main FireFox pull-down tile/grid menu did not get rendered.

And I was not able to quit/restart the browser normally, because the rendered was unable to show me "Are you sure?"-type of dialogue prompt. I had to use task switcher application to manually kill FireFox.exe process.

Comment 7

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/reply/question/etc.)]

(In reply to User Dderss from comment #5)
> There was no need for that, because I intentionally attached screenshot of
> corrupted "About Nightly" window -- on which none of plug-ins and extensions
> can have effect -- to show that it is low level issue, but I just repeated
> the experiment in absolutely new profile with the same result.
Stylish, Greasemonkey and other similar addons can, so that's why I asked for this.



Please update your GPUs drivers to the latest ones and post graphics section info from about:support if it still happens.
e10s is enabled? and it still happens if disabled?
disabling HW Acc helps in any way?

Comment 8

[User Dderss]

1) e10s is disabled;
2) drivers are up to date (besides, Google Chrome works fine with the same drivers in such such scenario);

I will try the same test in e10s mode, too, and with hardware acceleration off and will write back.

____________________________________________________________________________

Graphics

Adapter Description	AMD Radeon HD 6900 Series
Adapter Drivers	aticfx64 aticfx64 aticfx64 aticfx32 aticfx32 aticfx32 atiumd64 atidxx64 atidxx64 atiumdag atidxx32 atidxx32 atiumdva atiumd6a atitmm64
Adapter RAM	2048
ClearType Parameters	Gamma: 2200 Pixel Structure: R ClearType Level: 100 Enhanced Contrast: 50
Device ID	0x6718
Direct2D Enabled	true
DirectWrite Enabled	true (6.2.9200.16571)
Driver Date	11-20-2014
Driver Version	14.501.1003.0
GPU #2 Active	false
GPU Accelerated Windows	1/1 Direct3D 11 (OMTC)
Subsys ID	00000000
Vendor ID	0x1002
WebGL Renderer	Google Inc. -- ANGLE (AMD Radeon HD 6900 Series Direct3D11 vs_5_0 ps_5_0)
windowLayerManagerRemote	true
AzureCanvasBackend	direct2d 1.1
AzureContentBackend	direct2d 1.1
AzureFallbackCanvasBackend	cairo
AzureSkiaAccelerated	0

Comment 9

[User Dderss]

I now tested three more variants:

1) e10s with hardware acceleration on
2) e10s with hardware acceleration off
3) non-e10 with hardware acceleration off

In all three of them pictures has failed to be rendered after some browsing through the cute animals account. Hardware acceleration being on or off did not cause difference. 

There was difference between e10s and non-e10s mode: the former is typically damn slow, so the experiments #1 and #2 with e10s on took longer time. But it seemed to be that even being that slow the browser was more responsive than in non-e10s when rendering failure was already evident.

Comment 10

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/reply/question/etc.)]

It happens also in 32 bit version of Firefox?
It's reproducible in Firefox 37 (Aurora), 36 (beta) and 35 (stable)?

Also finding a regression range will be needed probably next:
- first in mozilla-central builds:
https://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/
- deeper in mozilla-inbound builds next:
https://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/

Comment 11

[User Dderss]

32-bit version of FireFox runs out of memory before rendering can fail; it is not reproducible there. So it is only in 64-bit, hence only reproducible in Nightly.

I do not believe there is regression in this field, things were probably that bad since very beginning, but I can try to download an old 64-bit Nightly and run test there. But I can not find, say, Nightly 31 with proper installer. All I see is fully unfolded Nightly files. Are there installable old Nightly? Where they are stored?

Comment 12

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/reply/question/etc.)]

No need to test deeper, thank you very much for tests.
It's probably issue with these settings:

// Discards inactive image frames and re-decodes them on demand from
// compressed data.
pref("image.mem.discardable", true);

// Prevents images from automatically being decoded on load, instead allowing
// them to be decoded on demand when they are drawn.
pref("image.mem.decodeondraw", true);

and images aren't probably redownloaded in these circumstances,
when they're removed from the RAM or disc cache, when cache or image limit is about to be reached

Comment 13

[Timothy Nikkel (:tnikkel)]

Thanks for the example page.

I see two main problems:

1) we don't discard animated images, so they stick around and take up memory. That's bug 686905. We should hopefully be able to fix that soon, but right now there are a lot of regressions in imagelib that need to be addressed before we can do that.

2) Even when we are scrolled to a part of the page that has no animated images we do a lot of painting. We shouldn't be painting at all, and if we do happen to get asked to paint each paint should be relatively quick as it doesn't have to do much work. The profile I looked it indicated this wasn't the case.

Comment 14

[User Dderss]

I can confirm that FF 47 has the same issue. Just yesterday I have browsed through Facebook page with many videos, and after a while the browser just has stopped rendering graphics, showing blank areas where video's images should be.

And this time the content is not even animated, no gifs on Facebook page for videos, which are presented as static tiles, and they do not autoplay either.

Daniel, could you please check if it is possible to move this issue forward?

It might be depending on bug #1261315 that is about killing whole PluginContainer process that has happened to me after I have seen images failing to render.

Comment 15

[u480271]

Milan, is there someone better to look at this?

Comment 16

[Milan Sreckovic [:milan] (needinfo for best results)]

About:memory reports would be useful for these failures.

Comment 17

[Wayne Mery (:wsmwk)]

(In reply to Milan Sreckovic [:milan] from comment #16)
> About:memory reports would be useful for these failures.

Comment 18

[User Dderss]

Attachment: memory-report.json.gz

Memory report at the time when a Tumblr page has stopped rendering the pictures.

The page itself as separate window object grew to about 242 MB, but the amount of images loaded in it totalled about 1.5 GB or something.

Comment 19

[Milan Sreckovic [:milan] (needinfo for best results)]

Timothy, anything out of the ordinary in the memory report when it comes to images?

Dderss, I'm assuming this is a report from a latest/recent nightly?

Comment 20

[Timothy Nikkel (:tnikkel)]

Looks like there are a lot of animated images in the memory report. We can't discard animated images (like we can non-animated images) so even if we determine them to be not visible (and not likely to be visible soon) we still keep around their decoded data.

Bug 686905 is to enable discarding of animated images.

Comment 21

[User Dderss]

Yes, this is the latest FireFox update, though not nightly; it is FireFox beta (48). Those it would not make a difference anyway since it is a very old issue.

But it is not really tied to animated graphics specifically. I get exactly same result when browsing for a Facebook videos in channels that have countless number of them. Those are absolutely static tiles, nothing animated about them, but if you will browse the page for long enough, they will stop rendering, too.

Comment 22

[Timothy Nikkel (:tnikkel)]

(In reply to User Dderss from comment #21)
> Yes, this is the latest FireFox update, though not nightly; it is FireFox
> beta (48). Those it would not make a difference anyway since it is a very
> old issue.
> 
> But it is not really tied to animated graphics specifically. I get exactly
> same result when browsing for a Facebook videos in channels that have
> countless number of them. Those are absolutely static tiles, nothing
> animated about them, but if you will browse the page for long enough, they
> will stop rendering, too.

Well then that is a different issue.

Comment 23

[Milan Sreckovic [:milan] (needinfo for best results)]

Maybe a memory report for that case?

Comment 24

[User Dderss]

Milan, could you please explain if it will it make a difference? It should show the same blown up page with difference that in case of Facebook videos there will be only static images linked versus Tumblr page where the most of the images is also static, but it also has some of dynamic images (gifs). I mean such test takes time, so doing it just for sake of doing is not the best variant, unless there would be something essentially different.

Comment 25

[Milan Sreckovic [:milan] (needinfo for best results)]

Right - I'm trying to understand where the memory is going, and if its explainable or not.  Comment 20 says that the memory report submitted suggests memory is going into animated gifs.  Comment 21 is suggesting that we also have "using too much memory" situations when animated gifs are not in the picture.  So, where is it going?

Comment 26

[User Dderss]

Thanks; I will try it tomorrow and write back.

However, I am not sure what is meant by "too much memory". FF is 64-bit in my case, but even 32-bit would also be fine since whole Tumblr page and images took about 1.5 GB. So this glitch should not be memory issue; lets see.

Comment 27

[User Dderss]

Attachment: memory-report.json.gz

This is how memory reports looks like on a giant Facebook videos page, when graphics rendering fall apart.

Comment 28

[User Dderss]

Attachment: FireFox 48 failure to render graphics in a huge page (safe mode, e10s on) 1.png

Notice the upper part of the screenshot: there you can still see the last image tiles/thumbnails that FF could render.

Comment 29

[User Dderss]

Attachment: FireFox 48 failure to render graphics in a huge page (safe mode, e10s on) 2.png

On closer inspection, the issue is not purely of an image renderer: if you continue browsing further as if you want to load more video tiles/thumbnails further in the history, you can see that even text such as number of views stops to be rendered. 

And, more importantly, HTML URL blocks stop to appear, even the blank ones -- when I moved the mouse over the white areas near the bottom of the page (as in the screenshot), the status line did not show any URLs, and mouse pointer never not changed from an arrow to a grabbing hand icon.

Comment 30

[Timothy Nikkel (:tnikkel)]

The memory report doesn't seem that high, so I'm guessing that issue is not a memory issue but a graphics issue.

Comment 31

[User Dderss]

Correct; it did not seem to be memory issue, so this bug is categorized as graphics rendering issue. However, as I mentioned, it might be more than that since HTML itself stops appearing, not only its visual representation.

Comment 32

[Milan Sreckovic [:milan] (needinfo for best results)]

OK, I got this reproduced on the tubmler page from comment 0.  I was aimlessly scrolling, went to about:memory a few times, and eventually got it so that the browser was almost frozen - while in about:memory.  Closing that tab (eventually) unfroze things, but now I was stuck with the "missing images".  I can't tell for sure, but it seems that the missing ones are the ones that I've seen before in the page - in other words, they were there, got expired/unloaded at some point.

Doing the "view image in a separate tab" type of thing on one of those missing images produced a page without the image, but with the (barely visible) "The image ... cannot be displayed because it contains errors" message from ImageDocument::OnLoadComplete.  Reload does nothing, shift reload shows the image in that tab.

:tn, anything of interest here?  Image that was showing, and is now doing the "... it contains errors...".

Comment 33

[Timothy Nikkel (:tnikkel)]

If your surface cache was full we'd refuse to decode new images (and it'd appear as an error like that). I think the surface cache will be 1gb. Does that seem like what you saw?

Comment 34

[Milan Sreckovic [:milan] (needinfo for best results)]

Probably - I'm currently showing 890MB under images in about:memory.

Comment 35

[Milan Sreckovic [:milan] (needinfo for best results)]

Converting this bug to "remove animated images from image cache".  Will open another bug for "remove the 1GB image cache limit for 64-bit builds".

Comment 36

[Milan Sreckovic [:milan] (needinfo for best results)]

This should be fixed now that bug 686905 landed.