366548 - Increased GDI usage (leading to repainting problems) when viewing sites with lots of images

Status: VERIFIED FIXED

(Core :: Graphics, defect)

Description

[Steve England [:stevee]]

1. New profile, start firefox
2. Go to http://www.knitemare.org/cats/index.php?type=all (441 pictures of cats)
3. Once the page has finished loading, open a new tab and observe that it redraws funny. (New tab content not rendered so you see the previous tab's content, firefox UI rendering problems, and you'll probably see UI rendering problems in other apps too)

Works ok:
Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9a1) Gecko/20060809 Minefield/3.0a1 ID:2006080907 [cairo]

Rendering problems:
Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9a1) Gecko/20060810 Minefield/3.0a1 ID:2006081004 [cairo]

Bonsai regression range:
http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=PhoenixTinderbox&branch=HEAD&branchtype=match&dir=&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=explicit&mindate=2006-08-09+06&maxdate=2006-08-10+05&cvsroot=%2Fcvsroot

I suspect bug 342366 or bug 343655.

There are a few bugs on rendering screwing up due to GDI usage; bug 204374 and bug 284978 are two such bugs. This bug is not about the rendering problem per se; it's more about the regression in firefox's behaviour between 20060809 and 20060810.

With 2006080907 (no redrawing problem), task manager reports:
276MB (Mem Usage), 479 (VMem), 1033 (GDI Objects)

With 2006081004 (redrawing problem), task manager reports:
39MB (Mem Usage), 28 (VMem), 1539 (GDI Objects)

So memory usage has dropped dramatically, at the expense of GDI Objects used. It doesn't appear to be a cairo issue since both builds that show the regression range use cairo.

Is it expected that GDI usage should increase between these builds? (The GDI increase seems to be roughly 1 GDI object per image rendered. Coincidence?)

Taking the liberty of CC'ng Vlad and Stuart.

Comment 1

[Steve England [:stevee]]

For comparison:

Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9
366MB (Mem Usage), 356 (VMem), 137 (GDI Objects)

Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.2pre) Gecko/20070102 BonEcho/2.0.0.2pre
374MB (Mem Usage), 364 (VMem), 178 (GDI Objects)

Comment 2

[Martijn Wargers (dead)]

This page also suffers from this bug:
http://www.igra4kite.com/catalog_products_with_images.php?osCsid=10854f1e511648b4ed9cb4abb9975b56
I found this on bug 360289, which is a duplicate of bug 359533.

Comment 3

[Neil P. Pena]

I experience this bug under Windows XP also, using the following build:

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a2pre) Gecko/20070129 Minefield/3.0a2pre

So, "Windows 2000" in the "OS" field is too specific then?

Comment 4

[Ryuichi KUBUKI]

Sometimes Windows Media Player can't play Windows Media video files correctly (without sound, can't skip, etc.) due to this problem. Closing Firefox always solves it.

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a4pre) Gecko/20070406 Minefield/3.0a4pre ID:2007040605 [cairo]

Comment 5

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

So much the same problem was bug 374272 -- and it was worked around by stopping to allocate GDI objects once we reach a certain point and go a slower route.  The fix happened on 0401, so I'm not sure what the last window media player issue is -- KL, are you talking about windows media player embedded into firefox?  Or a separate instance of the WMP app?  Does the firefox browser have any problems redrawing?

Comment 6

[Justin Kerk]

Well bug 374272 notwithstanding, current Minefield builds regularly stop repainting for me on Windows XP when I open even modestly image-heavy pages (I tend to have a lot of tabs open, which doesn't help). The smilies page in that bug doesn't trigger it though, so maybe it has something to do with a relatively small number of high-res images instead of tons of tiny images? I seem to recall reading that GDI space is limited by the total number of pixels involved, not the number of objects.

Comment 7

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Can you give me a testcase?

Comment 8

[Ryuichi KUBUKI]

(In reply to comment #5)
> So much the same problem was bug 374272 -- and it was worked around by stopping
> to allocate GDI objects once we reach a certain point and go a slower route. 
> The fix happened on 0401, so I'm not sure what the last window media player
> issue is -- KL, are you talking about windows media player embedded into
> firefox?  Or a separate instance of the WMP app?  Does the firefox browser have
> any problems redrawing?

A standalone instance of WMP which was not launched by FF. It's not limited to WMP, if the GDI usage gets over 7000 by using FF for more than 3 hours or so, other apps suffer too (like Explorer begins to lose menus) on a 1GB RAM machine. I haven't seen it on a 2GB RAM machine.

Comment 9

[Steve England [:stevee]]

(In reply to comment #7)
> Can you give me a testcase?

I've got the original page stored on my HD, but no where to put it (~42MB); it's essentially one big page with 425 jpg's on it (Image total ~40.8MB)

Comment 10

[Radek 'sysKin' Czyz]

I can confirm that browsing large images (rather than many smaller ones) is often the direct cause of rendering problems.

However, every time I notice high GDI usage I can restart minefiled and restore all windows/tabs (with restarter extension). Then, minefield will successfully display the same webpages with about half of GDI objects used. Can someone tell me if this is normal, or it might indicate a leak and I should pursue steps to reproduce?

Comment 11

[Bill Gianopoulos [:WG9s]]

(In reply to comment #10)
> However, every time I notice high GDI usage I can restart minefiled and restore
> all windows/tabs (with restarter extension). Then, minefield will successfully
This could be caused by GDI usage on pages in the bfcache.  If you have altered the preferences controlling the number of pages to cache from the default, this could be a problem you are causing for yourself.

Comment 12

[Radek 'sysKin' Czyz]

(In reply to comment #11)
> This could be caused by GDI usage on pages in the bfcache

Yes this could be it.
Also, if you open bookmarks menu (or sub-menus) the number of GDI objects goes up by about two per item opened and then just stays there forever. Are they cashed or something?

Comment 13

[Steve England [:stevee]]

(In reply to comment #7)
> Can you give me a testcase?

Also, the link in comment 2 shows the problem

Comment 14

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Over to me for investigation.

Comment 15

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Marking blocking for now until I investigate and see where the memory/GDI objects are going.

Comment 16

[Ryuichi KUBUKI]

I don' know why but this build seems good at restraining the GDI objects usage to 2400 and the RAM usage to 200MB~. Did anything change in-between?

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a7pre) Gecko/2007071018 Minefield/3.0a7pre ID:2007071018 [cairo]

Comment 17

[bugzilla.mozilla.org]

I reported https://bugzilla.mozilla.org/show_bug.cgi?id=381578 which seems a duplicate?

i provided some observations and a flexible test case

Comment 18

[Peter van der Woude [:Peter6]]

Just to show how badly the current FF3.0 is messed up (Works fine in FF2.0/IE6-7/Opera).
Try http://pvanderwoude.com/timelapse/clouds/timelapse.html
(an array of 486 10 kb large jpegs)
Chrome, content, tabs, it's all misrendered.

Comment 19

[Radek 'sysKin' Czyz]

(In reply to comment #18)
> Just to show how badly the current FF3.0 is messed up

Beautiful. There's only a single image displayed. I bet you can even remove it. Everything else is 486 times new Image(), which produced ~1600 GDI objects while not taking any memory (usage was 31 MB).

Comment 20

[Radek 'sysKin' Czyz]

Attachment: testcase

Tastcase based on Peter's page.
Firefox 2.0 does not create any GDI objects when reading images into an array.
Minefield 2007071717 creates exactly two GDI objects for each image loaded into an array.
No images are ever displayed. Does not happen if images have the same url. Does not work with data:// urls. Does not happen if image fails to load (404 or something).

Comment 21

[Radek 'sysKin' Czyz]

Attachment: Better testcase

Does not need to be an array and Image() objects don't need to be remembered anywhere.

By the way the testcase doesn't cause display problems because image used is small (the smallest I could find on mozilla.org/). Larger image will cause rendering problems, as per comment #18.

Comment 22

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Ok, I can reproduce this quite well using peterv's testcase, thanks!

There are a few problems that are going on:

1) We're creating at least 2, sometimes 3 GDI objects per image.  This is because we use a cairo win32 surface for each DIB or DDB, and it always creates a DC.  For images, this DC is only needed when rendering; we should be able to create it on demand.

2) Why are we creating 3 GDI objects in some cases?  We seem to for jpg and png images, never for gif images.

3) The limit seems to be not just a hard GDI object limit, but also a memory limit on the GDI side (which is why the testcase in the previous comment doesn't cause misrendering).

Comment 23

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Attachment: fix GDI usage

Ok, so there were a number of things going on that are fixed by this patch.  The core problem doesn't actually have anything to do with GDI object count, but instead with video memory -- CreateCompatibleBitmap allocates video/kernel memory, and we can easily run out of that on systems pre-Vista.  (On Vista this is virtualized, so this isn't a problem).

So:

- correctly handle error return from CreateCompatibleBitmap in cairo, and return OOM
- avoid allocating and keeping around region handles unless necessary
- limit DDB usage to images smaller than 512x512 (was 1024x1024)
- limit the total memory usage of DDB images to 8MB (calculated as w*h*4).

This works fine, but the only thing I'm not happy about is the 8MB limit -- it's really low, but given that video cards have a wide range of available video memory, I'm not sure what else we can do.  It's not enough to just handle out of memory (though, see below), because the memory is system-global -- other apps are affected if firefox uses up all the video memory available for bitmaps.

One thing we could do for the future is to handle the out of memory condition by de-optimizing all DDBs back into DIBs, but to do that without doing further GDI allocations requires Federico's decoded-image-throwaway patch.  That patch will also significantly help this problem as well, by not keeping DDBs around unless they're actually needed.

Another thing we could do with the de-optimize patch is to do 2-level deoptimization on win32 -- keep both the DDB and the DIB around, and get rid of the DDB aggressively (after a minute or so), falling back to the DIB (which would use the normal image timer).  But that may be totally unnecessary.

Comment 24

[Scott A.]

Please get this into the trunk soon, this is one of those show stopper bugs that prevent me from effectively(want to) test the current nightlies.

Comment 25

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Attachment: slightly different limit

More testing showed that the memory limit doesn't have anything to do with video memory; I can't actually figure out what it relates to.  On my system here with an intel shared memory thing and 2GB of system ram, with the video using between 16MB and 128MB, the video memory usage never went above 23MB, and I started getting errors after allocating more than around 220MB of bitmaps.

So, this patch just a) caps the total used to 64MB; b) caps each individual image to 4MB.  There are no width/height checks -- if a particular driver can't create a bitmap for a particular size, we'll handle that error as normal, without contributing to system instability due to running out of memory.

At some point we need to figure out how to make that 64MB limit dynamic, but I think it should be safe anywhere Fx3 runs on.

Comment 26

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Attachment: use PRPackedBool instead of recording the size

minor change; this is what's going in

Comment 27

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Checked in.

Comment 28

[Steve England [:stevee]]

Using my saved 441-images-of-cats page:

Before patch
- 60 MB Memory Usage (48MB VMem) / 1876 GDI Objects
  Scrolling whilst page loading fine, but rendering screws up

After patch
- 390 MB Memory Usage (418MB VMem) / 1190 GDI Objects
  Scrolling whilst page loading can be poor, rendering doesn't screw up

So the rendering problem appears to be fixed.


Same kind of stats with Peter's testcase in URL:
- Before Patch: 40 MB Memory Usage (30 MB VMem) / 1571 GDI Objects used
- After Patch: 342 MB Memory Usage (410 MB VMem) / 1195 GDI Objects used

Comment 29

[Steve England [:stevee]]

So to conclude.

With 20071003_1357_firefox-3.0a9pre.en-US.win32 (pre bug 366548):
- 441 cats shows rendering problems
- Peter's testcase (URL) shows rendering problems
- Martijn's link in comment 2 shows rendering problems.

With 20071003_1428_firefox-3.0a9pre.en-US.win32 (post bug 366548):
- 441 cats shows no rendering problems
- Peter's testcase (URL) shows no rendering problems
- Martijn's link in comment 2 shows no rendering problems.

So I hereby set this bug to VERIFIED. Thanks Vlad!

Comment 30

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Yep, we're definitely going to be using much more memory for the firefox process since at some point we're going to stop shoving things off into the kernel/GDI memory space.  The downside is that this may show up as "increased firefox memory usage", even though it's not really.  But, glad to see that things are fixed!

Comment 32

[bugzilla.mozilla.org]

this fix fixes the rendering issue reported in bug 381578 - but at the cost of much higher memory consumption. Much higher compared to Firefox 2 in the same situation. See comments in bug 381578 .

Comment 33

[Steve England [:stevee]]

jens, try reading comments 28, 29 and 30 in this bug..

Comment 34

[bugzilla.mozilla.org]

yes, but comments 28, 29 and 30 do not mention that "Much higher compared to Firefox 2 in the same situation." is to e expected? 

Comment 35

[Martijn Wargers (dead)]

Jens, please file a new bug for that (and mention the bug number here).

Comment 36

[bugzilla.mozilla.org]

ok, filed bug 398983