203667 - GDI objects overflow (Win) and X11 server size increase (Linux) when reloading images by javascript

Status: RESOLVED WORKSFORME

(Core Graveyard :: GFX, defect)

Description

[Daniel Kabs, reporting bugs since 2002]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3) Gecko/20030312
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3) Gecko/20030312

Hi there,

please enable Javascript in Mozilla and go to http://democam.mobotixserver.de.

The problem can be observed with Mozilla 1.3 and the latest alpha release
running either Windows 2000 or Linux.


Reproducible: Always

Steps to Reproduce:
1. Enable Javascript.
2. Open democam.mobotixserver.de
3. Use Taskmanager (Win2k) or 'top' to check the memory usage.

Actual Results:  
Windows 2000: The cpu usage is a squarewave, the memory usage show a sawtooth
form. This follows from the increasing allocation of GDI objects by mozilla
(according to task manager) and the ensuing garbage collection.

Linux: The X server size increases rapidly (at least at framerates of 12fps) and
reaches more than 300 Mbytes in a few minutes. Then the system locks up, the X
server freezes or is killed by the linux virtual machine.



Expected Results:  
The memory consumption should be constant as should the cpu usage.

Comment 1

[Sameer D. Sahasrabuddhe]

Works for me on:

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3) Gecko/20030312
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4a) Gecko/20030424 Mozilla Firebird/0.6

The CPU is usually 95% idle or more, and memory usage remains constant. No
adverse effect on XFree86.

Comment 2

[Hermann Schwab]

Seeing this on Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.4b) Gecko/20030427
Used a tool from bug 199443 to look at GDI resources, and BITMAP was increasing
with each reload. 

Bug 199443 GDI leak when table cell contains an image, and text is received in
multiple packets (testcase availible)

GDIusage.exe:
http://bugzilla.mozilla.org/attachment.cgi?id=121792&action=view

Comment 3

[Daniel Kabs, reporting bugs since 2002]

Hi Ashant,

I should have added that these effects only occur if you really 
reload the images very fast, e.g. with 12fps. 

I guess, your CPU load is low because your network bandwith limits 
the refresh rate to less than 1fps.  Mind you, if you view the
page using ISDN connection you will only see one image for every 3 seconds
as one image is about 20kbytes big.

Comment 4

[Hermann Schwab]

I´m seeing this on ISDN, with tool GDIusage.exe.
Number of Bitmaps is growing, from 334 to about 425, then it is reset to 334
again, at the same time the GDI resources are shrinking from 44% to below 40%,
the jumping to 44% again. If this garbage collection is controlled by a timer,
GDI would have been totally used by a highspeed connection before GC recycles.

Comment 5

[Hermann Schwab]

testing on DSL, 12 fps, Win98SE, 512MB, XP1600+
Number of allocated bitmaps is increasing for about 90 seconds to about 210,
the jumping to about 70 and staying there for about 20 seconds,
then this cycle starts again.
Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.4b) Gecko/20030425

Comment 6

[Daniel Kabs, reporting bugs since 2002]

Attachment: Snapshot of Taskmanager (W2k german)

This is a snapshot of the taskmanager to illustrate the described behaviour.

Comment 7

[Daniel Kabs, reporting bugs since 2002]

I forgot to add: The snapshot was taken on a PC that has a P4 CPU running at
1700MHz with 512MBytes of RAM.

Comment 8

[Benjamin Smedberg]

Isn't this an expected function of the bitmap cache?

Comment 9

[Hermann Schwab]

Bug 203698 memory consumption on reloading images in DHTML
looks like a DUPE, besides beeing in domain: Components: DOM HTML

Comment 10

[Daniel Kabs, reporting bugs since 2002]

Hi Benjamin,

I reckon that inflating the X server's size to more than a few
hundred MBytes is not an "expected function".

The bitmap cache is one thing, but a capable garbage collection
the other.

Cheers,
Daniel

Comment 11

[Gilles Durys]

CPU usage is at 30%
Memory usage for X and Mozilla stays stable (90MB for X, 45MB for Mozilla)
2003051222 trunk linux P3-800 768MB

Comment 12

[Daniel Kabs, reporting bugs since 2002]

Regarding Linux and the X11 server's increase in memory consumption:

I recon that there have to be two concurrent processes involved:
First mozilla "making" the X server cache some image data and
second a garbage collection insided the X Server cleaning
up this cache.

On my computer, the latter process is to slow (or the first too fast) and thus
my system hangs because 256 MBytes of swap have been filled up by the bloated
X11 server before the garbage collection does its job.

As the testcase was only a small image (320x240) this may not be the case on a
remote connection. Thus I've increased the resolution to 640x480. The picture
now is about 35KBytes. When checking the testcase make sure your connection
bandwidth is adequate.

Cheers
Daniel

Comment 13

[Daniel Kabs, reporting bugs since 2002]

Attachment: Snapshot of ksysguard: memory usage (Linux)

Created using ksysguard (Updated frequency=2s, vertical grid unit=1 minute)

Watching http://democam.mobotixserver.de using a "slow" link.

You can observe the same effect as on a windows, i.e. the memory usage
shows a sawtooth form (probably the result of a garbage collection
running at regular intervals).

Comment 14

[Jo Hermans]

related to bug 193865 ?

Comment 15

[Daniel Kabs, reporting bugs since 2002]

Sorry for griping, but today my linux system again "killed"
itself by swapping to death. :-(

I left the testcase page open with only 1fps selected for one hour.
When I returned to the computer the X server did not react to mouse
movements. Only the hard drive was very busy swapping data in and out.

I'll try to investigate further and to make the testcase more
"reliable".

Cheers
Daniel

Comment 16

[Daniel Kabs, reporting bugs since 2002]

Attachment: Linux: Snapshot of "top" (sorted by memory usage)

Refreshing images at 1fps, I managed to stop the images from being reloaded
just before all memory and swap was used up. Then I took the snapshot of top
showing
the X Servers memory consumption. 

After reloading the page the memory was freed immediately.

Comment 17

[Daniel Kabs, reporting bugs since 2002]

Attachment: Linux: ksysguard showing memory/swap usage

Created using ksysguard (Updated frequency=2s, vertical grid unit=1 minute)

Watching http://democam.mobotixserver.de on a fast link
and with two tabs opened in Mozilla.

The testcase is the "active" tab. The physical memory usage shows the 
sawtooh form as before, probably as a result of a periodically running
garbage collection that frees X Server memory.

When the testcase page was hid (by activating the other tab),
the memory is not released any longer instead swap memory usage 
increases dramatically. I had to abort the test when the swapping adversely
affected system stability.

Comment 18

[Lorenzo Colitti]

Reporter, I can't reproduce this on a 1.4 branch build dated 20030612 on WinME.

I've had it open for about a quarter of an hour, and System and GDI resources
are pegged at 72% free. On the other hand, the testcase in bug 159298 can take
my resources down to 5% in a matter of seconds.

Can you still reproduce the bug?
Maybe it happens on Linux only?

Comment 19

[Jim Dunn]

I believe this is "works as expected".
ImageLib, caches the decoded image in the memory cache and uses
the URL of the image as the key.  (currently every image on windows
also *stores* one GDI HBITMAP).

this url http://democam.mobotixserver.de/ is dynamically updating an
image, but in doing so changes the image url.
i.e.
 http://democam.mobotixserver.de/record/current.jpg?rand=309914
 http://democam.mobotixserver.de/record/current.jpg?rand=309915
 ...
and each decoded image is about 920k bytes (roughly 1 Meg).

Now depending on what your memory cache size is set to (mine is 32Meg)
you will *chew* up all of those 32Meg in a couple of minutes and at that
point, you will start bumping images out of the cache, so your memory
cache will eventually remain constant.  Also, as the cache is filling
you will chew up approx 32 GDI resources, but again as the memory cache
starts kicking out an image for each new one, we will *recycle* the GDI
resource, so GDI resources will stabilize as well.

If you lower your memory cache, less resources will be chewed up and
you will reach a stable period more quickly.

Linux should be roughly the same as windows except no GDI resources
are consumed.  Not sure why you are hanging, since I can load this
webcam and don't hang.  You can try disable'ing your memory cache
(NOTE this is not a long term solution, just for fact finding).
goto about:cache
scroll down till you see *browser.cache.memory.enable"
dbl click on it and change true to false.

To see memory cache goto about:cache?device=memory
(NOTE: with mem cache disabled, this about:cache won't work)

Each spike in the CPU is probably the decoding of the image + the
memory cache scrubbing.

There are a dozen or so similar bugs about memory cache and gdi leaks.
This isn't either (except for the fact that you are crashing on Linux).
Mostly this is just side effects of how ImageLib and the Memory cache
interact.  

Comment 20

[Daniel Kabs, reporting bugs since 2002]

Attachment: Snapshot of Taskmanager (W2k)

Indeed, the behaviour and interaction with the operating system has changed:
With Moz1.4RC2 the memory usage stays stable.

I added a snapshot of taskmanager showing memory and processor usage having
the testcase open:

Running Mozilla 1.3 the cpu usage is a squarewave, the memory usage shows a
sawtooth form (left, red field).

Using Mozilla 1.4 RC2 (right, green field) memory and cpu load stays
stable.

Comment 21

[Daniel Kabs, reporting bugs since 2002]

Jim,

fyi: my Mozilla is set to use a memory cache of 50000 KB. I could not find
anything configurable (like *browser.cache.memory.enable) in about:cache.
Maybe because I am using a release build.

At least it now wfm on Windows :-) I'll still have to try the Moz1.4RC2 on Linux.

Comment 22

[Jim Dunn]

oops, to disable memory cache
about:config and scroll down till you see browser.cache.memory.enable
double click and set it to false

NOTE: this is just for testing, it is not a solution

Comment 23

[Daniel Kabs, reporting bugs since 2002]

Just downloaded Mozilla 1.4 RC2 for Linux

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030617

and tried the testcase. Except for an increase in "cached system memory" the
X server's memory consumption stayed stable at about 100 MB.

Seems, this bug has gone. Resolved wfm!

Jim, thanks for pointing me to about:config which shows quite a
bunch of interesting features to configure :-)

Cheers folks.
Daniel
Germany