110048 - Slow memory leak with looping graphics

Status: RESOLVED WORKSFORME

(Core :: Graphics: ImageLib, defect)

Description

[Con Kolivas]

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.5+) Gecko/20011113
BuildID:    2001111314

This page (and all the radar loops on this weather site) loops through 4 images
on a radar image. In any version (previous and latest build) of mozilla I have
tried it on, it will continually use up memory till the computer crashes. The
faster the loop speed and the network connection, the faster mozilla will
consume memory.

Reproducible: Always
Steps to Reproduce:
1.Browse to URL (example) http://mirror.bom.gov.au/products/IDR022.loop.shtml
2.Leave it on the site
3.Increase the loop speed to reproduce it faster (not necessary but makes
reproduction easier)
4.Wait and watch memory usage

Actual Results:  Memory usage increases with time. It seems to cache every next
image.

Expected Results:  Should only cache the last four images and not use up any
more memory.

Comment 1

[Ashley Bischoff (blog at handcoding.com)]

WFM 2001111303 on Win2k; Mozilla's memory usage stays at a steady 143 MB.

-> ImageLib, but punt as needed

Comment 2

[Jo Hermans]

I see it in 2001110908 too (MacOS 9.2.1). Memory usage quickly jumped from 35MB
to 73MB. When I forced a GC by opening a new window and closing it again (while
the loop was still running), the memory dropped back, but started to climb
again. When I closed the window (killing the loop), and forced another GC, the
memory was back to 37MB.

Comment 3

[Diego Biurrun]

Confirming on yesterdays CVS build under Linux.

Comment 4

[Diego Biurrun]

WFM, my memory usage stays normal with 0.9.6.

Comment 5

[Con Kolivas]

Definitely still occurs in 0.9.6 with my linux build.

Comment 6

[Diego Biurrun]

Con, did you try a fresh profile?

Comment 7

[Con Kolivas]

Yes, I just tried it now and it still occurs. It is more pronounced on the 256km
loop than the 128km loop, but definitely keeps soaking up memory with a
completely fresh profile.

Comment 8

[Diego Biurrun]

Just checked back and I also leak memory. Don't know why I did not notice
earlier today...

Comment 9

[eberry]

*** Bug 115474 has been marked as a duplicate of this bug. ***

Comment 10

[Con Kolivas]

Confirmed as also occuring on mozilla 0.9.8 on Windows NT4

Comment 11

[Diego Biurrun]

Please retest this with a nightly, I do not see the leak anymore with my fresh
CVS build.

Comment 12

[Diego Biurrun]

Hmm, still there with Linux 2002020821, though not as pronounced as it used to be.

Comment 13

[Nivedita (gone)]

Tested on WindowsNT and Linux with the tip of cvs dated 19th feb'02. I am 
unable to reproduce the bug. 

Comment 14

[Nivedita (gone)]

Tested on WindowsNT and Linux with the tip of cvs dated 19th feb'02. I am 
unable to reproduce the bug.

Comment 15

[Diego Biurrun]

Just compiled from fresh CVS.  Be patient, memory usage _will_ increase.  This
has been slowed, but it is not gone.  REOPENING.

Comment 16

[Stuart Parmenter]

I was seeing something leak, however, I don't believe it was images.  I will try
and run this through purify.

Comment 17

[Nivedita (gone)]

I have  tried the following but could not reproduce the bug.
a) ran it on Linux overnight 
   top for the bug#110048 on Linux at 9:40 pm on 26 th feb'02
   16969 nivedita   9   0 35424  34M 25032 S     5.8 13.9   0:27 mozilla-bin
   top for linux at 9:00 AM on 27 th feb'02
   16969 nivedita   9   0 36164  35M 25040 S     5.7 14.2  39:17 mozilla-bin
b) ran it on WINNT as well and did not see any significant change in memory.
c) ran it through purify but could not see any potential leaks in the image lib.

pavlov,
Did you have a chance to run purify other this url? 

Comment 18

[Diego Biurrun]

Nivedita asked me to verify this again, so I downloaded Linux 2002022821 and
tried again.  I untarred it and ran it vanilla without any plugins.
Steps to reproduce:

1. Open URL.
2. Use the ">" control to the right of the map for making the animation quicker
   and make it absurdely fast.  All seems fine.
3. Now slow down the animation via the "<" button.
4. Mozilla starts leaking big time.

I ran top in the background to measure memory consumption.  The memory lead
seems almost gone with the fast animation, I could barely reproduce it.  With
the slowdown it does get very visible, though.

I run an up to date Debian testing system with XFree 4.1.0 and glibc 2.2.4.  If
you need further details or assistance, contact me, I will gladly run any tests
you might need.  I suspect that purify is a commercial product, otherwise I
might try to run it myself.

Con, Johan, the others on the CC list, please try to verify this and provide
some steps to reproduce or tell us if you do not see the problem.

Comment 19

[Diego Biurrun]

I use the adzapper filtering proxy.  It does somehow exacerbate the problem. 
Just tried again with Mozilla and a terminal running top as the only processes
in my GNOME environment and without adzapper.  The Leak still occurs, albeit
less pronounced.  Just play around a bit with the speed controls and memory
usage will go up.  Nivedita, maybe installing adzapper (it's free software) will
help you see the problem, but it does occur without it.

http://www.zipcon.net/~adamf/adzapper/

Comment 20

[scottputterman]

Moving Netscape owned 0.9.9 and 1.0 bugs that don't have an nsbeta1, nsbeta1+,
topembed, topembed+, Mozilla0.9.9+ or Mozilla1.0+ keyword.  Please send any
questions or feedback about this to [e-mail].  You can search for
"Moving bugs not scheduled for a project" to quickly delete this bugmail.

Comment 21

[Con Kolivas]

I am no longer seeing the problem (linux latest 0.9.9 build 2002030414)

Comment 22

[Diego Biurrun]

Yes, I just tried to reproduce this with my current CVS build, no memory leak
anymore.  Hooray!  Marking WORKSFORME.

Con, if you still cannot reproduce this following my instructions, please mark
this VERIFIED.  Thanks.

Comment 23

[Con Kolivas]

Woohoo it's gone. I am unable to reproduce this bug any more :-)

Comment 24

[Con Kolivas]

Oh dear. This has come back with a vengeance in 1.0rc2 on linux. I haven't
tested it on any other platform but it's seriously sucking memory again (build
2002051009)

Comment 25

[Diego Biurrun]

Yes, this leaks with 1.0RC2 under Linux.  I also tried 2002051809 from the 1.0
branch and I get increased memory usage after twiddling with the speed settings
a little bit.  It does not appear to grow completely out of bounds, however.

Comment 26

[Moiz Dohadwala]

wfm win2k sp2, mozilla 1.1a 2002061104

Comment 27

[Cees T.]

WFM with trunk 2002062304 on WinME.

Comment 28

[Cees T.]

Ok, it does keep downloading the images from the server and allocating memory
for them. In 10 minutes, allocated data rose from 64MB to 111MB, in memory data
from 54MB to 88MB, and in use data dropped from about 32MB to 24MB.

After about 10 minutes, mem usage dropped back to 66MB, 53MB, and 24MB.
After 20 minutes: 67, 54, and 25MB, respectively.

I increased, stopped, then decreased the loop speed (set to bounce), checking
memory usage with WinTop. i'm using a PIII900, with 256MB RAM, cable connection,
Mozilla memory cache at 4096KB, and disk cache at 12000KB.

Here's a snippet from The Proxomitron's log window after 18 minutes of
continuous testing:

+++GET 5292+++
GET /radar/IDR022.20020623151144.gif HTTP/1.1
Host: mirror.bom.gov.au
User-Agent: Mozilla/5.0 (Windows; U; Win 9x 4.90) Gecko/20020608 Netscape/7
Accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,video/x-mng,image/png,image/jpeg,image/gif;q=0.2,text/css,*/*;q=0.1
Accept-Language: en-us, en;q=0.50
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1, utf-8;q=0.66, *;q=0.66
Keep-Alive: 300
Connection: keep-alive
+++CLOSE 5291+++

+++GET 5293+++
GET /radar/IDR022.20020623145241.gif HTTP/1.1
Host: mirror.bom.gov.au
User-Agent: Mozilla/5.0 (Windows; U; Win 9x 4.90) Gecko/20020608 Netscape/7
Accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,video/x-mng,image/png,image/jpeg,image/gif;q=0.2,text/css,*/*;q=0.1
Accept-Language: en-us, en;q=0.50
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1, utf-8;q=0.66, *;q=0.66
Keep-Alive: 300
Connection: keep-alive

+++RESP 5292+++
HTTP/1.1 404 Not Found
Date: Sun, 23 Jun 2002 15:49:47 GMT
Server: Apache/1.3.9 (Unix)
Last-Modified: Fri, 22 Mar 2002 01:39:03 GMT
ETag: "2f1af6-1669-3c9a8b37"
Accept-Ranges: bytes
Content-Length: 5737
Content-Type: text/html
Match 5292: Tooltip fix (add titles to lone alts)
+++CLOSE 5292+++

+++RESP 5293+++
HTTP/1.1 404 Not Found
Date: Sun, 23 Jun 2002 15:49:47 GMT
Server: Apache/1.3.9 (Unix)
Last-Modified: Fri, 22 Mar 2002 01:39:03 GMT
ETag: "2f1af6-1669-3c9a8b37"
Accept-Ranges: bytes
Content-Length: 5737
Content-Type: text/html

+++GET 5294+++
GET /radar/IDR022.20020623151144.gif HTTP/1.1
Host: mirror.bom.gov.au
User-Agent: Mozilla/5.0 (Windows; U; Win 9x 4.90) Gecko/20020608 Netscape/7
Accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,video/x-mng,image/png,image/jpeg,image/gif;q=0.2,text/css,*/*;q=0.1
Accept-Language: en-us, en;q=0.50
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1, utf-8;q=0.66, *;q=0.66
Keep-Alive: 300
Connection: keep-alive
Match 5293: Tooltip fix (add titles to lone alts)
+++CLOSE 5293+++

+++GET 5295+++
GET /radar/IDR022.20020623151926.gif HTTP/1.1
Host: mirror.bom.gov.au
User-Agent: Mozilla/5.0 (Windows; U; Win 9x 4.90) Gecko/20020608 Netscape/7
Accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,video/x-mng,image/png,image/jpeg,image/gif;q=0.2,text/css,*/*;q=0.1
Accept-Language: en-us, en;q=0.50
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1, utf-8;q=0.66, *;q=0.66
Keep-Alive: 300
Range: bytes=4149-
If-Range: "37630c-3a54-3d15e6f8"
Connection: keep-alive

+++RESP 5294+++
HTTP/1.1 404 Not Found
Date: Sun, 23 Jun 2002 15:49:48 GMT
Server: Apache/1.3.9 (Unix)
Last-Modified: Fri, 22 Mar 2002 01:39:03 GMT
ETag: "2f1af6-1669-3c9a8b37"
Accept-Ranges: bytes
Content-Length: 5737
Content-Type: text/html
Match 5294: Tooltip fix (add titles to lone alts)
+++CLOSE 5294+++

+++RESP 5295+++
HTTP/1.1 206 Partial Content
Date: Sun, 23 Jun 2002 15:49:49 GMT
Server: Apache/1.3.9 (Unix)
Last-Modified: Sun, 23 Jun 2002 15:19:20 GMT
ETag: "37630c-3a54-3d15e6f8"
Accept-Ranges: bytes
Content-Length: 10783
Content-Range: bytes 4149-14931/14932
Content-Type: image/gif
+++CLOSE 5295+++

Comment 29

[Diego Biurrun]

This does indeed seem to get exacerbated by using a proxy.  I tried again with
adzapper and it leaks.  It's not as fast as it used to, it may take some minutes
before it gets noticeable.

Comment 30

[R.K.Aa.]

*** Bug 157691 has been marked as a duplicate of this bug. ***

Comment 31

[John Lemcke]

I am seeing this problem on Linux with Mozilla 1.1a (Build 2002061108)

The interesting thing is that it is not the mozilla process which is chewing up
the memory; it is the X server.
This suggests that the problem may be in the allocation of X server-side
resources such as pixmaps.

Below are before and after output from top (sorted by memory use).

---------------------------------------------------------------------------

 11:03am  up 23:52, 11 users,  load average: 0.05, 0.03, 0.00
108 processes: 105 sleeping, 3 running, 0 zombie, 0 stopped
CPU states:  2.9% user,  4.5% system,  0.0% nice, 92.4% idle
Mem:   191088K av,  161800K used,   29288K free,       0K shrd,    3248K buff
Swap:  297160K av,   54140K used,  243020K free                   78568K cached
 
  PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
 1088 root      12   0 59052  49M  7664 R     1.3 26.4   7:50 X
11373 john       9   0 35404  34M 14772 S     0.0 18.5   1:53 mozilla-bin
11375 john       9   0 35404  34M 14772 S     0.0 18.5   0:00 mozilla-bin
11376 john       9   0 35404  34M 14772 S     0.0 18.5   0:01 mozilla-bin
11377 john       9   0 35404  34M 14772 S     0.0 18.5   0:00 mozilla-bin
11379 john       9   0 35404  34M 14772 S     0.0 18.5   0:01 mozilla-bin
11443 john       9   0 35404  34M 14772 S     0.0 18.5   0:00 mozilla-bin

---------------------------------------------------------------------------

 11:07am  up 23:56, 11 users,  load average: 1.05, 0.48, 0.18                   
109 processes: 108 sleeping, 1 running, 0 zombie, 0 stopped
CPU states:  9.5% user, 12.9% system,  0.0% nice, 77.5% idle
Mem:   191088K av,  188680K used,    2408K free,       0K shrd,     280K buff
Swap:  297160K av,  170388K used,  126772K free                   60636K cached
 
  PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
 1088 root      15   0  219M 115M  3716 S     4.1 61.8   8:01 X
11373 john      18   0 28132  15M  8412 S     6.8  8.3   2:04 mozilla-bin
11375 john       9   0 28132  15M  8412 S     0.0  8.3   0:00 mozilla-bin
11376 john       9   0 28132  15M  8412 S     0.0  8.3   0:01 mozilla-bin
11377 john       9   0 28132  15M  8412 S     0.0  8.3   0:00 mozilla-bin
11379 john       9   0 28132  15M  8412 S     0.0  8.3   0:01 mozilla-bin
11443 john       9   0 28132  15M  8412 S     0.0  8.3   0:00 mozilla-bin

Comment 32

[Matt Foster]

This bug can be reproduced at some U.S. .gov and .edu weather sites.  See bug
127825, which appears to be a dupe of this bug.  I also see the added behavior
of the animation becoming populated with duplicate images...quite annoying.  

Also, we have ImageLib tracking bug 158715.  Should this bug be added to that
tracking bug's list?

Comment 33

[Diego Biurrun]

*** Bug 127825 has been marked as a duplicate of this bug. ***

Comment 34

[Diego Biurrun]

OK, some info from the dupe:

Also happens on this URL and it also memleaks the X server.

http://www.crh.noaa.gov/grr/forum/derecho_anim.html

Comment 35

[Kevin Brannen]

I've had this for sometime too.  For me, Mozilla forces a leak in the X server.
 It's not uncommon for the X server to be taking up over 300M of swap after a
few hours of doing nothing but browsing.  I suspect it's not releasing pixmaps,
GC's, fonts, or something in the X server.  I'm currently running 1.0 on Linux
(RH7.2 & Suse 8.0 under KDE), but I'm about to upgrade to 1.1 in a few minutes.
 If that fixes it, I'll report back.

BTW, a really inconvenient work-around is to boot your machine into run-level 3
(non-X), then when all of your swap and RAM are about to be used, exit your X
session.  That will release most of it.  As root, turn off all swap, then turn
swap back on; that will force the rest to be discarded.  Now you can run startx
again, being at less than 40M of memory total (like you were when you first booted).

Comment 36

[Asa Dotzler [:asa]]

This bug was reopened after TM was set. Resetting TM. From descriptions of the
problem it doesn't sound very "Rapid" any more. If that's the case can someone
correct the misleading summary. Thanks.

Comment 37

[Cees T.]

Sounds related to bug 81446.

Comment 38

[Olivier Cahagne]

*** Bug 181304 has been marked as a duplicate of this bug. ***

Comment 39

[R.K.Aa.]

*** Bug 198043 has been marked as a duplicate of this bug. ***

Comment 40

[Michael Kauffmann]

Are You sure about the duplicate?

In my case, it does not grow with time, it just grows while loading. 
Then it can loop forever without more growing. 
And if I close the site in any way, it shrinks to 25MB. 
And it is the mozilla-bin, not the X.

It seems not like a memory hole, but just a simple implementation. The browser
saves all decoded frames in memory, without any compression.

I think GIF is anyway thwe wrong format for this animation.
A vector format (Flash) would be much better.
But users of other browsers report that they handle the giant GIF gracefully.

Comment 41

[Henry Hojnacki]

Attachment: Additional Win2K Platform Information

This bug also occurs on Win2K

Comment 42

[tech]

The problem is still in the very last firefox release (but not in mozilla
suite) try http://facs.scripps.edu/surf/images/euranim.gif

Comment 43

[Ryan VanderMeulen [:RyanVM]]

All of the test cases posted in this bug WFM with the current trunk. I see no change in memory usage once the image has fully loaded and is allowed to loop indefinitely.

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a2pre) Gecko/20061220 Firefox/3.0a2pre