470481 - Java(?) mem leaks

Status: RESOLVED DUPLICATE of bug 478823

(Firefox :: General, defect)

Description

[Eric Youngdale]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b2) Gecko/20081201 Firefox/3.1b2
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b2) Gecko/20081201 Firefox/3.1b2


Firefox will gradually burn up more and more memory until it crashes because of address space exhaustion.  On my machine, that happens in around 12-18 hours.


Reproducible: Always

Steps to Reproduce:
1. Create 15 to 20 tabs.
2. Direct each tab to http://msdn.microsoft.com/en-us/library/system.idisposable.aspx
3. Wait.  No need to interact with this in any way.
Actual Results:  
If you watch the task manager you will see that small amounts of CPU are being consumed, and the memory footprint is gradually increasing.

I can reproduce this with all extensions turned off, and from a brand new profile.



I played with AdBlock plus, and found that by blocking this URL:

http://i2.msdn.microsoft.com/Platform/cjs/Silverlight-bn2036.0.js

that I can nearly completely resolve the issue.  I am not much of a Java programmer, so I can't easily say more than that.

I should add that I habitually open up lots of tabs and leave many of them pointing to various MSDN pages, like the one I gave the URL for.  I never seemed to have this problem with Firefox 2.  There seems to be something different in Firefox3 and 3.1 beta that causes these massive memory leaks.

Comment 1

[Mardeg]

Is it possible that javascript file loads Silverlight plugin objects into the page? You could find out by clicking on the Silverlight plugin in Tools -> Add-ons, "Plugins" section, and then click the "disable" button to see if you get the same resolution of the issue.

Comment 2

[Eric Youngdale]

There isn't a silverlight plugin in the plugins section.

Comment 3

[Tyler Downer [He/Him]]

Thank you for this report. However, in order to make it more useful, please follow the steps found at http://quality.mozilla.org/bug-writing-guidelines and report back with your results. That page has a section on Mem leaks, read and follow the steps on the links in that section.

Comment 4

[Eric Youngdale]

I followed the instructions and ran the script.  Unfortunately the script reported no leaks.   Based upon what I noted above, I am guessing that it is a Java/JVM related bug, and the NSPR module doesn't seem to be able to log those types of problems.

I should add that at one point I caught the browser before it crashed, but when it had over 1G of memory allocated.  I closed out all of the tabs, but the memory footprint was not reduced.  

Is there something I can run from the Java console to dump a summary of the different object types that are allocated?

Comment 5

[Tyler Downer [He/Him]]

What version of Java are you using? You should be using 6 Update 11.

Comment 6

[Eric Youngdale]

Yes, it is 6u11.

I pulled open the Java console, and ran the 'm' command a couple of times.  It doesn't show growth there either...

I have Purify on this machine, but I don't know how well it works with Firefox.  I might just give it a try to see what happens..

Comment 7

[Eric Youngdale]

Hmm, Purify can't instrument mozcrt19.dll.  Is there a debug version of these things available somewhere?

Comment 8

[Eric Youngdale]

I have done some digging on this.  I found the debug symbols for mozcrt19.dll, and within this there is a data structure that is used for the heap that is used internally to that dll.  You folks are using a custom allocator for Windows, and this does a nice job of capturing statistics about how much stuff has been allocated.  In particular, arenas[0].stats gives allocation statistics.

After it has been running for a while, I interrupted firefox in the debugger, and found these values:

arenas[0].stats:
	mapped	7500000
	npurge	2
	nmadvise 0xab
	purged   0x446
	ndecommit 0x12c
	ncommit 0x2d1b
	decommitted 0x84c5
	allocated_small 0x17ad29a
	nmalloc_small	0x2b6cd81
	ndalloc_small	0x2afd96f
	allocated_large 0x57e5000
	nmalloc_large	0x5d8e
	ndalloc_large	0x3ad8
At this point firefox was consuming 128Mb of memory.

I then let it run some more (didn't interact with it in any way), and requested the same info:

	mapped	8400000
	npurge	3
	nmadvise 0x137
	purged   0x64b
	ndecommit 0x1ca
	ncommit 0x33af
	decommitted 0x98b8
	allocated_small 0x1577942
	nmalloc_small	0x36239e7
	ndalloc_small	0x35bd730
	allocated_large 0x668f000
	nmalloc_large	0x6c10
	ndalloc_large	0x4461

Now the VM size was reported to be 144M

I ran it some more, and let it grow, and got this data:

	mapped	8f00000
	npurge	3
	nmadvise 0x137
	purged   0x64b
	ndecommit 0x1d7
	ncommit 0x36ec
	decommitted 0xa58b
	allocated_small 0x15811fa
	nmalloc_small	0x3ccc1cb
	ndalloc_small	0x3c6566c
	allocated_large 0x7050000
	nmalloc_large	0x7718
	ndalloc_large	0x4bd5

And at the time I collected this last bit of data, the VM size is now 155Mb.

The indications are that I can see that small allocations don't seem to be growing at all, but large allocations just keep growing and growing.  I am running out of time to fool with this for the day, but my next inclination is to set breakpoints on the functions where the large allocations and deallocations occur to see if I can identify who is allocating the memory and perhaps not freeing it.

Comment 9

[Eric Youngdale]

OK, I have some more information.  To begin with I came up with a wrapper dll for mozcrt19.dll that contains instrumentation of various kinds related to malloc, free, etc.  I set things up so that firefox would run for 10 minutes(to allow the thing to reach a steady state), mark every allocation that was recorded, then run another 60 minutes to allow the leak to grow, and finally to dump a list of all allocations that happened during the last 60 minutes that weren't freed.

Originally I had planned to simply exit firefox and collect the list of leaked memory, but it turns out that the leaked memory that I am looking for is actually freed before we reach DLL exit.  So the nature of the leak is such that internally firefox still knows about the memory in some way or another.

I grouped the allocations that have identical stacktraces, and I added code to give symbolic stacktraces for convenience.  Finally I dumped the arena stats for both the 10 and 70 minute marks:

mapped 3300000 5500000
npurge 1 1
nmadvise 32 32
purged 209 209
ndecommit 67 8d
ncommit 13a7 1fb2
decommitted 36d4 5cae
allocated_small 14fdeaa 1606bb6
nmalloc_small 5bc3a3 1985041
ndalloc_small 5585f3 191c0f7
allocated_large 19a0000 3ac5000
nmalloc_large 22a6 44ad
ndalloc_large 1699 2c45

There were a number of potential leaks, but there was one set of leaks that just jump right out at you:

size:8215 nalloc:264 total:2168760 from:
	1: 0012F32C 0012F334 0054113F plds4!PL_ArenaAllocate (e:\builds\moz2_slave\win32_build\build\nsprpub\lib\ds\plarena.c:229+0x7) 
	2: 0012F358 0012F334 100AF3A2 xul!CategoryNode::AddLeaf (e:\builds\moz2_slave\win32_build\build\xpcom\components\nscategorymanager.cpp:317+0xb) 
	3: 0012F384 0012F334 100AF181 xul!nsCategoryManager::AddCategoryEntry (e:\builds\moz2_slave\win32_build\build\xpcom\components\nscategorymanager.cpp:635+0x28) 
	4: 0073B1B8 0012F334 10016234 xul!nsUConverterRegSelf (e:\builds\moz2_slave\win32_build\build\intl\uconv\src\nsuconvmodule.cpp:410+0x94) 

size:8215 nalloc:997 total:8190355 from:
	1: 0012F358 0012F360 0054113F plds4!PL_ArenaAllocate (e:\builds\moz2_slave\win32_build\build\nsprpub\lib\ds\plarena.c:229+0x7) 
	2: 0012F384 0012F360 100AF3A2 xul!CategoryNode::AddLeaf (e:\builds\moz2_slave\win32_build\build\xpcom\components\nscategorymanager.cpp:317+0xb) 
	3: 0012F3B0 0012F360 100AF181 xul!nsCategoryManager::AddCategoryEntry (e:\builds\moz2_slave\win32_build\build\xpcom\components\nscategorymanager.cpp:635+0x28) 
	4: 0012F3DC 0012F360 1014E037 xul!RegisterStreamConverters (e:\builds\moz2_slave\win32_build\build\netwerk\build\nsnetmodule.cpp:367+0x1f) 
	5: 0012F488 0012F360 100529B6 xul!nsGenericModule::RegisterSelf (e:\builds\moz2_slave\win32_build\build\obj-firefox\xpcom\build\nsgenericfactory.cpp:420+0x17) 
	6: 0012F4A4 0012F360 100FCC63 xul!RegisterStaticModule (e:\builds\moz2_slave\win32_build\build\xpcom\components\nscomponentmanager.cpp:3260+0x0) 
	7: 0012F4BC 0012F360 100FCC37 xul!nsStaticModuleLoader::EnumerateModules (e:\builds\moz2_slave\win32_build\build\xpcom\components\nsstaticcomponentloader.cpp:141+0xe) 
	8: 10764750 0012F360 10142A82 xul!nsComponentManagerImpl::AutoRegister (e:\builds\moz2_slave\win32_build\build\xpcom\components\nscomponentmanager.cpp:3306+0x0) 

size:8215 nalloc:597 total:4904355 from:
	1: 0012F33C 0012F344 0054113F plds4!PL_ArenaAllocate (e:\builds\moz2_slave\win32_build\build\nsprpub\lib\ds\plarena.c:229+0x7) 
	2: 0012F368 0012F344 100AF3A2 xul!CategoryNode::AddLeaf (e:\builds\moz2_slave\win32_build\build\xpcom\components\nscategorymanager.cpp:317+0xb) 
	3: 0012F394 0012F344 100AF181 xul!nsCategoryManager::AddCategoryEntry (e:\builds\moz2_slave\win32_build\build\xpcom\components\nscategorymanager.cpp:635+0x28) 
	4: 0012F3B8 0012F344 10145393 xul!RegisterTypes (e:\builds\moz2_slave\win32_build\build\layout\build\nscontentdlf.cpp:511+0x0) 
	5: 0012F3DC 0012F344 10179F89 xul!nsContentDLF::RegisterDocumentFactories (e:\builds\moz2_slave\win32_build\build\layout\build\nscontentdlf.cpp:565+0x0) 
	6: 0012F488 0012F344 100529B6 xul!nsGenericModule::RegisterSelf (e:\builds\moz2_slave\win32_build\build\obj-firefox\xpcom\build\nsgenericfactory.cpp:420+0x17) 
	7: 0012F4A4 0012F344 100FCC63 xul!RegisterStaticModule (e:\builds\moz2_slave\win32_build\build\xpcom\components\nscomponentmanager.cpp:3260+0x0) 
	8: 0012F4BC 0012F344 100FCC37 xul!nsStaticModuleLoader::EnumerateModules (e:\builds\moz2_slave\win32_build\build\xpcom\components\nsstaticcomponentloader.cpp:141+0xe) 
	9: 10764750 0012F344 10142A82 xul!nsComponentManagerImpl::AutoRegister (e:\builds\moz2_slave\win32_build\build\xpcom\components\nscomponentmanager.cpp:3306+0x0) 

size:8215 nalloc:388 total:3187420 from:
	1: 0012F32C 0012F334 0054113F plds4!PL_ArenaAllocate (e:\builds\moz2_slave\win32_build\build\nsprpub\lib\ds\plarena.c:229+0x7) 
	2: 0012F358 0012F334 100AF3A2 xul!CategoryNode::AddLeaf (e:\builds\moz2_slave\win32_build\build\xpcom\components\nscategorymanager.cpp:317+0xb) 
	3: 0012F384 0012F334 100AF181 xul!nsCategoryManager::AddCategoryEntry (e:\builds\moz2_slave\win32_build\build\xpcom\components\nscategorymanager.cpp:635+0x28) 
	4: 0073B1B8 0012F334 10016234 xul!nsUConverterRegSelf (e:\builds\moz2_slave\win32_build\build\intl\uconv\src\nsuconvmodule.cpp:410+0x94)

Comment 10

[Jo Hermans]

dupe of bug 395378 ?