Runaway memory on folder compact (after installing RAM)

VERIFIED FIXED

Status

MailNews Core
Database
VERIFIED FIXED
14 years ago
10 years ago

People

(Reporter: Mike Holderness, Assigned: Bienvenu)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

5.22 KB, patch
(not reading, please use seth@sspitzer.org instead)
: review+
Scott MacGregor
: superreview+
(not reading, please use seth@sspitzer.org instead)
: approval1.6+
Details | Diff | Splinter Review
(Reporter)

Description

14 years ago
User-Agent:       Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.4) Gecko/20030624
Build Identifier: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.4) Gecko/20030624

Yesterday I upped my system RAM from 64M to 192M (largely because I got fed up
with Mozilla taking forever to swap after visiting a few pages - solution: avoid
broken MS VM). I reduced the swap file from 100M to 10M.

Since then, three times Mozilla has hung with RAM allocation maxed out
(according to Win NT Task Manager).

The last time was clearly during folder compaction on startup.

My largest mail folder has 1157 msgs in 11M. 

This may be related to Bug 73790 - reporeted fixed in 2001 - or to Tunderbird
bug 216535 - but have filed separately because it seems much more specific.


Reproducible: Sometimes

Steps to Reproduce:
1. Reboot machine. Start Mozilla;
2. Say "yes" to "compact folders?"
3. Cross fingers...

Actual Results:  
RAM usage maxes out, Mozilla hangs. (Once, Task Manager hung too!)

Expected Results:  
Compacted folders nicely and got on with life.
Confirming, I've seen this on 1.6b+

I've seen this to.  And afterwords, the memory isn't so bad as in the middle,
but we don't get it all back either.

This is a regression as it's never been this bad.  I've got 1GB of RAM in this
laptop.  I shouldn't be having 500MB's of it in use to compact my inbox.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Keywords: regression
OS: Windows NT → All
Hardware: PC → All

Comment 2

14 years ago
1.4 had some problems with memory, so I would suggest to upgrade to 1.4.1, if
you like these bugs (and others) fixed, 1.5, if you like new features and bugs,
or 1.6, when it comes.
1.4 is too old, to report bugs, so please retry with 1.4.1, or 1.6b, or better a
current nightly.

Robert, why did you change OS from WindowsNT to All?
Do you use All as OS on your notebook, and if so, which version of All?
I've seen this happen on OS X, and Windows, hence All.  There is no indication
that this is OS specific.
(Assignee)

Comment 4

14 years ago
I have noticed that compacting folders leaks some memory, but that shouldn't
cause Mozilla to hang. 

It could be that folder compaction is leaving db's open - I'll look.
I haven't had a hang... but then again, I have 1GB of memory, and not all of it
was occupied yet.  If I had 512MB, I don't know what would happen.  I guess my
computer would have froze up for a while at least.

Reporter:  Can you specify your specs (more curiousity).  RAM/CPU and how much
mail you have.
(Assignee)

Comment 6

14 years ago
yeah, compact all folders doesn't close the folder db's afterwards. I'm not sure
this is a regression...
(Reporter)

Comment 7

14 years ago
Need to note a caveat that an hour later my NT4 machine did something similar
while not running Mozilla. So I'm not in a good position to produce a clean test
case. 

Specs: 
Tosh Satellite 4300, Celeron 600MHz
192M RAM + 10M swap 
  (since yesterday, problem not seen before w/ 64M + 100M swap)
NT4 SP 6a + 2 security fixes since
Total in mail folders 52M (54M including .msf files)
Total in news folders 0.

Other processes running at the time: Netgear MA401 Wireless PC Card WiFi
monitor; MightyFax "print" driver; ZoneAlarm; CardWizard; MS Task Manager.

Plus Tosh standards: Power Saver; S3 Display; Battery Meter; Yamaha sound
manager; Country Selection Utility; MouseWare.
Removed regression keyword for now.

Blocking 1.6?
Since it doesn't look like a pre-christmas release is likely.  Can we hold 1.6
for this as well in that case?

It's not pretty to experience.  It feels enough like a crash for uses to
consider it a crash.

David, could this memory issue have been why I experienced some oddness viewing
my folders the other day?
Flags: blocking1.6?
Keywords: regression
(Assignee)

Comment 9

14 years ago
I can't account for 500MB of memory getting used, unless you have folders with
hundreds of thousands of messages.
Largest is 26,933 emails as of this post... a few attachments in there, but most
just email.
(Assignee)

Comment 11

14 years ago
Created attachment 137457 [details] [diff] [review]
close local folder db's after compact

this fix closes local folder db's after compact (and breaks out the code to do
that into a helper routine). It also removes a couple extra unneeded db commits
because ending the batch notification now does a commit.
(Assignee)

Comment 12

14 years ago
Comment on attachment 137457 [details] [diff] [review]
close local folder db's after compact

this also has a fix for creating intermediate local directories for local mail
folders...I can remove any or all of the other fixes if either of you guys
think I should.
Attachment #137457 - Flags: superreview?(mscott)
Attachment #137457 - Flags: review?(sspitzer)

Updated

14 years ago
Attachment #137457 - Flags: superreview?(mscott) → superreview+
Comment on attachment 137457 [details] [diff] [review]
close local folder db's after compact

r/a=sspitzer for 1.6 final
Attachment #137457 - Flags: review?(sspitzer)
Attachment #137457 - Flags: review+
Attachment #137457 - Flags: approval1.6+
(Assignee)

Comment 14

14 years ago
fix checked in. This should fix some bloat. 

Robert, can you try tomorrow's build? I'm still sceptical that the db's are
accounting for all the bloat, but it will be interesting to see what it looks
like tomorrow.
Status: NEW → RESOLVED
Last Resolved: 14 years ago
Resolution: --- → FIXED

Updated

14 years ago
Flags: blocking1.6?
Can I assume the nightly available now has this patch in it?
(Assignee)

Comment 16

14 years ago
yes, it should.
VERIFIED

per IM conversation with David.
Status: RESOLVED → VERIFIED
Product: MailNews → Core
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.