196628 - major mailnews leakage regressions

Status: RESOLVED INCOMPLETE

(Thunderbird :: General, defect)

Description

[Dan Mosedale (:dmosedale, :dmose)]

Transcript of a short IRC discussion I had with brendan:

<brendan> i see RSS jump by dozens of meg within the first few minutes of using
POP, getting messages, switching to my bugccs folder, junking mail
<brendan> i'm at 98M RSS now
<dmose> i'm not sure what that really indicates, given that any pages that are
already classified will be marked dirty and tossed if they're needed
* dmose looks at the mailnews test tinderbox
<brendan> i hear intense disk rattling (ibm thinkpad, noisy) when i switch folders
<brendan> i believe it's a leak
<dmose> could be
<brendan> if i keep doing normal stuff, i easily get past 256M SIZE
<brendan> i should reach steady state if JS needs to GC, or whatever
<brendan> it just keeps growing
<brendan> vm starts thrashing

I went and looked at the mailnews leak graphs, and we've been steadily getting
leakier over the past 250 days.  I suspect those graphs just show mailnews
startup & shutdown leaks (mcafee, can you confirm that this is the case).  If
so, that means they're probably only a small part of the issues that brendan is
seeing.

So we probably need to go investigate all the various upticks in that graph and
see if we can track down what happened.

Additionally we need to figure out how leaky the 1.3 branch is, and decide how
much, if any, of this problem we need to fix before 1.3 ships.

A few runs with the leak tools / valgrind / purify may also be instructive.

Comment 1

[Dan Mosedale (:dmosedale, :dmose)]

Note that the URL field of this bug has a link to the relevant chunk of the
leakage graph for the lhasa tinderbox.

Comment 2

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Bug 196624 is a recent mail composition leak regression (post 1.3).

Comment 3

[Brendan Eich [:brendan]]

dmose, anyone: any correlation of the leak graph spike with a checkin yet?

/be

Comment 4

[(not reading, please use [e-mail] instead)]

about major leakage, see bug #195163.

about the disk rattling, there's an issue with us calling Flush() too often for
local folders.  

(This regressed, something along the lines of we used to call flush() after a
batch, but now we call on ever message.)

also note that junk mail controls have also been enabled by default.  and there
could be leaks in that code, as well.

Comment 5

[Ere Maijala (slow)]

I don't see nsImapProtocol's destructor being called upon shutdown.

Comment 6

[Dan Mosedale (:dmosedale, :dmose)]

Chances are good that this still applies to Thunderbird, and it's the premier
mail app these days.

Comment 7

[Wayne Mery (:wsmwk)]

Would it make sense to hit mailnews memory issues [1] while we're sorta in a holding pattern as far as new mailnews features?  We'd gain somewhat from FF 3.0's QA process. Plus, won't it be a long wait to the next major gecko milestone, 2.0?  From my reading of the newsgroups it's not too late to hit the 1.9/FF 3 beta train.

Dan's URL is NA.  Also, do any new bugs need to be filed from the prior comments?

[1] https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=anywordssubstr&short_desc=&long_desc_type=substring&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=&resolution=FIXED&resolution=---&emailassigned_to1=1&emailtype1=exact&email1=&emailassigned_to2=1&emailreporter2=1&emailqa_contact2=1&emailtype2=exact&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&field0-0-0=keywords&type0-0-0=substring&value0-0-0=mlk&field0-0-1=short_desc&type0-0-1=anywordssubstr&value0-0-1=leak+memory&field1-0-0=product&type1-0-0=substring&value1-0-0=thunderbird&field1-0-1=component&type1-0-1=anywordssubstr&value1-0-1=mailnews%3A+network%3A+editor+address

Comment 8

[Phil Ringnalda (:philor)]

Whether or not we still have any of the same leaks we had in 2003, the odds of someone identifying one to the level of being able to usefully do something about it, and then determining that it had been around since before 2003, but had been added in the last half of 2002, and then deciding to use this bug for it, rather than just filing a new one with their patch, are infinitesimal. Had someone related jumps in the graph to checkins while the graph still existed, maybe it would still be useful, but "mailnews leaks increased in the 250 days before March 2003" just isn't going anywhere.