403473 - Big memory leak when opening and re-indexing large local mail folders

Status: RESOLVED INCOMPLETE

(Thunderbird :: General, defect)

Description

[George]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9
Build Identifier: version 2.0.0.6 (20070728)

Operating system: Linux RH4 U5, Dual processor
Installed plugins: Lightning 0.7 and Talkback 2.0.0.6

I see more memory leak reports, but cannot find the one I see. 

When re-indexing large mail folders (500 - 600 items ) (Edit => Folder properties => Rebuild index), the memory consumption rises rapidly up to 6G (this PC has 8G ram). This memory is not released, until Thunderbird is closed. Re-indexing the next large folder, increases the memory usage even more.

In the beginning that problem also happened when opening a large folder, but the re-indexing has solved that problem. However, every time I hit re index the memory usage goes through the roof.

Also leaving Thunderbird open over night, increases the memory usage slowly. So I need to restart at least every other day. However, that might be related to the new Lightning 0.7, because I haven't seen that problem with version 0.5 of lightning.



Reproducible: Always

Steps to Reproduce:
1. Click on a large local mail folder or sub folder (500 items should be enough)
2. Click Edit => Folder properties... => Rebuild index.
3. Watch your memory usage ramping up rapdily.. It won't go down until you close Thunderbird.
4. Try rebuilding the index a second or third time, and you will see it only go up again.

Weird enough this does not happen to small mail folders, no matter how often you  hit the "Rebuild index". 
Actual Results:  
Memory usage goes through the roof.. It won't go down until you close Thunderbird.

Expected Results:  
After rebuilding the index, the used memory should be released. 

Also the amount of memory used (4G to 6G) seems very high to me.

Same problem also exists in the 2.0.0.3 Version of Thunderbird. We haven't tried other versions.

Note that this is a Dual processor machine.

Comment 1

[George]

I've increased the priority. Most of our company members are using Thunderbird and several are running in the same problem.

Comment 2

[Wayne Mery (:wsmwk)]

500 items in a fold is not much at all.  
Compact first. Then reindex.
Results?
different results for pop, imap, local?

Comment 3

[George]

Sorry for my late response, haven't been checking my E-mail lately. 

I've copied the folder with almost 500 items to IMAP, and that makes a big difference. Rebuilding the index within the IMAP folder does not effect the memory use at all. But the message appearing is also different. 
When I do a rebuild index in "Local folders" it says "Rebuilding summary file". In the IMAP folder is says "Downloading headers", the "Rebuilding summary" message does not appear, or at least not visibly.

I don't use POP, only IMAP. I drag mail from my Inbox to the local folders. 

Compacting indeed makes a Huge difference. The problem is not gone, but much smaller after compacting. Now, after every re-index, I see memory usage increase about 1.5 Meg. That memory is not released until I close Thunderbird. It's better then it was, but the memory leak isn't solved yet.

Comment 4

[Wayne Mery (:wsmwk)]

George, what is normal size of a local folder?
http://kb.mozillazine.org/Performance_-_Thunderbird

Comment 5

[Wayne Mery (:wsmwk)]

George, you wrote "several [members of our company] are running in the same problem."  

Is it correct then that this problem doesn't happen with everyone?   Is there something that makes the group that sees the problem unique?

Comment 6

[Wayne Mery (:wsmwk)]

=> incomplete without feedback
if you still see the problem please try without lightning and add a comment