1486019 - 2gb - 4gb memory usage on linux. Likely Memory Leak -- over 100 MB/hr

Status: RESOLVED WORKSFORME

(Thunderbird :: Untriaged, defect)

Description

[michaeljgreenburg]

Attachment: tbird.tar.gz

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36

Steps to reproduce:

I'm using Ubuntu 18.04 on a Dell XPS 13. My Thunderbird (version 52.9.1) settings are pretty vanilla, I think--two email accounts, one for my personal work email and one for my work's collective email. I haven't really messed with global settings; I can attach a configuration/settings file if that will help, but I didn't see anything in .thunderbird that looked obvious (e.g. "settings.json" or something).

What I've done so far:
1. Found Thunderbird using 4.9GB memory one morning after leaving it on overnight (this has been an issue since I've used Thunderbird, that was just the first time I tried to quantify it).
2. Started in Safe Mode to see if it would fix it--it didn't
3. Went through steps 1-7 (just the ones relevant to Linux, of course) of https://wiki.mozilla.org/Thunderbird:Testing:Memory_Usage_Problems
4. Step 8 says that if memory is hundreds of MB more than expected, give some log info as a bug report--so:
5. I exported NSPR_LOG_MODULES=MSGDB:5,timestamp and NSPR_LOG_FILE=$HOME/tbirdlog.txt
6. $ thunderbird -safe-mode &
7. Grabbed the memory usage every minute or so while it was running (about 20 hours): $ watch -n 60 'echo $(date +%D-%T) $(ps -p `pidof thunderbird` -ho rss,vsz) >> tbirdmem.txt'. I'm on mountain standard time so the times don't line up with those in the log file--I didn't think about that, sorry. 12:00 my time = 18:00 UTC.
8. I plotted the RSS memory (attached) to get an idea of what's going on--it basically looks linear with some jumps.


Actual results:

Memory ramps up about linearly; maybe there is a bound, but it's pretty high (over 3-4 GB) if there is one.


Expected results:

I wouldn't expect Thunderbird to use more than a gigabyte.

Comment 1

[Wayne Mery (:wsmwk)]

at your convenience, please try version 60

Comment 2

[michaeljgreenburg]

I'll probably just wait until 60.1 and let Thunderbird update itself. If that doesn't fix the issue, I'll report back here. It's easy enough to just close it at lunch, then again at 5 so that the memory doesn't get too out of control.

Comment 3

[Wayne Mery (:wsmwk)]

no problem. let's set a bound

Comment 4

[Wayne Mery (:wsmwk)]

Michael, 

1. can you check how many fd are open when you it goes wonky?  If there are many fd, then this could be a match to bug 1240722

2. we are running a little behind, so here is a candidate version for you to test https://archive.mozilla.org/pub/thunderbird/candidates/60.1.0-candidates/build1/linux-x86_64/en-US/thunderbird-60.1.0.tar.bz2

Please post your results

Comment 5

[michaeljgreenburg]

1. I left my old (52.9.1) thunderbird open for about 3 hours (at which point RES was at 1.416G), and lsof showed 331 file descriptors open (of these, 28 were .msf files). That doesn't seem too high, and nothing like the 1000+ fd as described in bug 1240722, so I would assume it isn't the same issue. If there's more that you'd like me to try/more info that would help, let me know.

2. Even using the candidate version in safe mode, the same issue seems to persist (I'll attach a graph of memory usage over ~2 hr). I just extracted in a random directory, went into it, and did `./thunderbird --safe-mode`; is there something more I should be doing to test it properly?

I don't think a 1D-10T error on my end is out of the question. Is it possible that I just did something dumb to my settings? I'm happy to attach info on that if you let me know what would be helpful.

Would it be worth moving ~/.thunderbird, starting from scratch (I usually leave everything vanilla so that wouldn't be hard), and seeing if the issue persists? Or does safe mode pretty much preclude issues originating in ~/.thunderbird?

Comment 6

[michaeljgreenburg]

Attachment: Thunderbird 60.1 candidate mem usage (~2 hr)

Comment 7

[Wayne Mery (:wsmwk)]

We are exploring a similar linux issue in a much older bug 1240722 - no smoking gun yet, although his problem is made worse by the tidybird addon.

Do you have any RSS accounts?
How many pop accounts, how many imap accounts.
How many exchange accounts?
Roughly how many folders?

Comment 8

[Wayne Mery (:wsmwk)]

(In reply to Wayne Mery (:wsmwk) from comment #7)
> We are exploring a similar linux issue in a much older bug 1240722 - no
> smoking gun yet, although his problem is made worse by the tidybird addon.

Bleh -  clearly I cannot read and need to stop for today.  Yeah, with only 28 folder files open it doesn't sound like bug 1240722 - which his perhaps a good thing.

> Do you have any RSS accounts?
> How many pop accounts, how many imap accounts.
> How many exchange accounts?
> Roughly how many folders?

The above questions are still relevant.

> does safe mode pretty much preclude issues originating in ~/.thunderbird?

Safe mode only disables addons, some altered preferences and custom css.

Comment 9

[michaeljgreenburg]

> Do you have any RSS accounts?
No

> How many pop accounts, how many imap accounts.
0 pop, 2 imap

> How many exchange accounts?
None

> Roughly how many folders?
6 in my personal work account (Inbox, Drafts, Templates, Sent, Junk, Trash), 39 in the other one (including sub-directories).



I'm going to leave the candidate Thunderbird on overnight and track the memory usage--is there anything else that I should log?

Comment 10

[Wayne Mery (:wsmwk)]

> 1. Found Thunderbird using 4.9GB memory one morning after leaving it on overnight (this has been an issue since I've used Thunderbird, that was just the first time I tried to quantify it).

How long would you say this has been a problem?  One year, two years, three years?

> is there anything else that I should log?

no

Comment 11

[michaeljgreenburg]

It's been about 5 months now--ever since I started using it on this laptop.

Comment 12

[Wayne Mery (:wsmwk)]

(In reply to michaeljgreenburg from comment #11)
> It's been about 5 months now--ever since I started using it on this laptop.

Please dive deeper. Did you have prior installs of Thunderbird?  And if so, who does this install differ from prior ones?

Comment 13

[michaeljgreenburg]

I started using it with a clean install of Ubuntu 18.04. Thunderbird is the one that was included with the OS, and until I tried out the 60.1 release candidate, has been the only one that I've used.

Comment 14

[Wayne Mery (:wsmwk)]

Fantastic detail in your logs. Thanks!
Are you still seeing this issue?

In a ~12 hour period there are 3,308 hits against 
  closing database    /home/micgre93/.thunderbird/3j7iia42.default/ImapMail/mail.fsl.byu-1.edu/Templates
That seems a bit odd.

Have you set that folder to check the server for new mail?  Or added the hidden preference to check all messages for new mail?
And what add-ons do you have installed?

Comment 15

[michaeljgreenburg]

Sorry for the lack of updates--I uninstalled and reinstalled Thunderbird (nothing fancy, just using apt), moved .thunderbird, and started from scratch, and the issue seems to have reduced significantly. I can leave Thunderbird open for a few days without memory usage getting much over a GB (I think 1.4GB after 4 days (?) was the worst I saw)--so there still might be a leak, but it's not nearly as severe as before. The "leak", if that's what it is now, is small enough that I would be satisfied to mark this as resolved and chalk it up to a weird setting or something, especially since no one else has seen it. If it will help Thunderbird improve, though, I'm also happy to keep chipping away at it.

mail.fsl.byu-1.edu as a whole gets checked for new messages at startup and every 10 minutes; the Templates folder is empty if that makes any difference. That email gets quite a few incoming messages--maybe that accounts for some of the growth in memory? I can start in safe mode and stop the checks if you'd like.

I have Lightning, Provider for Google Calendar, Messaging Menu and Unity Launcher integration, and Markdown Here.

Comment 16

[Wayne Mery (:wsmwk)]

Thanks for the update.

Unfortunate that we could not identify the cause, but you are in better shape