Thunderbird uses 101% of CPU at rest when mail.db.idle_limit is set to default 300000

UNCONFIRMED
Unassigned

Status

defect
--
major
UNCONFIRMED
2 years ago
2 months ago

People

(Reporter: contact, Unassigned)

Tracking

({perf})

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(4 attachments)

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gecko/20100101 Firefox/54.0
Build ID: 20170616092053

Steps to reproduce:

OS : Linux Mint 18.1
Thunderbird : 52.2.1
Addons : none

Thunderbird regularly uses 101% of my CPU. I followed the instructions in http://rainbow.chard.org/2013/02/19/thunderbird-high-cpu/ to edit the mail.db.idle_limit pref, which seems to have helped.

However, according to the comment by Wayne on that thread, this issue should have been fixed in Thunderbird 38.


Actual results:

Thunderbird CPU usage has been reduced to under 10% when idle after I updated the mail.db.idle_limit pref.


Expected results:

I shouldn't have had to edit the pref, as the issue was said to have been fixed in Thunderbird 38.
Please see bug 1305314 comment 8 to help us debug, and post your feedback
Severity: normal → major
Component: Untriaged → Database
Flags: needinfo?(contact)
Keywords: perf
Product: Thunderbird → MailNews Core
Summary: Thunderbird uses 101% of CPU at rest → Thunderbird uses 101% of CPU at rest when mail.db.idle_limit is set to default 300000
Version: 52 Branch → 52
(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #1)
> Please see bug 1305314 comment 8 to help us debug, and post your feedback

hmm even resetting the mail.db.idle_limit pref back to the default value seems to not have brought back the issue. TB is running consistently at <1% CPU, with occasional bumps to ~5% (I'm not sure why). It does seem to go up to 25%+ when the application is brought to the foreground, though that seems to only be for a few seconds.

However, after following the linked instructions for enabling debug logs, the open DB count is definitely changing. I don't see a way to attach a file to the bug, and I don't want to just past the logs into a comment, so please let me know if they would be useful.

Thanks for the quick reply!
Flags: needinfo?(contact)
(In reply to contact from comment #2)
> I don't see a way to attach a file to the bug, and I don't want to just
> past the logs into a comment ...
Yes, logs clutter comments, so please use "Attach File" at the top of the page.
Attaching logs.
I've had to wipe and reinstall Mint, so this is a fresh install of Thunderbird. I do now have a few addons -

AddressBookTab
gContactSync
Lightning
Provider for Google Calendar

It seems to be holding steady at about 60% CPU usage all the time. So frustrating!
Here's the output of `$ top | grep thunderbird` for the same time that the logs are, starting with launch. It peaked around 109% while launching, which is also unreasonably high, though it didn't last long enough to get into the `grep` output.
After changing the mail.db.idle_limit to 3000000 (adding another zero to the default), it's back down under 1% CPU for most of the time. I thought this was supposed to have been fixed many versions ago?
The bug was fixed. The closing interval is now correctly at 5 minutes (300000 milliseconds), which is proved by your first log. There is first cleanup of the DBs at 21:17, the folder Inbox is kept open doe to being opened in a window. It is no longer open by the window at 21:21 and 21:22. It is only closed at 21:23. Similarly, folder Outbox is closed at 21:22 (5 minutes after start if the periodic checking). So the particular bug of closing DBs too fast was fixed in the past.

In your log we can see:
1. You didn't access too many folders in the time the log was running, there are only 21 DBs (folders) at most in the cache (that is below the limit of 30).
2. Thus the folders are only closed due after 5 minutes due to inactivity, not because there are too many of them open (point above).
3. The cleanup run fits within one second of wall time (the last digit of the timestamp doesn't change between messages of a single run), thus you do not have too many DBs open and the cleanup is fast and can't be causing the permanent CPu use.

So the CPU use can be caused by some other/new problem. That increasing the timeout helps with the problem is indicative, but does not mean the bugs found so far haven't been fixed as we claim.
So I have some questions:
The 'top' log is interesting, if I read it correctly and the 6th columns is RSS (resident set size) and shows that in a short 1.while the memory use of TB climbs from ~300MB to ~850MB. That is quite large.
2.So how many folders and accounts do you have in total? Are your accounts POP3 or IMAP?
3.When you let Inbox get closed by the DB purging (watch the Error console), and then you open the folder again (view it in the UI), how long does that take to show its contents? How many messages you have in the Inbox?
We do not question there may still be other bugs existing in this area as there are multiple reports. We want to get those fixed, but need more data to see what is going on.
E.g. bug 1305314 has similar symptoms, but the user has hundreds of folders. If he accesses many of them regularly, the default limits may be too low for him, which is understandable.
So if you have few folders and still have the problem, we need to get that info.
Also please see in tools->activity manager, if e.g. the Indexing isn't still running on some folder.
Also see if you may have bug 781304, some of the .msf files for folders being corrupt.
Also please see bug 1305207 comment 10 if you can set up such profiling of TB. That would be very helpful.
I've not booted my Linux box in a number of months. In the meanwhile, I've reorganized my email account to move a lot of messages out of the Inbox. It's also now the only email account I have hooked up to Thunderbird.

However, when I open Activity Manager, it's trying to Index nearly 200,000 messages. I didn't think I had that many in that account, but I'm guessing that's what's taking so much CPU. How do I get it to stop?
If you do not need the indexing, you can turn it off in Options->Advanced->Global search and indexing
See Also: → 1305314

Environment:

  1. Thunderbird 60.7.1 (64-bit)
  2. Add-ons: Lightning, Provider for Google Calendar, LookOut (disabled)
  3. uname -a: Linux veritech 3.16.0-7-amd64 #1 SMP Debian 3.16.59-1 (2018-10-03) x86_64 GNU/Linux

Symptom:
As per https://rainbow.chard.org/2013/02/19/thunderbird-high-cpu/ default mail.db.idle_limit of 300000 results in thunderbird becoming slow to respond. Restarting thunderbird temporarily clears the problem, but after a period of "normal" use (opening, replying-to, deleting, and moving messages from folder-to-folder) %CPU reported by top pegs to 100% for tens of seconds to several minutes then hovers somewhere in 60% - 80% range. After leaving the machine alone for several hours the fifteen minute load average settles around 0.80 where normally it would otherwise float around 0.10.

Work-around:

  1. As per https://rainbow.chard.org/2013/02/19/thunderbird-high-cpu/ changing the default mail.db.idle_limit of 300000 to 30000000 and restarting thunderbird eliminates the excess load and laggy behaviour.
  2. Changing the default mail.db.idle_limit from 30000000 back to 300000 and restarting appears to have had no effect, in other words the %CPU now remains low and the laggy behaviour remains absent.

Hypothesis:
In addition to the explicitly named configuration setting, another parameter is being altered in response to the mail.db.idle_limit when the default value is changed. After the new mail.db.idle_limit value is reverted back to the default value, this other unidentified configuration item closer to the actual offending piece of code (i.e. the real bug of interest) remains in the altered state and therefore the problem remains worked-around.

Correction:
I jumped to conclusions too soon, after reverting mail.db.idle_limit from 30000000 back to the default value of 300000 and restarting the problem has reoccurred. My hypothesis is incorrect, the bug remains attached to mail.db.idle_limit .

You need to log in before you can comment on or make changes to this bug.