1305207 - Thunderbird uses much cpu when idle, fixed by increasing mail.db.idle_limit

Status: NEW

(MailNews Core :: Database, defect)

Description

[j.ammon]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:49.0) Gecko/20100101 Firefox/49.0
Build ID: 20160916101415

Steps to reproduce:

Use Thunderbird, release channel


Actual results:

My battery life got somewhat shorter, the macbook warmer, and the Activity Monitor showed Thunderbird using ~30% CPU regularly. Restarting Thunderbird didn't help. 

I increased mail.db.idle_limit as suggested in http://rainbow.chard.org/2013/02/19/thunderbird-high-cpu/, now Thunderbirds cpu usage is around 3%.


Expected results:

In a comment (http://rainbow.chard.org/2013/02/19/thunderbird-high-cpu/#li-comment-40585) Wayne claimed, that these issues should be resolved as of version 38. Obviously for me they are not.

Thank you for this great product and for answering to user problems!

Comment 1

[Wayne Mery (:wsmwk)]

(In reply to j.ammon from comment #0)
> ...
> In a comment
> (http://rainbow.chard.org/2013/02/19/thunderbird-high-cpu/#li-comment-40585)
> Wayne claimed, that these issues should be resolved as of version 38.
> Obviously for me they are not.

heh. that would be me. But that is not quite what I said. Quote "As of version 38, most issues involving mail.db.idle_limit should be resolved."  Note the word "most".

> My battery life got somewhat shorter, the macbook warmer, and the Activity
> Monitor showed Thunderbird using ~30% CPU regularly. Restarting Thunderbird
> didn't help. 
> 
> I increased mail.db.idle_limit as suggested in
> http://rainbow.chard.org/2013/02/19/thunderbird-high-cpu/, now Thunderbirds
> cpu usage is around 3%.

Unfortunately you are not the first person to observe this. 

I am too tired at the moment to give much coherent advice, but get you some info and get the ball rolling to figure this out.
1. does your problem reproduce if you reset the preference and have thunderbird started in safe mode?
2. Can you find (with taskmgr or other tools) the running thunderbird.exe has characteristics other than high CPU - for example higher memory usage or higher IO under similar workload - compared to when you have changed the preference?
3. Other bugs which mention the preference https://mzl.la/2cKXrx7  (but not all of those bugs have the *this* problem - they just mention the preference)

> Thank you for this great product and for answering to user problems!
thank you for filing the bug report. 
Yes, we try. We'd welcome you to join us. https://wiki.mozilla.org/Thunderbird#Contributing


> Thank you for this great product and for answering to user problems!

Comment 2

[:aceman]

Just to be sure, what was the value of the pref and how much did you increase it?
How many folders do you have in total?

Comment 3

[:aceman]

This bug is not actionable until we get answer to comment 2.

Comment 4

[:aceman]

You can also help by restoring the value to default and the enabling debug output of the database cache manager. You set these 2 prefs to a value of "Debug" via the Advanced config method that you already know:

+// Should we output dbcache log via dump? Set to "Debug" to show.
+pref("mailnews.database.dbcache.logging.dump", "None");
+// Should we output dbcache log to the "error console"? Set to "Debug" to show.
+pref("mailnews.database.dbcache.logging.console", "None");

Then restart TB and open Tools->Error console. Every minute you will see a database purge run. Please observe if there are any oddities. Like databases closed sooner than 5 minutes. Less databases left open than the default 30? Or do you have always so many databases/folders cached, that some are always closed and need to be reopened soon after. Please provide some stats from the output.

Comment 5

[WADA:World Anti-bad-Duping Agency]

Following is a problem determination procedure provided in Bug 1305314 Comment #8 by aceman.
(1) Via. Option->Advanced->General,Config Editor, change next settings.
 mailnews.database.dbcache.logging.console : None => Debug
 mail.db.idle_limit : reset to default (default=300000 ==300sec==5minutes)
 mail.db.max_open : What value do you use?
(2) Restart Thunderbird, Open Error Console via Tools menu.
(3) Every 1 minute, report of "trying to close opened DBs" is shown at Error Console.
1st line of continuous logs is always next log.
> 2016-11-20 18:50:25 mailnews.database.dbcache INFO periodic check of cached dbs, count=NNN
Because each log line is separated log, "copy at Error Console, paste at text editor" is pretty hard.
There is no need to do copy&paste of many many log lines for reporting.
Because many log line are written, clear Error Console sometimes while you are testing, please.

(Q1) Is you problem consistently reproduced by setting mail.db.idle_limit=300000?

Comment 6

[j.ammon]

Sorry for the long delay, personal and occupational duties made it scroll down too fast.

I changed mail.db.idle_limit from the default (300.000) to 30.000.000.
I'm not sure how to determine, how many folders I have.

I'll try the problem determination procedure mentioned by WADA and get back to you, when I have results.

Comment 7

[j.ammon]

I restored mail.db.idle_limit to 300.000, mail.db.max_open is 30, I restarted thunderbird.
Now something like this appears every minute:

2016-11-20 17:43:45	mailnews.database.dbcache	INFO	periodic check of cached dbs, count=9
2016-11-20 17:43:45	mailnews.database.dbcache	DEBUG	folder open in window, name: Posteingang
2016-11-20 17:43:45	mailnews.database.dbcache	DEBUG	skipping cachedDB not open in folder: Gesendet
2016-11-20 17:43:45	mailnews.database.dbcache	DEBUG	skipping cachedDB not open in folder: Papierkorb
2016-11-20 17:43:45	mailnews.database.dbcache	INFO	open db count 7
...

After a while it is more like:
2016-11-20 17:50:45	mailnews.database.dbcache	INFO	periodic check of cached dbs, count=98
2016-11-20 17:50:45	mailnews.database.dbcache	DEBUG	skipping cachedDB not open in folder: Archiv
2016-11-20 17:50:45	mailnews.database.dbcache	DEBUG	skipping cachedDB not open in folder: 2001
... many more archived folders

The next minute it is again down to "periodic check of cached dbs, count=6".

Does this speak to you?

Thunderbird uses up to >70% CPU when I use the ui (just scroll up and down), but goes down to 0.1% when it is in the background or unused in the foreground.
The problem seems to have vanished.

[...10 hours later...]

Now Thunderbird is constantly at 5-8% CPU even when it is in the background.
In the log it is "periodic check of cached dbs, count=6" every minute.

Any ideas what I could do to come closer to the problem?

Comment 8

[:aceman]

Thanks for the test.

(In reply to j.ammon from comment #7)
> I restored mail.db.idle_limit to 300.000, mail.db.max_open is 30, I
> restarted thunderbird.
> Now something like this appears every minute:
> 
> 2016-11-20 17:43:45	mailnews.database.dbcache	INFO	periodic check of cached
> dbs, count=9
> 2016-11-20 17:43:45	mailnews.database.dbcache	DEBUG	folder open in window,
> name: Posteingang
> 2016-11-20 17:43:45	mailnews.database.dbcache	DEBUG	skipping cachedDB not
> open in folder: Gesendet
> 2016-11-20 17:43:45	mailnews.database.dbcache	DEBUG	skipping cachedDB not
> open in folder: Papierkorb
> 2016-11-20 17:43:45	mailnews.database.dbcache	INFO	open db count 7
> ...
> 
> After a while it is more like:
> 2016-11-20 17:50:45	mailnews.database.dbcache	INFO	periodic check of cached
> dbs, count=98

Now this case is interesting. 98 databases. It would explain the CPU use if you had >30 databases open all the time and every minute TB tried to get them down to 30. But that is not the case. Below you say it is around 6-7 every minute.

> 2016-11-20 17:50:45	mailnews.database.dbcache	DEBUG	skipping cachedDB not
> open in folder: Archiv
> 2016-11-20 17:50:45	mailnews.database.dbcache	DEBUG	skipping cachedDB not
> open in folder: 2001
> ... many more archived folders
> 
> The next minute it is again down to "periodic check of cached dbs, count=6".
> 
> Does this speak to you?

No, if you have only 6 active folders I cannot understand the background work yet.

> Thunderbird uses up to >70% CPU when I use the ui (just scroll up and down),
> but goes down to 0.1% when it is in the background or unused in the
> foreground.
> The problem seems to have vanished.

Please do the test again, but do not have the Error console open. Only open it after X hours if there is significant CPU activity while idle.

Comment 9

[WADA:World Anti-bad-Duping Agency]

(In reply to j.ammon from comment #7)

What is done by checkCachedDBs every 1 minute is:
> (1) Close by mail.db.idle_limit phase.
> (1-1) Report "periodic check of cached dbs, count=NNN"
> where NNN is # of msgDBs which was opened by someone(xxx.msf was read) at least once and was cached.
> If msgDB is for folder which is currently selected at folder pane, close at phase (1)/(2) is skipped.
> This is reported by log of "folder open in window, name: Posteingang" in phase (1)/(2).
> (1-2) If cached DB is not opened(msgFolder.msgDabase=null is already set), nothing is done on it,
> This is reported by "skipping cachedDB not open in folder: Gesendet".
> (1-3) if msgDB is opened before idle_limit milliseconds, store msgFolder.msgDabase=null(reduce 1 from reference count to msgDB)
> (2) Close by mail.db.max_open phase.
> (2-1) Report "open db count MMM" where MMM is still opened DB count after phase (1).
> (2-2) If # of still opened DB exceeds mail.db.max_open(default=30), msgFolder.msgDabase=null is set.
> i.e, # of opened DB is reduced to mail.db.max_open(default=30) at phase (2).
> 
> After it, following occurs.
> (A) If no one accessed DB after setting msgFolder.msgDabase=null,
>    (A-1) msgDB is completely closed sooner or later and xxx.msf is written back/closed by cachedDB management process.
>    (A-2) And, after a while, memory resources for the msgDB object is purged by garbage collector.
>     When (A) occurs, NNN of "periodic check of cached dbs, count=NNN" is declines.
> (B) When someone accessed msgDB for a folder, if it's still cached, cached DB is re-used, and reference count is incremented.
> (C) When someone accessed msgDB for a folder, if it's already purged by (A), entire xxx.msf file is read again, and msgDB object is re-created and cached.

In your case, # of cached DB is:
>    periodic check of cached dbs, count=9
> => periodic check of cached dbs, count=98
> => periodic check of cached dbs, count=6

> Does this speak to you?

Yes for me. Thanks for testing and reporting.

Difference between mail.db.idle_limit=300,000 and mail.db.idle_limit=30,00,000 is merely next.
> mail.db.idle_limit=300,000 == 300 sec == 5 minutes
>   msgFolder.msgDatabase=null is set after 5 minutes to almost all cached msgDB at step (1-2).
>   # of opened DB is reducerd to mail.db.max_open at step (2-2).
> mail.db.idle_limit=30,000,000 == 30,000 sec == 500 minutes == 8 hours + 20 minutes
>   msgFolder.msgDatabase=null is not set at step (1-2) while daylight time.
>   # of opened DB is reducerd to mail.db.max_open at step (2-2).
And, what done by checkCachedDBs is merely following, and workload of it won't depend on setting.
> Phase-1. Get all cachedDB via. enumerator, and sets msgFolder.msgDatabase=null for some cachedDB.
> Phase-2. Get all cachedDB via. enumerator, and sets msgFolder.msgDatabase=null for some cachedDB.

"periodic check of cached dbs, count=98" indicates that someone accessed(opened) many folders.
Even if msgFolder.msgDatabase=null is set by checkCachedDBs,
if cachedDB is re-used by (B) before (A) happens, (C) won't occur.
It may be following.
(a) "msgDB open->msgDB close/garbage collection->msgDB open" upon each folder open.
> +--> Someone periodically uses folders and opens many msgDBs
> |    -> checkCachedDBs sets msgFolder.msgDatabase=null after 5 minutes
> |    -> before someone accesses the msgDB again, (A) occurs and msgDB is purged --+
> |       and garbage collecter runs.                                               |
> |                                                                                 |
> +---------------------------------------------------------------------------------+
(b) The "someone" may be Gloda(Global Search/Indexer).
    The "someone" may be excess access to folder by addon(extension).
(c) If pretty big folder(number of messages is large, and xxx.msf file size is large),
    it takes long to read entire xxx.msf file and to re-construct msgDB object from scratch.

> Thunderbird uses up to >70% CPU when I use the ui (just scroll up and down),
> but goes down to 0.1% when it is in the background or unused in the foreground.
> The problem seems to have vanished.
I can't think ">70% CPU" can occur merely by (c) upon each periodic folder open by someone.
Because it's while you are using UI of Thunderbird, I feel that it's phenomenon by addon when (c) happens upon each folder open.

> [...10 hours later...]
> Now Thunderbird is constantly at 5-8% CPU even when it is in the background.
> In the log it is "periodic check of cached dbs, count=6" every minute.
It may be explained by "(a) msgDB open->msgDB close/garbage collection->msgDB open upon each folder open".
If "100 folder open -> 100 msgDB close/garbage collection -> 100 msgDB re-open by folder access" occurs periodically, I think "5-8% CPU" is used when big folders(big xxx.msf, large # of mails) are involved.

(Q1) What is your mail.db.max_open setting?
(Q2) Can you observe ">70% CPU" by mail.db.idle_limit with safe mode of Thunderbird?
    (thunderbird.exe -safe-mode, or Restart without addon)
(Q3) What addon do you use?

By the way, you parhaps can see your problem with following(similar to phenomenon in idle_limit=300 millisec era.)
 mailnews.database.dbcache.logging.console : None or Debug
 mail.db.idle_limit : larger than 30,000,000, for example 1,000,000,000
  (kill msgFolder.msgDatabase=null at phase-1)
 mail.db.max_open : 1
  (force msgFolder.msgDatabase=null at phase-2 on any cachedDB every 1 minute,)
  (except folder which is selected at foler pane)

Comment 10

[Wayne Mery (:wsmwk)]

(In reply to :aceman from comment #8)
> 
> Please do the test again, but do not have the Error console open. Only open
> it after X hours if there is significant CPU activity while idle.

J.Ammon,

Can you do the above please?

And also do a performance profile when it hits high CPU using the procedure at  https://developer.mozilla.org/en-US/docs/Tools/Remote_Debugging/Thunderbird which uses Firefox as the recorder

- follow instructions at https://developer.mozilla.org/en-US/docs/Tools/Remote_Debugging/Thunderbird
- after connect, pick "main"
- click "Performance" in the developer tool menu
- click "start recording performance" in the center
- do some actions in thunderbird which cause the problem or behavior to be recorded
- click "stop recording performance"  in the center
- click "Save" one the left
- email the json file or attach to this bug report

Edit - for much newer version of THunderbird, use the instructions at  https://support.mozilla.org/en-US/kb/profiling-thunderbird-performance

Comment 11

[:aceman]

Yes, the profiling could be helpful. Also please retest with TB52.

(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #10)
> (In reply to :aceman from comment #8)
> > 
> > Please do the test again, but do not have the Error console open. Only open
> > it after X hours if there is significant CPU activity while idle.
> 

I ask for NOT opening the console because we have observed that the action of opening any other TB windows (other than the main 3-pane window) affects the database caching and finally releases some of the databases already marked for closure.

Comment 12

[Scott]

This bug is still in Thunderbird as of version 60.3.3 on a Macbook running OS 10.14.1

It can still be fixed by following the instructions that have previously been posted:

If you set mail.db.idle_limit as suggested in http://rainbow.chard.org/2013/02/19/thunderbird-high-cpu/, Thunderbird's cpu usage is ~3%.

Comment 13

[:aceman]

Really, so what was the value before you changed it and how much did you make it?

Comment 14

[Scott]

Just as http://rainbow.chard.org/2013/02/19/thunderbird-high-cpu/ said I would find, mail.db.idle_limit was 300000.  I set it to 300000000 as they suggested.  Now Thunderbird is 4% or lower and my CPU fan doesn't start all day.  Much better.

Comment 15

[UliM]

Hello,

I had the same issue. TB was very slow for weeks, nearly unusable for 10-20 mins. After start, it gabe "No Response" ("Keine Rückmeldung"). I was close to quiting with TB, but then I found the mail.db.idle-limit hint from the above link. I changed the value by adding two Zeros (new value 30000000). Now TB uses a lot of CPU for ca. 1 min, but then the CPU usage is ok.

I have made a PerfExplorer Screenshot that shows CPU usage (8 core machine, i.e. 12.5 % means TB uses 1 core completely). Memory goes up and down, this seems to be related to the source of the problem. Once TB is finished (with whatever it is doing), memory is smooth and no CPU is required any more.

My TB version: 60.4 (just updated)
5 or so IMAP mail accounts that use up to 8 GB file storage (yes, more than one decade of mail )

Feel free to ask detailed questions (I am a software developer myself and know how important all information is when debugging).

Comment 16

[UliM]

Attachment: thunderbird-after-start-2018-12-27.png

Here is the screenshot mentioned in comment 15

Comment 17

[MHW]

I too have been seeing fairly heavy cpu demands by Thunderbird. Heavier at first (~20 to 30%) then dropping to ~5 to 25% after a few minutes. Routinely consumes significantly more than anything else running on my system. I have very large mail folders, like ULiM, well over a decade of mail. (Pack-rat...) I have both POP and IMAP accounts (4 POP, 2 IMAP). TB version 60.4.0 Routinely leave Calendar open. Running on 4-core Intel i5-6600 with 16 GB RAM, 64-bit Win 10Pro ver 1809.

Given 300000ms = 5 minutes, 3000000ms = 50 minutes, 30000000ms = 500 minutes = 8.3 hours. 500 mins seems a bit long so first tried 50 mins.

I first tried adding 1 zero for 50 minutes, saw no change. Then re-launched TB at 1:13pm. Immediately saw that TB is using 0% CPU. Within 1 min CPU briefly went up to ~5% and fluctuated 0-5%. On Get Messages went up to ~15% then back to 0 within seconds. Then watched for 3 min and mostly at 0% with very short spikes to 1%. At 1:30pm cpu still averaging 0%. So indeed immediate big decrease in cpu load upon relaunch of TB.

I considered the change to Global Index Search, but wanted to do single change at a time. Found that mail.db.idle_limit change of adding 1 zero did the trick so did not change global index search.

So add a single 0 to mail.db.idle_limit made immediate significant reduction in cpu load. A short spike every 50 minutes is very acceptable in my book, whereas constant load of 10-30% is too much.

(BTW, I've used Thunderbird for years, going back to days of SeaMonkey. Big fan of all Mozilla FireFox and Thunderbird. I also use Chrome browser, but never IE and rarely Edge. I don't even bother with the others. In other words, Mozilla rocks!)

Comment 18

[MHW]

Following up on my prior comment, over the past hour TB has been consistently using 0% of cpu. I watched esp. around the 50 min mark and only slight bump, but only very briefly to ~6%.

Comment 19

[MHW]

Then getting the message resulting from the post 2 minutes ago got brief bump to ~16% then quickly back to 0%. It would seem then that there is an issue and the mail.db.idle_limit fix of adding a single zero to time does the trick. TTFN.

Comment 20

[Wolf Rudolf]

I can confirm. Running Thunderbird 60.2.1 (64-Bit) on Kubuntu 18.04, CPU-usage from Thunderbird was constantly 50% or higher. After adding two zeroes to mail.db.idle_limit, it is now under 1 % most of the time. I have 6 IMAP-Accounts and many local folders and subfolders with thousands of messages.

Comment 21

[cokefridgefan]

Same issue, solved by changing the limit from the old 300 000 to 30 000 000 and my CPU load dropped from around 7-20 % permanently to 0-1 % when Thunderbird is running in background/in my tray. :)

Version 60.6.1 (32 bit) on a Lenovo Thinkpad W550s running Win 7 Home Premium SP1 (64 bit).
However, i use Thunderbird on this computer for around 2-3 years, so maybe this is an issue of an old version and then the limit just didn't update with the bazillion version updates.

Comment 22

[:aceman]

The default for mail.db.idle_limit is still 300 000 so if you had it at that value, you are updated correctly.

In this bug we would like to find out why it is beneficial to increase it for some users.

cokefrigefan, Wolf Rudolf, MHW, UliM, Scott, could you guys enable the logging temporarily per comment 4 and tell us the count of folders being open/purged there?

Comment 23

[MHW]

(In reply to :aceman from comment #22)

The default for mail.db.idle_limit is still 300 000 so if you had it at that value, you are updated correctly.

In this bug we would like to find out why it is beneficial to increase it for some users.

cokefrigefan, Wolf Rudolf, MHW, UliM, Scott, could you guys enable the logging temporarily per comment 4 and tell us the count of folders being open/purged there?

Ok, I've followed steps here in Comment 22 and in Comment 4. (Set idle_limit back to 300 000, and o mailnews.database.dbcache.logging.console/dump from "none" to "show".) As I run through TB (Get msgs, view Inbox, Empty Trash) CPU load is very reasonable until I Compact Folders.

First off, count of folders being open/purged: Initially cached folder databases count = 6. then it fluctuated slightly up or down 1 or 2. I didn't see any purges.

When I Compact Folders CPU load jumps to 25-32% and Error Console displays endless scrolling errors such as this:

gloda.index_msg WARN Observed header that claims to be gloda indexed but that gloda has never heard of during compaction. In folder: mailbox://pooder@mail.../Inbox sketchy key: 93867782 subject: Registration Confirmed - Journey to the Wasatch - 2019

The email listed after Subject is close to the oldest email in that Pooder Inbox. (Curious why it's not the last/oldest, but that's the pattern.) I've gone in a couple times and deleted that email and repeated steps. Same result but different email referenced in Subject. I've also set idle_limit back to 30 000 000 and same result.

I've used CCleaner to clear all internet cache, internet history, cookies, session and site preferences for TB. this has had no effect on results. Every time after Compact Folders CPU goes nuts and endless gloda.index_msg warning scroll.

I wonder if something is corrupted? In mailbox or folder, or in other settings. I suppose I could reinstall TB to see what effect that has. I've stopped at that point to see what replies I get from you all.

BTW, I've taken screenshots of Error Console at various points to catalogue results. I'm not really sure what other data-points would be useful. I can send screenshots if that would be helpful. (Where to send?) What other detail would help?

Comment 24

[MHW]

(In reply to MHW from comment #23)

(In reply to :aceman from comment #22)

The default for mail.db.idle_limit is still 300 000 so if you had it at that value, you are updated correctly.

In this bug we would like to find out why it is beneficial to increase it for some users.

cokefrigefan, Wolf Rudolf, MHW, UliM, Scott, could you guys enable the logging temporarily per comment 4 and tell us the count of folders being open/purged there?

Ok, I've followed steps here in Comment 22 and in Comment 4. (Set idle_limit back to 300 000, and o mailnews.database.dbcache.logging.console/dump from "none" to "show".) As I run through TB (Get msgs, view Inbox, Empty Trash) CPU load is very reasonable until I Compact Folders.

First off, count of folders being open/purged: Initially cached folder databases count = 6. then it fluctuated slightly up or down 1 or 2. I didn't see any purges.

When I Compact Folders CPU load jumps to 25-32% and Error Console displays endless scrolling errors such as this:

gloda.index_msg WARN Observed header that claims to be gloda indexed but that gloda has never heard of during compaction. In folder: mailbox://pooder@mail.../Inbox sketchy key: 93867782 subject: Registration Confirmed - Journey to the Wasatch - 2019

The email listed after Subject is close to the oldest email in that Pooder Inbox. (Curious why it's not the last/oldest, but that's the pattern.) I've gone in a couple times and deleted that email and repeated steps. Same result but different email referenced in Subject. I've also set idle_limit back to 30 000 000 and same result.

I've used CCleaner to clear all internet cache, internet history, cookies, session and site preferences for TB. this has had no effect on results. Every time after Compact Folders CPU goes nuts and endless gloda.index_msg warning scroll.

I wonder if something is corrupted? In mailbox or folder, or in other settings. I suppose I could reinstall TB to see what effect that has. I've stopped at that point to see what replies I get from you all.

BTW, I've taken screenshots of Error Console at various points to catalogue results. I'm not really sure what other data-points would be useful. I can send screenshots if that would be helpful. (Where to send?) What other detail would help?

I wasn't happy with my last post, feels like it was a bit of a red herring, so I went in a deleted several older emails in that inbox. (Note that I had set idle_limit back to 30 000 000.) Then repeated steps of Empty Trash and Compact Folders for the Pooder account. Now none of those gloda msgs. DBs open ranging from 7 to 9. Counts ranging 8 to 11. Purges were 0. TB CPU usage is idling nicely below 1% with slight bumps to <5% at each minute. These results are with idle_limit at 30 000 000.

Reset idle_limit back to default 300 000. Restarted TB. Repeated steps of delete a couple emails, Empty Trash and Compact Folders. No gloda errors, only small and short CPU bumps on Compact Folders. DB counts = 9. Purges = 0.

So now working fine with either idle_limit. Not sure what issue was in my post from an hour ago. TB is working fine, no CPU load to speak of. I'll continue watching next couple of days and will post if I see anything remarkable.

Comment 25

[:aceman]

Thanks. For that gloda error, you may want to join bug 1362483.

Still, in your tests now you can only produce high CPU usage by compacting folders. Previously you reported CPU load is high constantly while TB is idle, with the default mail.db.idle_limit.

Comment 26

[MHW]

(In reply to :aceman from comment #25)

Thanks. For that gloda error, you may want to join bug 1362483.

Still, in your tests now you can only produce high CPU usage by compacting folders. Previously you reported CPU load is high constantly while TB is idle, with the default mail.db.idle_limit.

Confirming idle_limit is set at default. Yes, now CPU load does not stay high while TB is idle, it is near-zero most of the time, except brief bump during compact folders. I just deleted a total of 300+ emails, emptied trash then compacted folders. During that whole sequence CPU load bumped up briefly then back down to zero within a few seconds. Completely not the same as when I initially reported 4 months ago.

My experience with errors today must have been an anomaly, that I won't worry about unless it repeats, as I'm not able to get it to repeat after deleting earlier emails. Although, I'll take a look at bug 1362483. Thanks for that.

Comment 27

[UliM]

(reply to comment22)
Update of current status with TB:

• After start of program, it runs for about 1-2 minutes with high load (ca. 15% with 8-core system, it runs singlecore I assume). Responsiveness of TB in this time frame is bad.
• Afterwards, CPU load drops and responsiveness is ok.
  This is much better than 1/2 year ago, when it took more than 15 mins. to go back to a usage mode.

Settings:

• idle_limit has been set to default
• 5 mail accounts + one local mail account
• Size of mail folder 8 GB
• One exchange account (accessed via a local DavMail server)
• Several calendars (local, nextcloud, exchange via DavMail)

Here is my debug log (dont know how to attach a file):
While creating services from category 'profile-after-change', could not create service for entry 'calendar-backend-loader', contract ID 'service,@mozilla.org/calendar/backend-loader;1'
Mutations-Ereignisse sollten nicht mehr verwendet werden. Verwenden Sie MutationObserver stattdessen. calendar-widgets.xml:512:20
Lightning:Calendar moz-storage-calendar:// has a dangling E-Mail identity configured. calProviderUtils.jsm:323
TypeError: this.parentNode is null[Weitere Informationen] tree.xml:1285:9
TypeError: this._parentMenupopup is null[Weitere Informationen] mailWidgets.xml:2728:9
Lightning:"Calendar https://www3.primuss.de/stpl/index.php?FH=fhin&Language=de&mode=ical&Session=e6333b59fc97160c3ba7195e5c7e0a12&User=xxx&sem=34&type=4 has a dangling E-Mail identity configured." calProviderUtils.jsm:323
Lightning:"Calendar https://www3.primuss.de/stpl/index.php?FH=fhin&Language=de&mode=ical&Session=e7d0ee9f18ed63ec80db421faeb6ec3e&User=xxx&sem=33&type=4 has a dangling E-Mail identity configured." calProviderUtils.jsm:323
PROPFIND
http://localhost:1080/principals/users/xxx.de/
[HTTP/1.1 401 Unauthorized 15ms]
Lightning:Couldn't find (UTC+01:00) Amsterdam, Berlin, Bern, Rom, Stockholm, Wien calTimezoneService.js:194
Lightning:Couldn't find (UTC+01:00) Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna calTimezoneService.js:194
2019-05-17 08:51:13 mailnews.database.dbcache INFO Periodic check of cached folder databases (DBs), count=3

2019-05-17 08:51:13 mailnews.database.dbcache DEBUG Skipping, DB open in window for folder: Posteingang

2019-05-17 08:51:13 mailnews.database.dbcache INFO DBs open in a window: 1, DBs open: 2, DBs already closing: 0

Lightning:Couldn't find Customized Time Zone calTimezoneService.js:194
Lightning:Couldn't find W. Europe calTimezoneService.js:194
Lightning:Couldn't find Customized Time Zone calTimezoneService.js:194
2019-05-17 08:52:15 mailnews.database.dbcache INFO Periodic check of cached folder databases (DBs), count=3

2019-05-17 08:52:15 mailnews.database.dbcache DEBUG Skipping, DB not open for folder: Posteingang

2019-05-17 08:52:15 mailnews.database.dbcache INFO DBs open in a window: 0, DBs open: 2, DBs already closing: 1

Lightning:Couldn't find Central Europe calTimezoneService.js:194
Lightning:Couldn't find Customized Time Zone calTimezoneService.js:194
Lightning:Couldn't find Customized Time Zone calTimezoneService.js:194
Lightning:Couldn't find W. Europe calTimezoneService.js:194
2019-05-17 08:53:15 mailnews.database.dbcache INFO Periodic check of cached folder databases (DBs), count=3

2019-05-17 08:53:15 mailnews.database.dbcache DEBUG Skipping, DB not open for folder: Posteingang

2019-05-17 08:53:15 mailnews.database.dbcache INFO DBs open in a window: 0, DBs open: 2, DBs already closing: 1

2019-05-17 08:54:15 mailnews.database.dbcache INFO Periodic check of cached folder databases (DBs), count=3

2019-05-17 08:54:15 mailnews.database.dbcache DEBUG Skipping, DB not open for folder: Posteingang

2019-05-17 08:54:15 mailnews.database.dbcache INFO DBs open in a window: 0, DBs open: 2, DBs already closing: 1

2019-05-17 08:55:15 mailnews.database.dbcache INFO Periodic check of cached folder databases (DBs), count=8

2019-05-17 08:55:15 mailnews.database.dbcache DEBUG Skipping, DB not open for folder: Posteingang

2019-05-17 08:55:15 mailnews.database.dbcache INFO DBs open in a window: 0, DBs open: 7, DBs already closing: 1

2019-05-17 08:56:15 mailnews.database.dbcache INFO Periodic check of cached folder databases (DBs), count=15

2019-05-17 08:56:15 mailnews.database.dbcache DEBUG Closing expired DB for folder: Postausgang

2019-05-17 08:56:15 mailnews.database.dbcache DEBUG Skipping, DB not open for folder: abc

2019-05-17 08:56:15 mailnews.database.dbcache DEBUG Skipping, DB not open for folder: xxx

2019-05-17 08:56:15 mailnews.database.dbcache DEBUG Skipping, DB not open for folder: m

2019-05-17 08:56:15 mailnews.database.dbcache DEBUG Skipping, DB not open for folder: Rxxx

2019-05-17 08:56:15 mailnews.database.dbcache DEBUG Skipping, DB not open for folder: urlaub

2019-05-17 08:56:15 mailnews.database.dbcache DEBUG Skipping, DB not open for folder: friends

2019-05-17 08:56:15 mailnews.database.dbcache DEBUG Skipping, DB open in window for folder: 1 mal 1

2019-05-17 08:56:15 mailnews.database.dbcache INFO DBs open in a window: 1, DBs open: 7, DBs already closing: 7

Comment 28

[:aceman]

(In reply to UliM from comment #27)

Thanks, so your maximum count of folders in the time of high CPU load was only 15.

(reply to comment22)
Update of current status with TB:

• After start of program, it runs for about 1-2 minutes with high load (ca. 15% with 8-core system, it runs singlecore I assume). Responsiveness of TB in this time frame is bad.

And even this initial slowness is solved when you increase mail.db.idle_limit to above 300 000?

• Afterwards, CPU load drops and responsiveness is ok.
  This is much better than 1/2 year ago, when it took more than 15 mins. to go back to a usage mode.

Nice to hear. Did you upgrade from TB52 to TB60 in this timeframe?

Comment 29

[:aceman]

Just to make sure, everybody has mail.db.max_open set to at least the default of 30 ?

Comment 30

[UliM]

(in reply to comment 28)
Currently I have TB 60.6.1 (32-Bit) on the release channel (dont know what I had back then, but I am always on the release channel).

I just tested again with mail.db.idle_limit of 300.000 (default) and 30.000.000. In both cases, TB runs for about 2 1/2 Minutes with high CPU load (and bad responsiveness). I cannot detect a difference here, the setting seems to be irrelevant.

In the first 150 seconds, the Process Explorer shows:

• constant CPU load,
• a lot of I/O in the first 45 seconds, but then no more I/O
• however, memory slowly grows, and drops in steps (like a zig-zag curve).
  (can I upload a performance graph screenshot somewhere? https://imgur.com/a/5ZUShDl )

Afterwards, nearly no CPU, nearly no I/O, constant memory usage.

What is TB doing with the memory?

max_open is the default of 30.

Comment 31

[MHW]

(In reply to UliM from comment #30)

(in reply to comment 28)
Currently I have TB 60.6.1 (32-Bit) on the release channel (dont know what I had back then, but I am always on the release channel).

I just tested again with mail.db.idle_limit of 300.000 (default) and 30.000.000. In both cases, TB runs for about 2 1/2 Minutes with high CPU load (and bad responsiveness). I cannot detect a difference here, the setting seems to be irrelevant.

In the first 150 seconds, the Process Explorer shows:

• constant CPU load,
• a lot of I/O in the first 45 seconds, but then no more I/O
• however, memory slowly grows, and drops in steps (like a zig-zag curve).
  (can I upload a performance graph screenshot somewhere? https://imgur.com/a/5ZUShDl )

Afterwards, nearly no CPU, nearly no I/O, constant memory usage.

What is TB doing with the memory?

max_open is the default of 30.

Yes, mine is set to 30.

Comment 32

[MHW]

(In reply to :aceman from comment #29)

Just to make sure, everybody has mail.db.max_open set to at least the default of 30 ?

Yes, mine is set to 30. (Meant to put that here.)

Comment 33

[Wolf Rudolf]

(In reply to :aceman from comment #22)

cokefrigefan, Wolf Rudolf, MHW, UliM, Scott, could you guys enable the logging temporarily per comment 4 and tell us the count of folders being open/purged there?

2019-05-21 20:01:31 mailnews.database.dbcache INFO Periodic check of cached folder databases (DBs), count=15
2019-05-21 20:01:31 mailnews.database.dbcache DEBUG Skipping, DB open in window for folder: @t-online.de
2019-05-21 20:01:31 mailnews.database.dbcache INFO DBs open in a window: 1, DBs open: 14, DBs already closing: 0

I also have a lot of entries (about one every second) like this:
2019-05-21 19:59:31 gloda.index_msg WARN Observed header that claims to be gloda indexed but that gloda has never heard of during compaction. In folder: mailbox://nobody@Local%20Folders/%40t-online.de sketchy key: 36132114 subject: Retourenlabel

Comment 34

[Wayne Mery (:wsmwk)]

Wolf, (or anyone - but at least someone :)

Can you run the performance profiler, using some modified value of mail.db.idle_limit (that doesn't make the problem entirely go away)?

You must be using Thunderbird 68 or newer - betas from https://www.thunderbird.net/en-US/channel/ or current nightly build from https://archive.mozilla.org/pub/thunderbird/nightly/latest-comm-central/
Install profiler add-on into thunderbird 68 (or newer https://www.thunderbird.net/en-US/channel/ ) - get the add-on file from https://github.com/firefox-devtools/Gecko-Profiler-Addon/blob/master/gecko_profiler.xpi?raw=true and in Tools > add-ons click the gear to install add-on from file
Follow instructions at https://profiler.firefox.com/ (videos BASED ON FIREFOX at https://profiler.firefox.com/docs/#/./videos-intro )
Create a profiler URL and post it here.

Comment 35

[Wolf Rudolf]

@Wayne Mery
I didn't notice any performance problems yet after upgrading to TB68. I tried to follow the instructions for the profiler but could't get it to work in Thunderbird.

Comment 36

[MHW]

(In reply to Wayne Mery (:wsmwk) from comment #34)

Wolf, (or anyone - but at least someone :)

Can you run the performance profiler, using some modified value of mail.db.idle_limit (that doesn't make the problem entirely go away)?

You must be using Thunderbird 68 or newer - betas from https://www.thunderbird.net/en-US/channel/ or current nightly build from https://archive.mozilla.org/pub/thunderbird/nightly/latest-comm-central/
Install profiler add-on into thunderbird 68 (or newer https://www.thunderbird.net/en-US/channel/ ) - get the add-on file from https://github.com/firefox-devtools/Gecko-Profiler-Addon/blob/master/gecko_profiler.xpi?raw=true and in Tools > add-ons click the gear to install add-on from file
Follow instructions at https://profiler.firefox.com/ (videos BASED ON FIREFOX at https://profiler.firefox.com/docs/#/./videos-intro )
Create a profiler URL and post it here.

Hi @Wayne

I had forgotten about this until Wolf's post came through. I've followed your instructions and created a profile at https://perfht.ml/2riB7ai. I have no idea if it is a long enough snapshot or if it has the details you are looking for. Please let me know if you'd like me to run another and with what details. I'll be glad to do whatever will help you.

BTW, I have had no issues with TB 68.2.2. It is working like a charm.

Comment 37

[MHW]

BTW, I didn't change the value of mail.db.idle_limit as I wasn't sure what to change it to, as I'm no longer seeing the Golda issue...

Comment 38

[Wayne Mery (:wsmwk)]

(In reply to MHW from comment #37)

BTW, I didn't change the value of mail.db.idle_limit as I wasn't sure what to change it to, as I'm no longer seeing the Golda issue...

right click in config editor and pick "reset"

Comment 39

[MHW]

(In reply to Wayne Mery (:wsmwk) from comment #38)

(In reply to MHW from comment #37)

BTW, I didn't change the value of mail.db.idle_limit as I wasn't sure what to change it to, as I'm no longer seeing the Golda issue...

right click in config editor and pick "reset"

Okay, I reset mail.db.idle_limit. It's at 300000. Ran profiler for a few minutes while downloading emails... Profile is at: https://perfht.ml/2r6j9rG

I hope that helps.

Comment 40

[myelement88]

Hello!

changing the mail.db.idle_limit to a much higher amount does not work for me. Also i encountered this issue the first time with 68.2.x. Before i have never noticed a high CPU load. Will do a profiling as stated form Wayne in comment #34 shortly.

I also did try mailnews.database.dbcache.logging.console/dump from "none" to "show" like mentioned in comment #27.

Should i revert all made changes back to default before i do that profiling? Or does it make no difference? (CPU load is always high after about 30 minutes with or without these changes.)

If there is something else i can do to help or solve the issue please let me know.

PS: Only thing i can do right now is to give Thunderbird a lower process priority to take not so much CPU load.. which is not really great solution.

Comment 41

[myelement88]

(In reply to myelement88 from comment #40)

Will do a profiling as stated form Wayne in comment #34 shortly.

Profiling done with default values. Here's the link: https://perfht.ml/38riYYC

What i noticed so far is, that it only does record in 20 second intervals... so is it important to catch the moment when CPU load gets up?

Comment 42

[myelement88]

What i noticed so far is, that it only does record in 20 second intervals... so is it important to catch the moment when CPU load gets up?

A edit option for Bugzilla would be great... so third comment:

Profiling done with catching the 'starting point of chaos' - also with default values: https://perfht.ml/35a0pGt

Comment 43

[naught]

Not sure how related this is to the original bug report, but I'm seeing constant 100% CPU in thunderbird when I have a Search Folder open (this particular one looks at messages <30 days old that are unread in 17 mailboxes across 3 accounts). As soon as I change the view to another mail box, the CPU usage drops to <10%. If I leave it open, it stays at 100% indefinitely.

mail.db.idle_limit doesn't seem to have an effect.

Comment 44

[myelement88]

Update: After updating to version 68.4.2 today the issue was the same, high CPU load after a few minutes. So i decided to redo my whole profile. I copied the 'Mail' folder only and then decided on file per file basis which i also would need/want according to mozillazine.org. And that worked, at least the last few hours. No high CPU load any more.

I am keeping the old profile if i can help with some tests to find the issue.

Comment 45

[Mast]

On my MacOSX 10.13 I have been on TB 60.9.1 for a while because all versions after 60.x were using the CPU while idle too much. I'm surprised people are happy with 4% CPU usage at idle (albeit, it's better than 70%!) ... 4% is very much on laptops because it's enough to prevent the CPU from entering a low power state and hence it consumes more power and heats more.

On MacOSX I expect idle apps to "nap", and TB was indeed entering "app nap" state after a short while of being idle, and was using practically 0% CPU.

Recently 60.9.1 also started to use 4% CPU at idle ... and I have no idea why. I disabled all add-ons. I tried mail_idle limit to 300k and 30m. 300k seems better in my case.

I have big mail folders. 3 IMAP accounts, each with 50,000+ emails (between inbox and sent), and 2 more with about 5-10k emails. The only change was that I switched from Gmail to Posteo (and had a glass of wine to celebrate), so I wiped my Gmail IMAP account and added a new IMAP account. Can't see why this would affect anything.

What's the best debugging procedure?

Comment 46

[myelement88]

Update: First I'd like to mention that the cpu is different than in my last comment (it's now an i7-8565U / HP ProBook 650G5 / Windows 10 Pro - Version 10.0.18362 Build 18362 ). Also Thunderbird now runs on x64 architecture (current version is 68.10.0).

After a few weeks/months the Thunderbird cpu usage when idle goes again up to 14%. Sometimes it drops to 6% but never below that on AM. On PM the cpu usage is much lower, about 1%.

For me it looks like a daily routine where Thunderbird is reindexing the whole account profile including all mails.

Comment 47

[MHW]

Just for the record, I've not had an issues with Thunderbird since my last post here 8 months ago. TB is currently idling at 0% to 1%. As of 4 days ago I've upgraded to TB 78. No issues, let alone any relating to this thread. I do have 3 POP3 accounts and total emails file size is 3.39 GB. I'm very grateful for TB, shudder the thought of having to put up with Outlook or webmail. Perhaps there are other alternatives, but I'm quite happy with TB.

Comment 48

[UliM]

Update: since my recent upgrade from TB 68 to 78, all my memory / CPU load issues are gone. TB starts fast now (with 8 GB of emails) and is responsive from the beginning. Thank you for your good job on this! I also very much like the new OpenPGP integration. Great to see that TB evolving - I will happily give a donation for such an important open source project.

Comment 49

[m]

Running Thunderbird on iMac i9, and the CPU usage is between 8% and 16%, sometimes going up to 25%. Increasing mail.db.idle_limit didn't help.

Here is the link to profiler report https://share.firefox.dev/2TCLZuo

A friend of mine briefly looked into it, and this attracted his attention:

• Excess calls to PresShell::Paint when idle
• nsRefreshDriver::tick too 'active'
• Too often emitting refresh event.

I hope I conveyed correctly, what he noticed.

Any advice?

Comment 50

[Wayne Mery (:wsmwk)]

(In reply to myelement88 from comment #44)

...
I am keeping the old profile if i can help with some tests to find the issue.

If you still have the profile, and can reproduce the problem using version 78, a performance profile would be nice using https://github.com/thunderbird-conversations/thunderbird-conversations/wiki/Profiling-Conversation's-Performance

Comment 51

[Wayne Mery (:wsmwk)]

(In reply to naught from comment #43)

Not sure how related this is to the original bug report, but I'm seeing constant 100% CPU in thunderbird when I have a Search Folder open (this particular one looks at messages <30 days old that are unread in 17 mailboxes across 3 accounts). As soon as I change the view to another mail box, the CPU usage drops to <10%. If I leave it open, it stays at 100% indefinitely.

mail.db.idle_limit doesn't seem to have an effect.

Given the last statement, you're in the wrong bug report. If you can still reproduce an issue when using version 78 please file a new bug report and include a performance profile using https://github.com/thunderbird-conversations/thunderbird-conversations/wiki/Profiling-Conversation's-Performance

Comment 52

[Wayne Mery (:wsmwk)]

Scott,
Wolf,

If you are using version 68, or can get on 78, a performance profile would be great.
See https://github.com/thunderbird-conversations/thunderbird-conversations/wiki/Profiling-Conversation's-Performance

(In reply to Wolf Rudolf from comment #35)

@Wayne Mery
I didn't notice any performance problems yet after upgrading to TB68. I tried to follow the instructions for the profiler but could't get it to work in Thunderbird.

Comment 53

[Wayne Mery (:wsmwk)]

(In reply to Mast from comment #45)

What's the best debugging procedure?

If you you can reproduce using version 78 please create a profile https://github.com/thunderbird-conversations/thunderbird-conversations/wiki/Profiling-Conversation's-Performance. If changing mail.db.xxx settings hasn't helped, then you should create a new bug report rather than post here.

Comment 54

[David Rosenstrauch]

I recently upgraded to TB 78(.4.3) from TB 68, and TB has been sucking up 100% of an entire CPU core ever since. No idea what it's doing or why. I disabled the Indexer, and also tried updating mail.db.idle_limit to 30000000 as well. No change. Profile here: https://share.firefox.dev/3kOQ4Xu

Would love to find some way to get this fixed, as it's making my desktop nearly unusable.

Comment 55

[MHW]

(In reply to David Rosenstrauch from comment #54)

I recently upgraded to TB 78(.4.3) from TB 68, and TB has been sucking up 100% of an entire CPU core ever since. No idea what it's doing or why. I disabled the Indexer, and also tried updating mail.db.idle_limit to 30000000 as well. No change. Profile here: https://share.firefox.dev/3kOQ4Xu

Would love to find some way to get this fixed, as it's making my desktop nearly unusable.

While this may not be directly helpful, I've not had any issues with TB since 12 months ago. I was having the same memory issues you describe. It's been a wile so details are a bit fuzzy, but I do recall that issues were a pain until resolved once a newer TB sub-version was issued. I'm currently using TB 78.5.0 on Win10 system and TB idles consistently at < 1%. Not sure what Wayne would say, but I'd think it would be smart to start with a clean install of TB. If you've already done that and are still having issues then I'd just keep problem-solving. Be methodical and keep good notes. Best wishes.

Comment 56

[David Rosenstrauch]

Hmmm ... and today, Thunderbird cpu is normal. Weird. Will report back if I see it again.

Comment 57

[David Rosenstrauch]

Scratch that. I'm still seeing the issue. It seems to be happening intermittently. Will try to pin down the exact conditions that are causing it.

Comment 58

[David Rosenstrauch]

Here's another profile I grabbed during a period of pinned CPU:

https://share.firefox.dev/2KBnUD5

Running TB 78.4.3 on Arch Linux.

Step taken were:

• update mail.db.idle_limit to 3 x 10^7
• update mailnews.database.global.logging.dump to true
• start TB with $ thunderbird -console
• let it run for a while
• after a while it starts to download msgs from a folder on my IMAP account. (Note that I yesterday deleted and then recreated that account in TB, as well as deleted the global-messages-db.sqlite file.)
• then ... watch TB eat my CPU. (See http://darose.net/ThunderbirdCPU.png)

Any idea what's causing this? I definitely was not experiencing this until I upgraded to TB 78. (From 68.12.0.)

Comment 59

[David Rosenstrauch]

FYI, I upgraded to TB 78.5.0 last night. That version doesn't seem to be showing the issue anymore so far. Will report back if I see otherwise.

Comment 60

[David Rosenstrauch]

"Otherwise". :-(

TB 78.5.0 has been exhibiting same behavior - sucks up 100% of a CPU core for no apparent reason. Wish there was some fix for this. It pretty much makes the app unusable ...

Comment 61

[Wolf Rudolf]

(In reply to Wayne Mery (:wsmwk) from comment #52)

Scott,
Wolf,

If you are using version 68, or can get on 78, a performance profile would be great.
See https://github.com/thunderbird-conversations/thunderbird-conversations/wiki/Profiling-Conversation's-Performance

(In reply to Wolf Rudolf from comment #35)

@Wayne Mery
I didn't notice any performance problems yet after upgrading to TB68. I tried to follow the instructions for the profiler but could't get it to work in Thunderbird.

I haven't had any problems like ethis in a while (TB 68.10.0) even with mail.db.idle_limit = 300000. Are you still interested in a performance profile?

Comment 62

[David Rosenstrauch]

Interesting data point: I noticed today that TB was exhibiting the CPU issue when I'm attached to my USB docking station (which connects the machine to an HDMI screen, and an ethernet connection). But when I disconnected, and went back to using the laptop on wifi, the issue stopped. Is this just coincidence, or could there somehow be a relation here?

Comment 63

[Magnus Melin [:mkmelin]]

Tried with/without hardware acceleration enabled (through the Thunderbird options)

Comment 64

[David Rosenstrauch]

Disabling hardware acceleration using the 2 preferences layers.acceleration.disabled and gfx.direct2d.disabled, as described in https://www.askvg.com/how-to-disable-hardware-acceleration-gpu-rendering-in-mozilla-firefox-4-0-to-fix-font-problem/ seems like it may have finally fixed the issue for me.

Comment 65

[David Rosenstrauch]

Nah. Still pinning CPU.

Comment 66

[David Rosenstrauch]

Only actual workaround seems to be to unplug the HDMI external monitor. CPU drops instantly as soon as I do that.

Comment 67

[David Rosenstrauch]

Actually, the issue is even easier to reproduce than that. No need to unplug my HDMI monitor or disconnect from the USB dock. Just open the lid on my laptop and CPU instantly drops dramatically in both TB and FF. Close the lid and CPU shoots up.

Comment 68

[David Rosenstrauch]

My apologies - after much digging, I've determined that this is not a TB issue. Apparently my laptop bios seems to cap the CPU frequency at 400MHz whenever I close the lid.

Comment 70

[Wayne Mery (:wsmwk)]

(In reply to Wolf Rudolf from comment #35)

@Wayne Mery
I didn't notice any performance problems yet after upgrading to TB68. I tried to follow the instructions for the profiler but could't get it to work in Thunderbird.

I haven't had any problems like ethis in a while (TB 68.10.0) even with mail.db.idle_limit = 300000. Are you still interested in a performance profile?

Wolf, no, thanks.

But are things still good with version 91? And, had you changed anything significant prior to your performance returning to normal?

Comment 71

[Wayne Mery (:wsmwk)]

Jaun, can you describe your accounts / folder counts?

Comment 72

[Wolf Rudolf]

(In reply to Wayne Mery (:wsmwk) from comment #70)

But are things still good with version 91? And, had you changed anything significant prior to your performance returning to normal?

I had reset mail.db.idle_limit = 300000 and haven't seen the problem for a while. Also no problem in TB 91.

Comment 73

[blowe]

Hi, I've been having high CPU for a while, I just updated to Thunderbird 91.4.1 and still seeing 12% to 16% CPU when idle. I tried the idle_limit fix and no improvement.

Nothing shows in the bottom status bar as processing.

I ran a profile, loaded here https://share.firefox.dev/3mMFK67, hopefully it is helpful. Let me know if I can do anything else.

Thanks and happy 2022,
Brian Lowe

Comment 74

[blowe]

(In reply to blowe from comment #73)

Hi, I've been having high CPU for a while, I just updated to Thunderbird 91.4.1 and still seeing 12% to 16% CPU when idle. I tried the idle_limit fix and no improvement.

Nothing shows in the bottom status bar as processing.

I ran a profile, loaded here https://share.firefox.dev/3mMFK67, hopefully it is helpful. Let me know if I can do anything else.

Thanks and happy 2022,
Brian Lowe

By the way I'm in Windows 10, 64 bit.

Comment 75

[myelement88]

(In reply to Wayne Mery (:wsmwk) from comment #50)

(In reply to myelement88 from comment #44)

...
I am keeping the old profile if i can help with some tests to find the issue.

If you still have the profile, and can reproduce the problem using version 78, a performance profile would be nice using https://github.com/thunderbird-conversations/thunderbird-conversations/wiki/Profiling-Conversation's-Performance

No, I can not reproduce the problem, it seems to be solved. Thanks!

Comment 76

[ineiti]

I'm a bit late to the party, but I still have Thunderbird using 50% of my CPU constantly. I'm using Thunderbird 91.5.0 (64 bits) on MacOSX Monterey 12.1 on an Intel MacBook Pro. I tried to increase mail_idle_limit with an additional 0, but the CPU usage is still the same. So I did a performance profile that you can see here:

https://share.firefox.dev/3GpQT4F

Thanks for any help you can give,

Linus

Comment 77

[mach]

It's constantly using some CPU on Windows 11 as well for me. Email is not being download or anything, but it is constantly using CPU for the main inbox account. Thunderbird 91.5.0 (64-bit).

https://share.firefox.dev/34OtcoP

Thanks for looking into this.

Comment 78

[Daniel Margoliash]

I am newbie here. Same problem: MacOS 12.1, Tbird 91.5, on an 3 year old MacBook Air at idle Tbird CPU usage hovers at 50-75%. This happened after upgrade to Monterey (I believe).

Comment 79

[Daniel Margoliash]

(In reply to Daniel Margoliash from comment #78)

I am newbie here. Same problem: MacOS 12.1, Tbird 91.5, on an 3 year old MacBook Air at idle Tbird CPU usage hovers at 50-75%. This happened after upgrade to Monterey (I believe).

CPU usage is back to normal. I am not aware that I did anything to account for the change. Just a guess, maybe there was some background processing that finally completed. Good for me, but apologies that I cannot provide any useful insight.

Comment 80

[Don Holeman]

Found my way here after discovering my default email account is consuming between 6% and 12% cpu and 39% ram (6GB!) and won't let go of either. This is on an Asus laptop. Here is a profile, hope it helps.
https://share.firefox.dev/3FcOigK

Comment 81

[Wayne Mery (:wsmwk)]

In recent years there are still reports at https://rainbow.chard.org/2013/02/19/thunderbird-high-cpu/ where increasing mail.db.idle_limit helps.

Comment 82

[Wayne Mery (:wsmwk)]

blowe,
ineiti,
mach,
Don,
Does this still reproduce for you with version 115?

Comment 83

[Don Holeman]

Wayne, I'm not sure but it might. On 12-24-2023, two days ago, I had to reinstall my bios yet again to solve a problem with my laptop fan spinning up and never winding down. These were the symptoms when I reported here last year. It happened again a couple of times since then, the only fix is to reinstall the bios which I do now as a routine matter when it happens (without checking for a cause so I didn't check Tbird ram or cp consumption. That is to say I don't know whether the overspin is due to a Tbird update but since you asked here is a snapshot just now.

https://share.firefox.dev/3RDKSIl

Comment 84

[blowe]

Version 115 runs about 10-12% CPU during idle on my Windows 10 based system

Processor AMD Ryzen 3 3200U with Radeon Vega Mobile Gfx 2.60 GHz
Installed RAM 32.0 GB (29.9 GB usable)
System type 64-bit operating system, x64-based processor

I switched to Mailspring a few months ago so I can leave mail open in the background without the fan running.

Comment 85

[ineiti]

I'm currently running 120.0b7 (64-bit) on an Apple M2 Max and don't have any problems anymore. So for me this is fixed, according to the comments, not for others :(

Comment 86

[Wayne Mery (:wsmwk)]

(In reply to Don Holeman from comment #83)

https://share.firefox.dev/3RDKSIl

Did you intend to profile Firefox instead of Thunderbird?

Comment 87

[Don Holeman]

Oops. Here is the Thunderbird performance profile. Let me know if it is not sufficient.

https://share.firefox.dev/3GZ5Lcf

Comment 88

[Wayne Mery (:wsmwk)]

(In reply to Don Holeman from comment #87)

Oops. Here is the Thunderbird performance profile. Let me know if it is not sufficient.
https://share.firefox.dev/3GZ5Lcf

The expert julienw on matrix write, "slowness but not when idle ... looks like long restyles / long displaylist in the first one ... I also see a lot of css transitions, are they all really useful?" ... which would not be related to the original premise of this bug report. Although there are no updates from the reporter after comment 7.

Comment 89

[GoldenBugatti]

Having sporadic but extremely serious performance problems, slows down too much to even type an email and need to force quit. Tried troubleshooting extensions and couldn't find a likely culprit. Upping mail.db.idle_limit doesn't help.

Comment 90

[gene smith]

(In reply to GoldenBugatti from comment #89)

Having sporadic but extremely serious performance problems, slows down too much to even type an email and need to force quit. Tried troubleshooting extensions and couldn't find a likely culprit. Upping mail.db.idle_limit doesn't help.

When you see this is the "progress bar" over active (throbbing or pulsing), i.e, in an indeterminate state? I've noticed that when the progress bar is in this state (e.g., when a connection to a server is not responding) that CPU shows well over 100% in linux "top" display. This could be slowing down other actions that TB needs to do.
Just to maybe be clear, the "progress bar" is typically located inside the bottom "status" line at the far right bottom corner.