1560084 - High CPU usage for hours or several days for 400k message folder (Search and indexing turned off. Activity Manager empty)

Status: RESOLVED INCOMPLETE

(Thunderbird :: Untriaged, defect)

Description

User Agent: Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:67.0) Gecko/20100101 Firefox/67.0

Steps to reproduce:

Replroduction is very difficult, and yet so easy.
I have like 7 milaservers configured, with millions of messages.
Sometimes, few times per day, sometimes once per week thunderbird start using 100% (or multiples of that) cpu. Letting it burn for several days makes no difference, it appears never to stop.
Search and indexing is turned off.

Actual results:

The CPU uses 100% (2, 300%) CPU and slows everything down.

Expected results:

I think thunderbird should at least tell me what it is doing (using claws-mail I can open a network-log which than shows me activity, and I could explain that). Thunderdbird has no such thing, so I cannot know what it is doing.

Comment 1

[Wayne Mery (:wsmwk)]

(In reply to Arjen Heidinga from comment #0)

...
I think thunderbird should at least tell me what it is doing (using claws-mail I can open a network-log which than shows me activity, and I could explain that). Thunderdbird has no such thing, so I cannot know what it is doing.

tools > Activity Manager will show some things.

Sometimes, few times per day, sometimes once per week thunderbird start using 100% (or multiples of that) cpu. Letting it burn for several days makes no difference, it appears never to stop.
Search and indexing is turned off.

Please try the beta from https://www.thunderbird.net/en-US/channel/ Take particular note of the release notes https://www.thunderbird.net/en-US/thunderbird/68.0beta/releasenotes/

Comment 2

[Arjen Heidinga]

I am using the beta as of now. The interface overall fells 'snappier' inied. However, opening a large folder makes is complete unresponsive.
The activity window is empty, and shows nothing.

Comment 3

[Benjamin Flanagin]

How large is the large folder? If you can reproduce the error it might be worth looking into as a possible regression.

Its possible that the issue is the same in both versions of TB, but the outcome is different due to code changes.

Arjen, if the large folder issue is reproducible would you mind running a repair on that folder and checking if anything changes?

That should narrow things down a bit.

Comment 4

[Arjen Heidinga]

Sorry for the delay... Work happened.

It ap[pears the beta is coping better, but still is affected. The folder was repaired, took the whole night, next day killed it, restarted the repair, couple of hours later is started (re)downloading messages.

The folder is 426151 messages in size.

Comment 5

[Wayne Mery (:wsmwk)]

The only time I've seen extended CPU usage is when gmail redownloads mail (or Thunderbird reindexes a folder, which cannot be happening here), which would show up in Tools > activity manager.

So you have 7 imap accounts? All imap? Any gmail? Is the mail stored on local disk (not a file share)?

Comment 6

[Benjamin Flanagin]

To add to Wayne's information request:

Do you have any filters setup to run automatically? Do you do spam filtering on those millions of emails via TB or do you use an external service?

Comment 7

[Arjen Heidinga]

Spam filtering is done before it arrives. The spam filter is enabled, without actions except flagging. (the default)
On some folders I have enabled automatic deletion after 30 days.

I must say, now I use thunderbird 60. I appear to see this behaviour less. I saw more full lockups (needing a kill, sigterm).

Comment 8

[Wayne Mery (:wsmwk)]

Still questions...

(In reply to Wayne Mery (:wsmwk) from comment #5)

So you have 7 imap accounts? All imap? Any gmail? Is the mail stored on local disk (not a file share)?

And nothing in the error console?

Comment 9

[Wayne Mery (:wsmwk)]

Still need more info from reporter

Comment 10

[Stephen D. Williams]

This bug is back with
Indexing Inbox, as seen in Activity Manager, is very slow, taking many days on an imap4 account with 1.2M messages.