Closed Bug 1657565 Opened 4 years ago Closed 4 years ago

thunderbird freeze high CPU sorting Subject, Date or Correspondent - "Not Responding"

Categories

(Thunderbird :: Folder and Message Lists, defect, P1)

Product:
Thunderbird
Component:
Folder and Message Lists
Type:
defect

Tracking

(thunderbird_esr68 unaffected, thunderbird_esr78+ affected, thunderbird80 affected)

Status:
RESOLVED DUPLICATE of bug 1658062
Tracking Flags:
Tracking Status
thunderbird_esr68 --- unaffected
thunderbird_esr78 + affected
thunderbird80 --- affected

People

(Reporter: krasnyitoster, Unassigned)

References

Details

(4 keywords)

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:79.0) Gecko/20100101 Firefox/79.0

Steps to reproduce:

We have IMAP account and Thunderbird 78.1.0. There is about 50000 messages in INBOX. When I click to Correspondent, or Topic, or Data, Thunderbird will freeze on 1-5 minute or more.

Actual results:

Thunderbird will freeze for a long time (1-5 minutes or more for some users). If we delete Thunderbird and profile, install Thinderbird 86.10.0, configure profile, sorting will be work very quick. Problem is in Thunderbird 78.0.1, 78.1.0.

Expected results:

Thunderbird 78 should not freeze, when sorting many messages.

Please Start Windows' safe mode with networking enabled

Also, how big are your address books?
Any calendar or LDAP?

Flags: needinfo?(krasnyitoster)
Keywords: perf

Sounds like bug 1657034.

We tried to go Windows safe mode and run Thunderbird in safe mode. There is no changes. Sorting will freeze for a long time, or hang up at all.
By the way: we have a lot of computers in office. We have this problem on Windows 7 32-bit and Windows 10 at the same time. Problem only in 78 version of Thunderbird. We tried to install 68 version of Thunderbird and problem have gone. Address book is not very big (users have about 100-600 recipients). Problem is only if folder have many messages (30000 or more). I tried disable hardware acceleration in Setting, but it not help.
Configuration of computers: 3-4 GB DDR2 / DDR3, Pentium Dual-Core E5400-E5700 2.7-3.0 GHz.

Flags: needinfo?(krasnyitoster)

We have not any calendar or LDAP.

(In reply to krasnyitoster from comment #0)

We have IMAP account and Thunderbird 78.1.0. There is about 50000 messages in INBOX. When I click to Correspondent, or Topic, or Data, Thunderbird will freeze on 1-5 minute or more.

Guess you mean Subject and Date in the message list.

I can easily reproduce this. 80.0b1, Macbook air, 25k message in folder, clicked correspondent (which I rarely do) - I'm waiting now several minutes. Basically hung.

Tested with 800 messages - there is a detectable delay (.5 - 1.5 sec) switching between correspondents and subject.

With 2.5k messages it was obvious - 1-2 seconds. It worked several times, but for some reason eventually hung after a two quick clicks - I think it was correspondent and then subject.

I had another go at the 800 message draft folder - flipping between subject and correspondent I got it to delay several seconds - which was more than the first test.

Blocks: tb78found
Severity: -- → S2
Status: UNCONFIRMED → NEW
status-thunderbird80: --- → affected
status-thunderbird_esr68: --- → unaffected
status-thunderbird_esr78: --- → affected
tracking-thunderbird_esr78: --- → +
Component: Untriaged → Folder and Message Lists
Ever confirmed: true
Flags: needinfo?(geoff)
Keywords: hang
Priority: -- → P1
Summary: thunderbird freeze → thunderbird freeze high CPU sorting Subject, Date or Correspondent - "Not Responding"

Alice, can you reproduce?
Not sure what we did in this area except bug 1626841.

Flags: needinfo?(alice0775)
Keywords: regression, regressionwindow-wanted

I now think this is more likely to be contact / addressbook related - a caching issue maybe?

I cannot reproduce it when clicking between columns that don't involve contacts - for example size, subject, date

(In reply to Magnus Melin [:mkmelin] from comment #7)

Alice, can you reproduce?
Not sure what we did in this area except bug 1626841.

I cannot reproduce the issue.
There seems to be no difference in CPU load and processing time between 68 and 78.
Tested with imap inbox(190mails) and local folder(5370mails)

Flags: needinfo?(alice0775)

I'm also often seeing delays when deleting messages.

Walt or Eckard, do you have any folders in the over 2k range?

Flags: needinfo?(wls220spring)
Flags: needinfo?(de.berberich)

(In reply to Wayne Mery (:wsmwk) from comment #10)

I'm also often seeing delays when deleting messages.

Walt or Eckard, do you have any folders in the over 2k range?

Only one email, four newsgroup folders over 2k in TH 78.1.1.

TB 78.1.1 froze when I sorted my Bugzilla email folder by Correspondent, but doesn't appear to be reproducible after resorting by date.

Flags: needinfo?(wls220spring)

https://github.com/thunderbird-conversations/thunderbird-conversations/wiki/Profiling-Conversation's-Performance describes the procedure for getting a performance profile. Let's compare results.

Flags: needinfo?(wls220spring)
Flags: needinfo?(krasnyitoster)
Flags: needinfo?(de.berberich)

Profiles:

  1. a few seconds for 2500 messages - https://share.firefox.dev/2XI3cVB - Inbox, 2500 messages, sorted by subject, start profiler, then sort by correspondent, stop
  2. high gc/cc 1-2 minutes for 5000 messages - https://share.firefox.dev/3kr9faY - crash folder, 5000 messages, sorted by subject, start profiler, then sort by correspondent, stop

Note - case #2 was actually a short wait. Just previous to that I had caused Thunderbird to hang, and had to restart.
Possible complication - I'm finding google oauth2 prompt for accounts which are not logged in when I start the developer. I think it's hitting js that triggers checking these accounts for new messages. I see high CPU when oauth prompt is open.

As Wayne cc'ed me I took a quick look.

(In reply to Wayne Mery (:wsmwk) from comment #14)

  1. a few seconds for 2500 messages - https://share.firefox.dev/2XI3cVB - Inbox, 2500 messages, sorted by subject, start profiler, then sort by correspondent, stop

From a profile perspective, there isn't loads of processing in here. There's an ~800ms segment where TB is at 100% CPU, that mainly looks like the view sorting algorithm, and various calls to decode headers. This would need someone a bit more experienced/up to date with the views to look at.

  1. high gc/cc 1-2 minutes for 5000 messages - https://share.firefox.dev/3kr9faY - crash folder, 5000 messages, sorted by subject, start profiler, then sort by correspondent, stop

There's about 7s worth of CPU where javascript is running at 100% here. Probably around 300ms are the same as per the first case. Of the rest, there's a few interesting things:

  • The _prefBranch and uuid fields on the address book directory get called quite a bit ~1.3s worth for the _prefBranch, though only 70ms worth for uuid. These the patch in bug 1658062 should help.
  • getIntValue on the card is called a lot to get the position, ~672ms worth I think. This calls through to the _prefBranch to get an integer value - so caching the position rather than getting it every time would help a lot.
  • cardForEmailAddress takes ~1900ms, most of this is getCardFromProperty which is taking ~1500ms.
    • It looks like cardForEmailAddress may call getCardFromProperty twice, maybe unnecessarily especially as that makes two calls (could combine into one SQL).
    • May be worth considering the use of db.executeCached() here, places uses it a lot. I don't know if it will help or not, might be worth talking to some of the storage team about it.
  • On top of those, it may be worth looking at what the DBViewWrapper is actually doing wrt the sort, and why it appears to get all the message fields again to do the sort. I don't know enough about it to know if this is just how the tree view works, or if there's something that could be done to cache/improve it.

For the rest of the time ~3 minutes, it looks like the garbage collector is cleaning up. I would expect that fixing some/all of the above should help that quite a bit. It'd certainly be interesting to see what impact landing bug 1658062 has on it.

Depends on: 1658062

(In reply to Mark Banner (:standard8) (afk until 13 Aug) from comment #15)

As Wayne cc'ed me I took a quick look.

(In reply to Wayne Mery (:wsmwk) from comment #14)

  1. a few seconds for 2500 messages - https://share.firefox.dev/2XI3cVB - Inbox, 2500 messages, sorted by subject, start profiler, then sort by correspondent, stop

From a profile perspective, there isn't loads of processing in here. There's an ~800ms segment where TB is at 100% CPU, that mainly looks like the view sorting algorithm, and various calls to decode headers. This would need someone a bit more experienced/up to date with the views to look at.

profile has:
14 address books with 2500 total contacts .
Corporate ldaps
Plus Mac AB? (0 contacts)

Note, I can make the 2500 message Inbox hang like case #2, but I didn't what to risk hanging thunderbird.

Blocks: 1649103
See Also: → 1582578
See Also: 1582578 →

krasnyitoster, how is version 78.2.0 for you?

Flags: needinfo?(krasnyitoster)

(In reply to Wayne Mery (:wsmwk) from comment #18)

krasnyitoster, how is version 78.2.0 for you?

We are rolling out new version of Thunderbird. I will write later, when I get some statistics from users.
Thank you for help.

Flags: needinfo?(krasnyitoster)

Seems to work fine. Thanks for help.

Thanks for the update

Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.