833661 - Thunderbird hangs regularly for 5-10 seconds due to large folder(s)

Status: RESOLVED DUPLICATE of bug 816327

(Thunderbird :: Search, defect)

Description

[Jonathan Kamens]

For months I've been seeing regular hangs of 5-10 seconds in Thunderbird trunk builds.

Blake Winton suggested installing the Gecko Profiler, and I did that, and lo and behold, a few seconds after I activated it, I saw another hang. I dumped the profile immediately after TB unhung, and the vast majority of the time was spent in gloda_worker_messageIndex.

I've uploaded the profile and linked to it below.

I'm going to try turning off global search and running without it enabled for a while to confirm that it is the cause of the hangs.

Any other information I can provide or anything else I can do to help track this down, please let me know.

Comment 1

[Jonathan Kamens]

Apparently it's not just Gloda. With global search turned off, I just got another hang. This one shows the vast majority of the time inside "Timer::Fire", but there's no granularity about what within Timer::Fire is causing the hang. Here's the link to the uploaded profile:

http://people.mozilla.com/~bgirard/cleopatra/?1358914082240#report=de12e29b6398c22604123a04fe876e56da79e3f1

Comment 2

[Wayne Mery (:wsmwk)]

Jonathan, 
- On average, how often do you see this performance hit - every hour, half hour?
- You say you've been seeing this for months - implying there was a time when you didn't see this. Can you estimate please what version/month you started seeing it?

Comment 3

[Wayne Mery (:wsmwk)]

(didn't intend to mark this New)

Comment 4

[Jonathan Kamens]

It comes and goes, but when it's happening, it's every couple minutes, i.e., quite frequently.

I tried to use old nightly builds to bisect when it started happening, but I couldn't come up with anything definitive, so I don't have a good answer about when it started.

Comment 5

[Mike Conley (:mconley) (:⚙️)]

Jonathan:

Which version of the Profiler are you using? Do you have the Chat feature enabled in TB?

I ask, because I know that for a while, the Profiler caused a major performance regression on start-up after the Chat module had initialized.

-Mike

Comment 6

[Jonathan Kamens]

Gecko Profiler 1.11.9.

I only just installed the profiler yesterday. The problem has been happening for months. I don't think the profiler is the problem.

I don't know whether I have Chat "enabled". I've never used it and don't have any chat accounts configured.

Comment 7

[Mike Conley (:mconley) (:⚙️)]

(In reply to Jonathan Kamens from comment #6)
> Gecko Profiler 1.11.9.
> 
> I only just installed the profiler yesterday. The problem has been happening
> for months. I don't think the profiler is the problem.
> 
> I don't know whether I have Chat "enabled". I've never used it and don't
> have any chat accounts configured.

Ok, good to know.

Let's get a little data here:

- How many accounts do you have in your profile?
- How many total folders?
- Can you give an estimate of how much mail you have?
- You're running Nightly - do you experience these problems on the current release (17.0.2)?

Comment 8

[Andrew Sutherland [:asuth] (he/him)]

I've seen comments from Taras about a big improvement in the profiler on linux coming up soon... maybe whatever he is talking about will fix the suspiciously missing native stack frames from all over the place?

For the gloda profile case, it seems very likely from the JS stack (but again, native stacks are missing), that gloda is opening a huge folder synchronously like gmail's all mail.  Or just hitting some bad I/O waits because of other stuff going on on the system.  Bug 551209 is about doing that asynchronously and is fairly low hanging fruit.

Comment 9

[Jonathan Kamens]

> - How many accounts do you have in your profile?

2 IMAP (1 of them GMAIL), 1 POP3 which I rarely use.

> - How many total folders?

Around 60.

> - Can you give an estimate of how much mail you have?

The Mail directory in my profile is 2.3MB; the ImapMail directory is 1.9GB. I just did File | Compact Folders and the ImapMail directory went down to 1.6GB.

> - You're running Nightly - do you experience these problems on the current
> release (17.0.2)?

I don't experience the problem in _either_ the current release or Daily anymore, now that I've shrunk my Gmail "All Mail" folder down to 70.15MB. At the same time as doing that, I also turned of global search for a while and then turned it back on. I don't know which of these, or both in combination, made the problem go away.

Comment 10

[Wayne Mery (:wsmwk)]

bug 551209 will no doubt help, but I think the proximate cause here is bug 816327

Comment 11

[Wayne Mery (:wsmwk)]

(In reply to Jonathan Kamens from comment #9)
> I don't experience the problem in _either_ the current release or Daily
> anymore, now that I've shrunk my Gmail "All Mail" folder down to 70.15MB. At
> the same time as doing that, I also turned of global search for a while and
> then turned it back on. I don't know which of these, or both in combination,
> made the problem go away.

Jonathan, has your problem continued to be in remission?

Comment 12

[Jonathan Kamens]

Hasn't recurred.