superbad UI performance, fairly unresponsive while Compact Folders plus indexing

RESOLVED FIXED in Thunderbird 3.0rc1

Status

--
major
RESOLVED FIXED
9 years ago
9 years ago

People

(Reporter: wsmwk, Assigned: asuth)

Tracking

({perf})

Thunderbird 3.0rc1
x86
Windows XP
Bug Flags:
blocking-thunderbird3 -

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [no l10n impact][need to attempt to reproduce now that gloda correctness has landed])

Attachments

(1 attachment)

(Reporter)

Description

9 years ago
Created attachment 404462 [details]
Actiivity mgr screen shots (ignore cpu chart)

superbad UI performance, fairly unresponsive while Compact Folders plus indexing in progress

STR
1. initiated compacting
2. something triggered indexing scan

almost impossible to do anything in UI

cpu usage looks fine for index activity, i.e. not abnormally high.

guess based on status bar, activity mgr and  is 
* indexing scan started after Compact Folders was initiated
* no significant fetching messages while it compacted

per AMgr screen shot, it does not look like any sync activity occurred during the period, except 2 messages from inbox filtered to "moz" folder. indexing  continued through the top of the AM log.

in addition, perhaps indexing should not be permitted during compact because compact of folder is known to cause bad problems associated with locked folder.

xref bug 495572, which doesn't mention performance
 "gloda should explicitly handle compaction"
Flags: blocking-thunderbird3?
(Reporter)

Comment 1

9 years ago
implicit, but I should be explicit, is that compact too much longer than normal to run. normal is 30-60 seconds to iterate 42 folders. it's not clear to me which operation impacted the other most, including something other than compact and index, or if it was mutual destruction.
Whiteboard: [no l10n impact]
Assignee: nobody → bugmail
Flags: blocking-thunderbird3? → blocking-thunderbird3+
(Assignee)

Updated

9 years ago
Whiteboard: [no l10n impact] → [no l10n impact][check after gloda correctness patch lands]
Target Milestone: --- → Thunderbird 3.0rc1
(Assignee)

Comment 2

9 years ago
The gloda correctness patch has landed on trunk and the comm-1.9.1 branch and should hopefully resolve this issue for you.  Please attempt to reproduce tomorrow with new nightlies.
Whiteboard: [no l10n impact][check after gloda correctness patch lands] → [no l10n impact][need to attempt to reproduce now that gloda correctness has landed]
Believed "good enough" now that correctness has landed.  Please renominate if that's not correct.
Flags: blocking-thunderbird3+ → blocking-thunderbird3-

Comment 4

9 years ago
Not sure if this is this bug, but compacting my *primary inbox* still takes too long (longer than compacting any other folders & inboxes) and freezes not only Thunderbird's UI but all my applications + my OS.

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6pre) Gecko/20091108 Lightning/1.0pre Shredder/3.0pre

General info:          IMAP, SSL/TLS, mark as deleted
IMAP server directory: [blank]
Next 3 checkboxes:     all checked
personal namespace:    "#mh/","#mhinbox",""
Public (shared):       "#public/","#news.","#ftp/","#shared/"
Other users:           "~"
[x] Allow server to override these namespaces
(Reporter)

Comment 5

9 years ago
(In reply to comment #4)
> Not sure if this is this bug, 

It's not if you see this problem with indexing turned off.


> but compacting my *primary inbox* still takes too long 

"too long" is much to vague on which to diagnose an issue


> (longer than compacting any other folders & inboxes) 

This bug doesn't care what folder in being indexed


>  freezes not only Thunderbird's UI but all my applications + my OS.

Sounds like you have issues that go beyond Thunderbird. 
Can you reference what "support people" have said about this?

Comment 6

9 years ago
(In reply to comment #5)
> (In reply to comment #4)
> > Not sure if this is this bug, 
> It's not if you see this problem with indexing turned off.

I see this bug with indexing off (I just tested it - incl. a restart of Tb).

> > but compacting my *primary inbox* still takes too long 
> "too long" is much to vague on which to diagnose an issue

Approx. 7 seconds to compact one e-mail. Another inbox: <<1 second

> > (longer than compacting any other folders & inboxes) 
> This bug doesn't care what folder in being indexed

Then it's another bug. But which? It seems a very important issue.

> >  freezes not only Thunderbird's UI but all my applications + my OS.
> Sounds like you have issues that go beyond Thunderbird. 
> Can you reference what "support people" have said about this?

This phenomenon only occurs within Thunderbird when compacting my primary inbox. It happens *no* other time in *no* other application. 

I am the "support people". ;-)

Comment 7

9 years ago
(In reply to comment #4)
> compacting my *primary inbox* still takes too long 

Oh, and there's a *huge* amount of (i.e., constant) hard-drive thrashing during the 7+ seconds of compacting.
(Reporter)

Comment 8

9 years ago
(In reply to comment #2)
> The gloda correctness patch [bug 465618] has landed on trunk ...  Please attempt to reproduce
> tomorrow with new nightlies.

Andrew, looks good. I tested 20091107 nightly on the same desktop and don't see extreme badness. I attempted 3 scenarios:
1. compacting my 1gb of local folders - which should have taken a long time but for some reason did not ... odd
2. imap (the basis of comment 0)
3. rss - which I can normally count on for longish compact runs, and it didn't let me down.

To "initiate" indexing I can normally trigger by clicking RSS folders - which again didn't disappoint.  So I did have indexing occurring while some folder compacts were in progress.  Although I am not 100% sure I replicated the original conditions, I am satisfied enough to declare this is FIXED.

Can you clarify one point, so that I know what to look for in the future ... am I correct that the patch does not prevent *all* compaction during indexing, nor does it prevent *all* indexing during a compact ... it only prevents compaction and indexing in the same folder at the same time?

That is what I take away from the tests, and the comment
 // Stay out of folders that:
   3.296 -      // - are compacting
   3.297 +      // - are compacting / compacted and not yet processed
Status: NEW → RESOLVED
Last Resolved: 9 years ago
Depends on: 465618
Resolution: --- → FIXED
(Assignee)

Comment 9

9 years ago
(In reply to comment #8)
> Can you clarify one point, so that I know what to look for in the future ... am
> I correct that the patch does not prevent *all* compaction during indexing, nor
> does it prevent *all* indexing during a compact ... it only prevents compaction
> and indexing in the same folder at the same time?

You interpret correctly.  Gloda will index while compaction is occurring; it just stays out of folders that are being compacted.  The idea is that (thanks to rkent) the adaptive indexer will basically idle the indexing process while the compaction is doing heavy work.

(In reply to comment #7)
> Oh, and there's a *huge* amount of (i.e., constant) hard-drive thrashing during
> the 7+ seconds of compacting.

You should spin this off into a separate bug or find an existing bug and be sure to note how many messages are in the inbox versus your comparison folder.
You need to log in before you can comment on or make changes to this bug.