Closed Bug 723016 Opened 12 years ago Closed 12 years ago

Thunderbird hogging CPU after upgrade to Version 10 (Gloda reindexing after schema change occurs without prior warning)

Categories

(Thunderbird :: Search, defect)

Product:
Thunderbird
Component:
Search
Version:
10 Branch
Platform:
x86_64
Windows 7
Type:
defect
Priority:
Not set
Severity:
major

Tracking

(Not tracked)

Status:
RESOLVED INVALID

People

(Reporter: stephan, Unassigned)

References

(
URL
)

Details

(Keywords: perf, uiwanted, ux-userfeedback, Whiteboard: [gs]) 

Just upgraded to version 10.0 of Thunderbird.

Machine performance plummets.

View of task manager indicates that Thunderbird, while not seemingly DOING anything, is chewing up anywhere from 55%-65% of the CPU cycles.
Same problem for me.
Just upgraded from 9 to 10 and my cpu is always at 30%-40% and hard disk is running all the time. My OS is Windows XP SP3.
yes, this clearly needs fixing. same problem here.
i just upgraded to 10.0 and thunderbird.exe eats up around 25% of my cpu. also, it keeps the hard-disk busy. my OS is XP SP3 on an hp elitebook (with L9440 1.86GHz Core2 Duo CPU).
check tools | error console ... do you see indexing activity?  I suspect (hope) so.

some period of indexing activity will be "normal" when updating to version 10 because improvements to global indexing/searching (aka gloda) required a change to the database schema, which in turn require your message store to be reindexed.  (this would have also happened when you first updated to 3.x from version 2, and iirc going from version 3.x to version 5)

It should stop once your messages are fully reindexed. And, presumably, this indexing activity is supposed to be throttled to some degree when you are actively using thunderbird so that thunderbird remains reasonable responsive. (the same cannot be said however when you are not actively using thunderbird - indexing rate is supposed to throttle up, so that messages get indexed faster)

QUESTION - IF the performance of the PC during indexing is materially affecting your productivity/workflow, please describe 
a) what you are doing (window name/menu item/etc) in thunderbird that is being affected
b) age and type of machine (laptop) you are on
c) resource usage of both CPU, memory and read/write activity as reported by your OS's task manager
Keywords: perf
Can we relnote schema changes in Gloda when they hit future releases? Even though there was a brief message in the Error Console that the schema has changed and thus the database needs reindexing (I don't have Gloda enabled = no problem), it would be good as a heads-up for people that it's going to happen with the update.
alright, cpu consumption and hard-disk activity are indeed back to normal. sorry for the trouble. but i do agree with rsx11m. a heads-up about the upcoming indexing activity would have been nice.
(Solved) I left my laptop on for 2 hours and now everything seems to work fine. Surely it was a Gloda reindexing.
(Suggestion) A warning message to inform users about it.
(In reply to Wayne Mery (:wsmwk) from comment #3)
> check tools | error console ... do you see indexing activity?  I suspect
> (hope) so.

It is in tools->Activity manager.
(In reply to :aceman from comment #8)
> (In reply to Wayne Mery (:wsmwk) from comment #3)
> > check tools | error console ... do you see indexing activity?  I suspect
> > (hope) so.
> 
> It is in tools->Activity manager.

I have been following this issue my two cents:

- I found that Thunderbird was indexing my GLODA database but I didn't notice it because my computer seems to be powerfull to do this without problems, but probably, it's a better idea show a warning message, a pop-up or a status bar message... 

- Wayne,I think that not everybody knows what the activity manager is or what it is used for?

My two cents about this...

Best Regards!
Component: General → Search
QA Contact: general → search
Summary: Thunderbird hogging CPU after upgrade to Version 10 → Thunderbird hogging CPU after upgrade to Version 10 (Gloda reindexing after schema change occurs without prior warning)
I think something should be done to warn the user, especially if you consider enterprise's users migrating there xxGo of mails from TB3.1 to TB10. They need to know that Thunderbird won't be available during some minutes/hours.
(In reply to caméléon from comment #10)
> They need to know that Thunderbird won't be available during some
> minutes/hours.

TB is available to use at any time even before the messages are reindexed. (I'd just imagine the search would not work during that period).
It is just that some advanced users can notice it is using CPU and disk heavily for unknown reasons.

I agree there could be some notice to the user (maybe status bar message). Yes, there is the Activity manager but not everybody will find it.
Keywords: uiwanted, ux-userfeedback
(In reply to :aceman from comment #11)
> TB is available to use at any time even before the messages are reindexed.
Well, GSFN users seems to have a different advice and find Thunderbird simply not usable:
http://getsatisfaction.com/mozilla_messaging/topics/updated_thunderbird_today_and_now_it_just_hangs

For some others, it seems it can cause crash during reindexing http://www.geckozone.org/forum/viewtopic.php?f=4&t=102331
(In reply to :aceman from comment #11)
> It is just that some advanced users can notice it is using CPU and disk
> heavily for unknown reasons.

Well, all you need is an older or low-end processor and indexing is a substantial drag on performance. Now one can argue that other software does some background housekeeping as well (including system components like .NET which seem to need some "optimization" after certain Windows updates), but such activity should be more discoverable if it's occupying substantial resources.

> I agree there could be some notice to the user (maybe status bar message).

That's what was done in early versions by showing some "Indexing x of y" which was considered an annoyance by other users. Maybe as a compromise, have some time-deferred message on the status bar "Working on the database" or simply an announcement "Need to do some database housekeeping, this may take a while" when a schema change was detected (optionally with a "Details" link opening the Activity Manager, or something like that).
Thanks for reporting. The database upgrade is actually designed to yield better performance in the end, by trimming unnecessary information from the database, thus making the database smaller, faster, and more responsive.

It is true that we could have somehow given a heads-up warning, but the same things happens when you set up a new account with Thunderbird. Should we pop-out something saying "Beware! This is software! It's about to do STUFF!"? ;-)
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → INVALID
Note that I have just written a new revision (waiting for approval) of the SUMO article dedicated to TB10 in order to explain this behavior. Maybe it will help some people to better understand why Thunderbird may be slow.

The article is there: https://support.mozillamessaging.com/en-US/kb/new-thunderbird-10/history
Wayne, I suggest we create another bug like "Make sure Gloda sends a message through the notification bar when throwing away the database.". We could say something like "Thunderbird is performing a database upgrade to ensure better performance and responsiveness. Please bear with us while the operation is in progress. You can monitor activity through Tools > Activity Manager.".
That's a good thing, thanks.
(In reply to Jonathan Protzenko [:protz] from comment #16)
> Wayne, I suggest we create another bug like "Make sure Gloda sends a message
> through the notification bar when throwing away the database.". We could say
> something like "Thunderbird is performing a database upgrade to ensure
> better performance and responsiveness. Please bear with us while the
> operation is in progress. You can monitor activity through Tools > Activity
> Manager.".

Yes, I like this idea. And, it would also be nice to alert the user so if they do have a problem, when they file a bug or support post they can *state a useful context*.

On the flip side
- we don't have a way to inform the user when indexing is done
- the vast, vast majority of users don't ever see a problem
- is it worth alerting users with trivially sized message stores?
We could do some heuristics, like « if the gloda size is greater than a given threshold (e.g. 100MB) before performing the upgrade, warn the user in a vague but informative way ». And users can still check out the activity manager to see if there's still some heavy indexing going on...
(In reply to Jonathan Protzenko [:protz] from comment #19)
> We could do some heuristics, like « if the gloda size is greater than a
> given threshold (e.g. 100MB) before performing the upgrade, warn the user in
> a vague but informative way ». And users can still check out the activity
> manager to see if there's still some heavy indexing going on...

..And something like doing the Gloda modification before starting the user interface, in a similar way that a lot of windows installer do...for example?
If you're willing to wait for possibly a couple hours for your Thunderbird to start up, why not?
(In reply to Jonathan Protzenko [:protz] from comment #21)
> If you're willing to wait for possibly a couple hours for your Thunderbird
> to start up, why not?

Not I'm not willing to wait a couple hours for the Thunderbird startup... I'm just givin ideas about how to attack and solve this issue...
(In reply to caméléon from comment #12)
> For some others, it seems it can cause crash during reindexing
> http://www.geckozone.org/forum/viewtopic.php?f=4&t=102331

caméléon can you post a relevant bug# here, and if there is not a bug report for it, please file one. Thanks.
(In reply to Wayne Mery (:wsmwk) from comment #23)
> (In reply to caméléon from comment #12)
> > For some others, it seems it can cause crash during reindexing
> > http://www.geckozone.org/forum/viewtopic.php?f=4&t=102331
> 
> caméléon can you post a relevant bug# here, and if there is not a bug report
> for it, please file one. Thanks.

The reporter doesn't see the crash reporter windows, and no crash reports have been recorded, so I am afraid that we cannot have enough feedback to file a bug :-(
Well, I have started a spreadsheet trying to summarize Thunderbird behavior depending of the computer and Gloda size. Please feel free to add you own values.

Step to reproduce:
- close Thunderbird
- delete the file global-messages-db.sqlite in your profile
- open Thunderbird
- monitor indexing thought the activity manager 
- try to write a new mail, open address-book, just to see if Thunderbird is usable during indexing process.
- wait until indexing process finished
- write all your observation in this spreadsheet: https://docs.google.com/spreadsheet/ccc?key=0AqTs3W0yLftDdGFLR3lUcVVmZ3k1RzI3U3BLWlVaS0E
(In reply to caméléon from comment #25)
> Well, I have started a spreadsheet trying to summarize Thunderbird behavior
> depending of the computer and Gloda size. 
>..
> - write all your observation in this spreadsheet:
> https://docs.google.com/spreadsheet/
> ccc?key=0AqTs3W0yLftDdGFLR3lUcVVmZ3k1RzI3U3BLWlVaS0E

http://www.belarc.com/free_download.html is a handy tool one can use to get machine details.  For example disk model # and manufacture, which can be googled to get disk RPM spec
The reindex slowed down my computer noticebly for a couple of hours. 

I suggest to fork re-indexing into a separate process and nice that one down, so it stops stealing CPU from other apps.
You need to log in before you can comment on or make changes to this bug.