Closed Bug 794401 Opened 12 years ago Closed 11 years ago

Thunderbird freezes or is extremely slow, using 100 % CPU, when loading folders or message headers

Categories

(Thunderbird :: Mail Window Front End, defect)

Product:
Thunderbird
Component:
Mail Window Front End
Version:
15 Branch
Platform:
x86
Linux
Type:
defect
Priority:
Not set
Severity:
major

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: federicoleva, Unassigned)

References

(
URL
)

Details

(Keywords: perf, regression, Whiteboard: [regression:TB15])

Attachments

(1 file)

gecko.png
211.76 KB, image/png
Details 
Split from bug 787348: upgrading to 15.0.1 didn't help.
I'm on fedora 17, AMD E350, 4 GiB RAM.

I use Thunderbird with SMTP and POP for some email addresses, I have some dozens of folders totalling about 2,4 GiB in size.

Since I "upgraded" to 15, almost everything I do is extremely slow and takes 100 % CPU for several second before completion: for instance, scrolling the list of folders to change folder (up to 10-15 s if the folders I want to see haven't been shown for some minutes), or scroll the list of messages to select a message, or loading the quoted text when replying to a message (5-10 seconds). This happens consistently, every time I perform such an action.
This is extremely annoying: not only I lose a lot of time but I often end up on the wrong folder and so on.
Depends on: 787348
Severity: normal → major
Keywords: hang, perf, regression
Whiteboard: regression:TB15
Rkent, can this be a manifestation of bug 793455? If we close DBs after 3 seconds reopening them again can take a while when they are as big as the reporter has (I can confirm opening a large folder with 600000msgs/2GB takes some seconds).

The automatic closing of DBs after 3 seconds (instead of minutes) was introduced in bug 723248 that went into TB15 that is why I am suspecting it.

But does scrolling the folder list pane cause the DBs to be reopened?
Component: General → Mail Window Front End
No longer depends on: 787348
Keywords: hang
Target Milestone: Thunderbird 15.0 → ---
(In reply to :aceman from comment #1)
> Rkent, can this be a manifestation of bug 793455? If we close DBs after 3
> seconds reopening them again can take a while when they are as big as the
> reporter has (I can confirm opening a large folder with 600000msgs/2GB takes
> some seconds).

Yes. Technically we check open DBs every minute, and if they have been open more than 3 seconds and there are no other references to them, they are closed. The "3 seconds" has now been changed to the original intent of 5 minutes.

> 
> The automatic closing of DBs after 3 seconds (instead of minutes) was
> introduced in bug 723248 that went into TB15 that is why I am suspecting it.
> 
> But does scrolling the folder list pane cause the DBs to be reopened?

Yes the folder pane seems to interact with the database in some cases. It really shouldn't, but I think that it does.
(In reply to Irving Reid (:irving) from bug 793455 comment #4)
> I don't think we need to worry about people who have overridden this pref,
> so just changing the default is fine.

So for now I've done exactly this, setting the pref to the most recent default value of 5 min; I'll let you know if this is enough to fix it.
(In reply to Kent James (:rkent) from comment #2)
> > But does scrolling the folder list pane cause the DBs to be reopened?
> Yes the folder pane seems to interact with the database in some cases. It
> really shouldn't, but I think that it does.

You are right, just expanding accounts/folders in the folder pane, without clicking any of them causes TB to open SOME of the corresponding .msf files of the exposed folders.
(In reply to Federico from comment #3)
> (In reply to Irving Reid (:irving) from bug 793455 comment #4)
> > I don't think we need to worry about people who have overridden this pref,
> > so just changing the default is fine.
> 
> So for now I've done exactly this, setting the pref to the most recent
> default value of 5 min; I'll let you know if this is enough to fix it.

Setting it to 5 min might have increased the period of grace, but outside that timespan 15 is still horribly slower than 14, comment 0 still applies.
Federico, what does it mean exactly? Does TB work fine and quickly inside the 5 minutes but then slows down again? After a bit of slowdown (opening folders) does it again work quickly for another 5 minutes?
(In reply to :aceman from comment #6)
> Federico, what does it mean exactly? Does TB work fine and quickly inside
> the 5 minutes but then slows down again? After a bit of slowdown (opening
> folders) does it again work quickly for another 5 minutes?

Yes, it looks so: I haven't used Thunderbird with a timer in my hand, but it /seems/ to be not slow sometimes in some cases. This non-slowness is anyway very rare: for instance, when I finish reading an email, replying to it (i.e. quoting its text) is already very slow.
Should I try and set mail.db.idle_limit to something like 500 minutes to see if DB closure is the source of the problem or will this cause disruption? Of course I'd like to avoid things like bug 794398 or worse cases of database corruption.
Yes, of course try that. Just remember the pref is in milliseconds so for 500 minutes give the value as 30000000 (500 * 60 * 1000).
(In reply to :aceman from comment #8)
> Yes, of course try that. Just remember the pref is in milliseconds so for
> 500 minutes give the value as 30000000 (500 * 60 * 1000).

500 min have surely not passed and everything is slow again now that I opened TB after lunch.
I have also restarted it to check the new pref, just in case.
If you see in tools->Activity manager is there any operation in progress (like Indexing)?

Also go into Help->troubleshooting info and paste the exact "Application Build ID" here. Also see the Graphics section there if there isn't any GPU acceleration enabled.
Thank you for your continued assistance!

(In reply to :aceman from comment #10)
> If you see in tools->Activity manager is there any operation in progress
> (like Indexing)?

Nothing there, except messages download.

> 
> Also go into Help->troubleshooting info and paste the exact "Application
> Build ID" here. 

20120911155028

> Also see the Graphics section there if there isn't any GPU
> acceleration enabled.

Do you mean "windows with GPU acceleration"? It says 0/4. Pasting the original text in Italian:

Descrizione scheda grafica: X.Org -- Gallium 0.4 on AMD PALMID 
produttore: .Org
ID dispositivo: PALM
Versione driver: 2.1 Mesa 8.0.4
Rendering WebGL: X.Org -- Gallium 0.4 on AMD PALM -- 2.1 Mesa 8.0.4
Finestre con accelerazione GPU: 0/4
OK, looks like proper 15.0.1.

Have you already tried TB Safe mode (help->restart with addons disabled)?
(In reply to :aceman from comment #12)
> OK, looks like proper 15.0.1.
> 
> Have you already tried TB Safe mode (help->restart with addons disabled)?

Did so now, but it's definitely worse, probably because it's ignoring all settings as far as I can see.
Strange, it should only be faster as it does not load addons.

How much RAM is TB using?
(In reply to :aceman from comment #14)
> Strange, it should only be faster as it does not load addons.

But it uses the default 3 s setting to close DB.

> How much RAM is TB using?

I've just restarted it and it's in normal mode now:

15:46:18          PID  minflt/s  majflt/s     VSZ    RSS   %MEM  Command
15:46:18        15198      0,35      0,00  754260 328540   8,79  thunderbird
(In reply to Federico from comment #15)
> (In reply to :aceman from comment #14)
> > Strange, it should only be faster as it does not load addons.
> But it uses the default 3 s setting to close DB.
Why do you think so? prefs should be honored even in safe mode.
Just to be clear, the original timed closing of the db was meant to be a failsafe in case somebody left the database open inadvertently. The database will still get closed when the reference count goes to zero, or the folder db is nulled by some other code.

So setting this preference is not some assurance that the database does not get closed in that interval.

My Bug 792915 - "Remove mDatabase from folder object" is one proposal to actively manage the lifetime of summary db objects.
(In reply to :aceman from comment #16)
> (In reply to Federico from comment #15)
> > (In reply to :aceman from comment #14)
> > > Strange, it should only be faster as it does not load addons.
> > But it uses the default 3 s setting to close DB.
> Why do you think so? prefs should be honored even in safe mode.

Well, the UI turns to English, size of columns is not the usual one, etc.
You probably have your UI language as an addon (language pack) and it got disabled in Safe mode. I am not sure about the columns, that should probably not happen.
Then see in preferences->advanced->general->config editor what is the value of the idle pref, while you are in Safe mode.
(In reply to :aceman from comment #19)
> You probably have your UI language as an addon (language pack) and it got
> disabled in Safe mode. I am not sure about the columns, that should probably
> not happen.

Ok.

> Then see in preferences->advanced->general->config editor what is the value
> of the idle pref, while you are in Safe mode.

I had done so and it was correct (30000000), but I've no idea if it was respected.

All these tests around this preference are probably useless after comment 17 I guess... Either the closure of DBs should be made less aggressive or the opening should be more efficient.
Or closing the DBs has nothing to do with your problems :(
All prefs should be respected even in Safe mode.
(In reply to :aceman from comment #21)
> Or closing the DBs has nothing to do with your problems :(

this is very possible. 
- for example if the problem is mainly cpu (and not memory) some AV software has been known to cause high TB CPU (at least per taskmgr).
- for example there are recent high memory usage reports not (yet) linked to being caused by open folder dbs
(In reply to :aceman from comment #21)
> Or closing the DBs has nothing to do with your problems :(

Changing mail.db.idle_limit did help a bit, so it seems definitely related. Otherwise how can it be explained that after scrolling the folders list once doing it again is no longer slow for a few seconds?

(In reply to Wayne Mery (:wsmwk) from comment #22)
> this is very possible. 
> - for example if the problem is mainly cpu (and not memory) some AV software
> has been known to cause high TB CPU (at least per taskmgr).

I have no AV.

> - for example there are recent high memory usage reports not (yet) linked to
> being caused by open folder dbs

Memory is not a problem.
TB 15 also occasionally freezes when trying to move messages to another directory: with right click and "move to" the dropdown never appears (and I had to restart it), with dragging the cursor disappears while hovering the directories list.
The bug still strike **** 16.0.2.
Why is this bug still marked UNCONFIRMED? What are you waiting for exactly? The problem has been confirmed since comment 1!
URL: http://forums.mozillazine.org/viewtop...
Priority: P1 → --
Whiteboard: regression:TB15 → [regression:TB15]
The topic is not really relevant to this discussion, anyway I confirm that after downgrading to 12.0 (very easy on fedora) everything is fine again; I'll stick to 12 until this bug is fixed.

(In reply to Federico from comment #25)
> The bug still strike **** 16.0.2.
> Why is this bug still marked UNCONFIRMED? What are you waiting for exactly?
> The problem has been confirmed since comment 1!

ditto
Federico can you reproduce this with version 17?


> (In reply to Federico from comment #25)
> > The bug still strike **** 16.0.2.
> > Why is this bug still marked UNCONFIRMED? What are you waiting for exactly?
> > The problem has been confirmed since comment 1!
> 
> ditto

confirmed means someone other than the reporter can reproduce.
Flags: needinfo?(federicoleva)
(In reply to Wayne Mery (:wsmwk) from comment #27)
> Federico can you reproduce this with version 17?

I'm using 12 now, I'm not very fond of the idea of upgrading and downgrading again at every TB release until someone marks this bug confirmed... But maybe I'll find the time to try once.

> > (In reply to Federico from comment #25)
> > > The bug still strike **** 16.0.2.
> > > Why is this bug still marked UNCONFIRMED? What are you waiting for exactly?
> > > The problem has been confirmed since comment 1!
> > 
> > ditto
> 
> confirmed means someone other than the reporter can reproduce.

And :aceman reproduced it.
Flags: needinfo?(federicoleva)
given comment 5 I would say this was caused by bug 793455 - DB cache is closing databases too frequently (regression in 15.0)  which is fixed in version 16.  

Therefore, please try version 17 and if it works you can set the status to resolved and resolution to worksforme. If it doesn't work, then we can move forward to other possibilities.
Flags: needinfo?(federicoleva)
Whiteboard: [regression:TB15] → [closeme 2013-01-18][regression:TB15]
(In reply to Wayne Mery (:wsmwk) from comment #29)
> given comment 5 I would say this was caused by bug 793455 - DB cache is
> closing databases too frequently (regression in 15.0)  which is fixed in
> version 16.

I've already tested that fix and it didn't work, see comment 25... I'm more and more confused, what else do you want me to test?
Flags: needinfo?(federicoleva)
(In reply to Federico from comment #30)
> (In reply to Wayne Mery (:wsmwk) from comment #29)
> > given comment 5 I would say this was caused by bug 793455 - DB cache is
> > closing databases too frequently (regression in 15.0)  which is fixed in
> > version 16.
> 
> I've already tested that fix and it didn't work, see comment 25... I'm more
> and more confused, what else do you want me to test?

ah. so you did. my bad.

sounds a little like bug 563677.  but that bug is not a regression in version 15.

acelists, short of working out a regression range, what do  you think is best way forward - a profile?  In which case Federico will want to install https://addons.mozilla.org/en-us/thunderbird/addon/gecko-profiler/ and follow steps 3-8 of check steps 3-8 of http://mikeconley.ca/blog/2012/06/15/gecko-profiler-now-works-in-thunderbird-daily/
Whiteboard: [closeme 2013-01-18][regression:TB15] → [regression:TB15]
Federico, thunderbird profile is on a local drive in the computer, not on a network drive or USB?
(In reply to Wayne Mery (:wsmwk) from comment #31)
> sounds a little like bug 563677.  but that bug is not a regression in
> version 15.

I'm sure this bug started in 15. Does bug 563677 also involve things like slowing down scrolling of directories?

(In reply to Wayne Mery (:wsmwk) from comment #32)
> Federico, thunderbird profile is on a local drive in the computer, not on a
> network drive or USB?

Yes, local hard disk.
I have only identified potential problems, but I could not really the big slowness you mentioned.
carefully read and try the suggestion here, if you like:
https://bugzilla.mozilla.org/show_bug.cgi?id=615272#c10

note that you should select *each* folder you have before you do this.  if not, renaming panacea.dat may lose folder pretty names and you will see hash values (name on disk) and they will have to be renamed manually.  for newsgroups, it will be like a new subscription.
(In reply to alta88 from comment #35)
> carefully read and try the suggestion here, if you like:
> https://bugzilla.mozilla.org/show_bug.cgi?id=615272#c10

you are suggesting then, that Frederico is missing .msf files on startup?
Flags: needinfo?(alta88)
no, rather that panacea.dat is corrupted.  at startup .msf files are autocreated if missing..
Flags: needinfo?(alta88)
Thank you for clarifying your suggestion. That solved bug 829017; to know if this bug is solved by that, I'd need to upgrade again from 12 to 15+, but is it safe?
A weird directory appeared, named nstmp, with about 15k messages: they seem to be duplicates of other messages in the trash directory (which has 23k right now), what should I do?
of course you should update.  first, it isn't useful to talk about bugs in versions older than the release as they will not/can not be addressed meaningfully.

if you are able, empty trash and restart.  if it's still there, rename it and restart.  if nothing untowards happens, you can then delete it.

i suggest this bug be closed, as the cause/fix seems to be panacea.dat, in which case someone should implement the suggestion in bug 615272#c10.  or change getStringProperty() to be much smarter about going to the msgDb, as this can be very expensive given how nsITreeView works. the msgDb should really never have to be accessed for folderpane reasons.
(In reply to alta88 from comment #39)
> i suggest this bug be closed, as the cause/fix seems to be panacea.dat, in
> which case [A] someone should implement the suggestion in bug 615272#c10.  or
> [B] change getStringProperty() to be much smarter about going to the msgDb, as
> this can be very expensive given how nsITreeView works. the msgDb should
> really never have to be accessed for folderpane reasons.

Kent, Irving, what are your thoughts on [A] and [B] above?
Flags: needinfo?(kent)
I'm not really an expert on panacea.dat. alta88 from comments here, and jcranmer from previous work trying to replace it, are more familiar with it.

As for this bug, I don't see any confirmation that comment 37 is the cause, though I generally trust that alta88 is probably correct in what he says. In any case, the issue remains. So this bug should either be duped to a bug that proposes the fixes recommended in comment 35, or else this bug should be transformed into implementing those suggestions.
Flags: needinfo?(kent)
perhaps the OP can confirm or not if a new panacea.dat solved this report, as it did the one in comment 38.

the irony of the failure in getSmartFolderName(), which calls getStringProperty("smartFolderName"), is that there seems to be nothing in the codebase that _sets_ "smartFolderName".

the error can be simulated by going to a Unified folders view (mode='smart').  there is bad contextmenu checking and a rename of the top level Archive folder, eg, is mistakenly allowed which, if attempted, will result in errors spamming the console on every hover.
(In reply to alta88 from comment #42)
> perhaps the OP can confirm or not if a new panacea.dat solved this report,
> as it did the one in comment 38.

(In reply to alta88 from comment #39)
> of course you should update.  first, it isn't useful to talk about bugs in
> versions older than the release as they will not/can not be addressed
> meaningfully.
> 
> if you are able, empty trash and restart.  if it's still there, rename it
> and restart.  if nothing untowards happens, you can then delete it.

I did all this, and upgraded to 17.0.2.
I confirm the problems are still there (it shows in particular if Thunderbird is left alone for some minutes), and I also have in the error console:

Data e ora: 30/01/2013 00:03:54
Errore: [Exception... "Component returned failure code: 0x80004003 (NS_ERROR_INVALID_POINTER) [nsIMsgFolder.getStringProperty]"  nsresult: "0x80004003 (NS_ERROR_INVALID_POINTER)"  location: "JS frame :: chrome://messenger/content/folderPane.js :: getSmartFolderName :: line 2436"  data: no]
File sorgente: chrome://messenger/content/folderPane.js
Riga: 2438

Remember that I still have mail.db.idle_limit set to 30000000.
I'll revert to 12 soon, so if you need more tests be quick please. :-)
Thank you very much for your help in debugging this.
the error you posted is the topic, ie if it is fixed does the symptom persist.

if you have selected all folders/subfolders, closed Tb, renamed panacea.dat, and after a while this starts happening, that is odd.  surely you have restarted Tb 17 with all extensions disabled?  and View->Folders->All.  

do these errors spam the console continually without your hovering the folderpane?  is the Global Indexer enabled?  if it is, it needs to complete its indexing, which can take a long time.  hours if you have large folders and are upgrading that many versions.  uncheck it in Options Advanced and restart.

the codepath is not that complex.  something may be causing the folder tree to repaint constantly and call that function.  and it is specific to your profile.  beyond that, in boca al lupo..
First of all, the title of this bug should be reduced to "Thunderbird freezes or is extremely slow, using 100 % CPU." The current title is misleading as there is no evidence in the original post that this bug were specifically related to folder loading.

The bug has also been reported by others here: https://bugs.launchpad.net/ubuntu/+source/thunderbird/+bug/927515

I am also experiencing this bug with TB 17.0 on Linux Mint Debian Edition with an XFCE 4.8 desktop. 

What I have found is that CPU usage tends towards 100% on one core when displaying the contents of a message in the Message Pane (use F8 to switch view). When the Message Pane is empty or not visible it does not happen.

The behaviour happens both with IMAP and POP3 accounts and both with text and HTML messages.
Earlier suggested indexing or encryption mechanisms have nothing to do with it.

Workaround: Do not use the Message Pane. Use F8 to switch it off.
(In reply to Serge Stroobandt from comment #45)
> First of all, the title of this bug should be reduced to "Thunderbird
> freezes or is extremely slow, using 100 % CPU." The current title is
> misleading as there is no evidence in the original post that this bug were
> specifically related to folder loading.

The reporter as stated several times that folder scrolling is slow. This often directly related to folder/file opening.

> The bug has also been reported by others here:
> https://bugs.launchpad.net/ubuntu/+source/thunderbird/+bug/927515
> 
> I am also experiencing this bug with TB 17.0 on Linux Mint Debian Edition
> with an XFCE 4.8 desktop. 
> 
> What I have found is that CPU usage tends towards 100% on one core when
> displaying the contents of a message in the Message Pane (use F8 to switch
> view). When the Message Pane is empty or not visible it does not happen.
> 
> The behaviour happens both with IMAP and POP3 accounts and both with text
> and HTML messages.
> Earlier suggested indexing or encryption mechanisms have nothing to do with
> it.

If I may suggest - let's focus on testing the reporter's issue, and if it works to your benefit and others then great.  It's great to have other reports of problems. But reports that don't exactly match the reporter's in both symptoms and time frames may I fear further muddy the already dark waters.
 
> Workaround: Do not use the Message Pane. Use F8 to switch it off.


Federico, 

Could you test F8 please to see if and how it helps?

There are unanswered questions [1] but my sense with the information at hand is either a memory usage issue or related to folders/files opening and closing. We have perhaps ruled out panacea.dat.  So I think the next thing to do is get a msgdb log. Instructions at https://wiki.mozilla.org/MailNews:Logging

And fwiw, earlier you wrote "it's definitely worse [in safe mode], probably because it's ignoring all settings as far as I can see."  But you can always set mail.db.idle_limit after you start and it will take immediate effect, if I remember correctly.



[1] questions

* You are successfully using version 12 and it doesn't work in Tb15. Unless I am mistaken you haven't mentioned testing TB13 and TB14. Have you tested them?

* Comment 44 - going through https://wiki.mozilla.org/Thunderbird:Testing:Memory_Usage_Problems  should include the above questions

* As part of that, please elaborate on "Memory is not a problem." in response to 
> - for example there are recent high memory usage reports not (yet) linked to
> being caused by open folder dbs
Personally I think 750MB vsz, which you quoted earlier, is on the high side. Specifically, actual numbers are most helpful - how much memory is installed, and how much memory Thunderbird is using when you see performance problems?
Flags: needinfo?(federicoleva)
Well. I'll try an answer for him. My system is also at 100% CPU and insane loads cause of the newest Thunderbird.

16735 ibotpeac  20   0 1248m 524m  41m S 113.8  4.4   0:21.80 thunderbird

F8 didn't help at all. Load still stayed high and CPU usage still moved around from 100% to 120%

http://pastebin.com/UmSyYPEk msgdb log for about 30 seconds
connor, can you do a longer log, from startup to when performance starts being affected?  And please add ",timestamp" so we have a sense of frequency in given time period?

And can you comment on why template folder might be opened/closed so often?


There are very few bugs fixed in TB15 [1] that IMO would affect performance, and bug 723248 is one of those few. So provisionally marking this related to bug 723248

[1] https://bugzilla.mozilla.org/buglist.cgi?f10=CP&f1=OP&o3=substring&list_id=6235361&f8=short_desc&f0=OP&o9=allwordssubstr&resolution=FIXED&classification=Client%20Software&classification=Components&o2=anywordssubstr&f9=short_desc&j7=OR&f4=CP&query_format=advanced&j1=OR&f3=keywords&f2=short_desc&f11=CP&f5=CP&f6=OP&f7=OP&product=MailNews%20Core&product=Thunderbird&target_milestone=Thunderbird%2015.0&o8=substring
Blocks: 723248
Wayne,

I don't have the problem anymore. For some reason TB decided to re-index all 200,000 of my emails and somehow I lost my templates folder which was causing the opening/closing.

I talked with someone in #maildev and we pulled logs and couldn't figure out what caused it. I can provide more logs, but without the problem anymore I doubt it'll help.
I'm jumping in here to share my experience with *mail.db.idle_limit*. I had a feeling my laptop battery was burning out a bit faster than it was supposed to so I started looking for ways to keep the CPU hoggers down. Thunderbird seemed to be the main culprit, spiking the CPU uncomfortably often (not constantly though, like the bug title seems to suggest).

So I was thinking maybe someone has written some addon or something to turn off TB's expensive operations based on AC/DC power status. Googling for "thunderbird laptop battery" surprisingly turned up this blog post http://rainbow.chard.org/2013/02/19/thunderbird-high-cpu/. Turned out out to be revelation, that one.

After running with the 500min tweak for a day, this is what I wrote to the comments of that post:

"Okay, here’s a follow-up. Tweaking this setting has a real effect. My system load monitor is visible all the time in my i3bar, so I have a pretty good idea what it should be like under various conditions. My previous idlish 1min load index was constantly hovering around 0.9-1.1+, with TB spiking CPU every so often and raising load. This is a laptop core i5 hyperthreading cpu, so 4,00 == maxed out.

After tweaking this *idle_limit* setting, I’m now seeing my idle system 1min load dropping all the way down to 0.4+ range and *staying there* until I actually do something. This is an amazing improvement."

So while I'm not sure what can be concluded of this or where exactly (what bug #) this info belongs to, I figured I'd write it up somewhere, before I forget about it. Let me know if there's something other info or related activity you need on this.
Hello, thanks for your patience in helping debug this problem. I think this bug can be closed, as I've not experienced any noticeable problem for a while (I'm using 17.0.7), even after reverting mail.db.idle_limit to the default.
Flags: needinfo?(federicoleva)
Federico, thanks for updating the bug report.

conversely, http://ubuntuforums.org/showthread.php?t=2167927 claims performance whent bad after moving from fedora to ubuntu

like Leho's comment, a posting at http://askubuntu.com/questions/158171/why-is-thunderbird-pegging-a-core-at-100/335610#335610 talks about setting mail.db.idle_limit super high.
Status: UNCONFIRMED → RESOLVED
Closed: 11 years ago
Resolution: --- → WORKSFORME
(In reply to Leho Kraav (:macmaN @lkraav) from comment #50)
> So I was thinking maybe someone has written some addon or something to turn
> off TB's expensive operations based on AC/DC power status.

Not that I know of.  See the items I and others have been organizing in bug 446418 and bug 506975.   See https://bugzilla.mozilla.org/buglist.cgi?f1=OP&o3=anywordssubstr&list_id=7738788&v3=battery&resolution=---&classification=Client%20Software&classification=Components&o2=anywords&f4=CP&query_format=advanced&j1=OR&f3=status_whiteboard&f2=short_desc&v2=power%20battery&product=Core&product=Firefox&product=MailNews%20Core&product=Thunderbird&product=Toolkit for an overall picture.  Not all of those bugs are relevant - but the ones with "battery" in whiteboard most certainly are.  In general, there's a lot of organizing, research and fixing left to be done. Please help out by triaging and organizing them.

> After running with the 500min tweak for a day, this is what I wrote to the
> comments of that post:
> ...
> idlish 1min load index was constantly hovering around 0.9-1.1+, with TB
> spiking CPU every so often and raising load. This is a laptop core i5
> hyperthreading cpu, so 4,00 == maxed out.
> 
> After tweaking this *idle_limit* setting, I’m now seeing my idle system 1min
> load dropping all the way down to 0.4+ range and *staying there* until I
> actually do something. This is an amazing improvement."

Leho please file a bug report and give us the bug number
Flags: needinfo?(leho)
Hmmm, I'm not sure what I am supposed to say in the filing.  Seems to me like idle_limit works exactly as advertised i.e. feature not a bug (one time, yeah!).

OK, but really I have no idea what idle_limit is supposed to do other than the educated guess by looking at its name.
Flags: needinfo?(leho)
(In reply to Leho Kraav (:macmaN @lkraav) from comment #54)
> Hmmm, I'm not sure what I am supposed to say in the filing.  Seems to me
> like idle_limit works exactly as advertised i.e. feature not a bug (one
> time, yeah!).
> 
> OK, but really I have no idea what idle_limit is supposed to do other than
> the educated guess by looking at its name. 

see Bug 723248 which implemented mail.db.idle_limit and mail.db.max_open.  

We should expect that it would not have zero impact. But we should also expect that it would NOT have unwanted performance effects. (a gross example being Bug 793455)

So I hope you file a bug report. It might be caused by something that was not anticipated when idle_limit was implemented.  Also, for people with problems, it might be more appropriate to adjust mail.db.max_open before adjusting mail.db.idle_limit
I recently had this problem when I started to download a very large IMAP account. The problem was resolved by running CCleaner, with everything under "Thunderbird" checked except password.
I'm again facing serious slowness after the upgrade to 24.1.0 on fedora18. For now I've "repaired" most of my directories (it's quite a pain, clicking them one by one), I'll again check the other instructions and see if something improves the situation or not...
I am facing this issue as of late.  Have been running on latest TB available on Ubuntu 12.04 for 6+ months and no problem until about a month ago.  I have implemented the change in the idle_limit config setting.  I would appreciate any further information: TB 24.2.0; on Ubuntu 12.04: 3.5.0-45-generic #68~precise1-Ubuntu SMP Wed Dec 4 16:18:46 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

Thanks.
Flags: needinfo?
(In reply to Federico from comment #57)
> I'm again facing serious slowness after the upgrade to 24.1.0 on fedora18.
> For now I've "repaired" most of my directories (it's quite a pain, clicking
> them one by one), I'll again check the other instructions and see if
> something improves the situation or not...

Federico, is your slowness with imap folders?  Is it again helped by 
mail.db.idle_limit?  At what value of mail.db.idle_limit?
Flags: needinfo? → needinfo?(federicoleva)
No, I don't have any IMAP folder. I upgrade to fedora 20 and I now have Thunderbird 24.3.0, I don't think I can downgrade to 12 any longer as I did previously. I again have slowness problems and every now and then Thunderbird crashes too, but I didn't have time to investigate/debug/tune settings other than repairing some folders, which didn't help.
Flags: needinfo?(federicoleva)
(To clarify, my last comments means that I have no idea whether my problems are with this bug or a new issue.)
I have both IMAP (gmail via IMAP) and Exchange (with add-on) accounts setup in Thunderbird.  The start of the issue doesn't seem to coincide with adding or removing either (I first started using TB with Exchange Exquilla add-on and later added my gmail account).
I changed the idle_limit to 30000000 per an Ubuntu or Bugzilla post, forget which, pointing out the constant refresh with the initial value (missing some 0s :).  It has not helped after 5 calendar days of change.
I did NOT have fedora installed prior to installing Ubuntu (no upgrade issue).  Direct install of 12.04 LTS.
Flags: needinfo?
(In reply to Federico from comment #60)
> No, I don't have any IMAP folder. I upgrade to fedora 20 and I now have
> Thunderbird 24.3.0, I don't think I can downgrade to 12 any longer as I did
> previously. I again have slowness problems and every now and then
> Thunderbird crashes too, but I didn't have time to investigate/debug/tune
> settings other than repairing some folders, which didn't help.

If no imap, then mail.db.idle_limit would have zero effect, contrary to comment 43.  Did you never have imap?   

>  every now and then Thunderbird crashes too
Do you get crash reports?
See https://support.mozillamessaging.com/en-US/kb/Mozilla-Crash-Reporter#w_viewing-crash-reports
Flags: needinfo?
sorry. my comments about mail.db.idle_limit are dumb. It certainly can help the non-imap case.
izz, please file a support request using instructions at https://support.mozillamessaging.com/en-US/kb/ask#w_how-to-ask-your-question and then give us the URL to the support topic.
Flags: needinfo?(izz)
No: no IMAP, no crash reports, nothing ABRT caught. I have no idea if that's related... I also never understood what had fixed the problem for me earlier, might have been the config or the directories repair or who knows what.
Thanks:
http://gsfn.us/t/4ft55
Flags: needinfo?(izz)
(In reply to :aceman from comment #10)
> If you see in tools->Activity manager is there any operation in progress
> (like Indexing)?
> 
> Also go into Help->troubleshooting info and paste the exact "Application
> Build ID" here. Also see the Graphics section there if there isn't any GPU
> acceleration enabled.

Process-Explorer shows, for a moment, where ist does freeze, 500Mio Cycles per Second in for the thread thinderbird.exe with the following stacktrace.
So according to "http://msdn.microsoft.com/en-us/library/windows/hardware/ff553302%28v=vs.85%29.aspx" (Remark 3.) it seems to be waiting for a semaphor ("http://msdn.microsoft.com/en-us/library/windows/hardware/ff563928%28v=vs.85%29.aspx") during its Interrupthandling.
Here is the aforementioned stacktrace.:
______________________________________________________________________________
ntoskrnl.exe!KeSynchronizeExecution+0x2246
ntoskrnl.exe!KeRemoveQueueEx+0xf5e
ntoskrnl.exe!KeRemoveQueueEx+0x9f7
ntoskrnl.exe!KeWaitForSingleObject+0x248
ntoskrnl.exe!PsGetProcessWow64Process+0x9c
ntoskrnl.exe!KeRemoveQueueEx+0x28d9
ntoskrnl.exe!KeRemoveQueueEx+0x111e
ntoskrnl.exe!KeRemoveQueueEx+0x9f7
ntoskrnl.exe!KeWaitForMultipleObjects+0x40a
win32k.sys!W32pArgumentTable+0x15f4
win32k.sys!EngAcquireSemaphore+0xc431
win32k.sys!EngDeleteSemaphore+0x9403
ntoskrnl.exe!_setjmpex+0x34b3
wow64cpu.dll!TurboDispatchJumpAddressEnd+0x598
wow64cpu.dll!TurboDispatchJumpAddressEnd+0x55e
wow64.dll!Wow64LdrpInitialize+0x22e
wow64.dll!Wow64LdrpInitialize+0x172
ntdll.dll!LdrInitShimEngineDynamic+0x2383
ntdll.dll!memset+0x11ca4
ntdll.dll!LdrInitializeThunk+0xe
USER32.dll!WaitMessage+0xc
xul.dll!JSD_GetValueForObject+0x84be3
xul.dll!XRE_AddJarManifestLocation+0x3572
xul.dll!NS_UnregisterXPCOMExitRoutine+0x27ef4f
xul.dll!NS_UnregisterXPCOMExitRoutine+0x93136
xul.dll!NS_InvokeByIndex+0xe8f8
xul.dll!NS_InvokeByIndex+0xe9c7
xul.dll!JSD_GetValueForObject+0x847ea
xul.dll!NS_ShutdownXPCOM+0xdd0c
xul.dll!NS_UnregisterXPCOMExitRoutine+0x27dfb7
xul.dll!NS_NewNativeLocalFile+0xcfe6
xul.dll!XRE_InitCommandLine+0x1eeb
xul.dll!XRE_main+0x34
thunderbird.exe+0x13bb
thunderbird.exe+0x1556
ntdll.dll!RtlSetLastWin32Error+0x3e
ntdll.dll!RtlCreateUnicodeString+0x86
MSVCR100.dll!malloc+0x36
MSVCR100.dll!??2@YAPAXI@Z+0x10
thunderbird.exe+0x1d51
ntdll.dll!RtlInitializeExceptionChain+0x84
ntdll.dll!RtlInitializeExceptionChain+0x5a
______________________________________________________________________________
_Tschuess,
__Michael.
thunderbird hangs for 30 sec every 30 sec or so. I'm running Thunderbird 31.2.0 in Fedora 21 on a 20 core 3.5 Ghz Xeon e5 v2 with 32 G of ram. cpu is not pegged, thunderbord just hangs. This sucks, what a piece of trash.
Same with me - 30 sec every 30 sec or so, no extensions installed. [Gmail].sbd folder is 4.1G, imap.googlemail.com (including [Gmail].sbd) - 7.4G

I've reverted to TB 11.0.1 to continue use it w/o problems.
Have you guys tried TB38 which has another problem with the closing of folder databases fixed?
I'm having this problem with TB 41.0 (beta, 64-bit) on an Ubuntu 14.04.3 (64-bit) system with 32 GB of RAM.

I tried running TB in safe mode (all addons disabled), and the problem persists.

TB had become essentially unusable for me because of this bug.
I believe I was able to work around this bug by deleting ALL the *.msf files in my Thunderbird profile.  TB recreates the *.msf files the next time it launches.  TB is no longer freezing continually for me, as it had previously been doing.

This same solution appears to work for a TB instance (also 41.0 beta) which I am running on a Windows 7 Professional (64-bit) system.

One word of caution regarding this workaround.  If you have custom column display settings, these will be lost when the *.msf files are deleted.  Since I am accessing multiple accounts (but all with the same custom column display settings), I deleted the *.msf files for only one account at a time -- then, the next time I launched TB, I used the "Apply columns to folder and its children" function to re-copy my custom columns from another account to the account whose *.msf files I had just deleted.  By doing this separately for each account, I was able to keep my custom columns on all my accounts.

I still think this issue is a bug, because ordinary users may not have the technical expertise to perform the workaround without losing data.
I was able to work around by making a filter to delete non essential email after 5 days. I do this for several high volume mail lists to which I subscribe. The freezing is greatly reduced, although still occasionally occurs. Thunderbird, like many Java based apps, simply doesn't deliver scalable performance, and is not capable of handling large amounts of data. It's so frustrating to see developers respond to these reports with "it works for me", rather than address performance and scaling issues. Too bad.
Do you guys use IMAP? How many messages are there in the largest folders (by msg count) ?
(In reply to Rich Wales from comment #72)
> TB had become essentially unusable for me because of this bug.

I finally threw in the towel for TB and installed Outlook on a windows VM.  The local VMWare player uses less resources than what TB was doing for me a year ago.  SMH...
I have 33681 message in one of my inboxes, 16142 in another one. Until August 2015 I was on thunderbird v11 and it was handling these w/o problems
And yes, IMAP
Similar here: IMAP connections with the main one to GMail. Currently 34k messages in inbox; it's been as high as 125k before. 215k in the "all" view, and a few more ~100k folders.
Can you try toggling TB into offline mode (file->offline->work offline)? To see if the syncing with server is what takes the time.
This always takes a long time for me -- around 10 seconds to check each folder if there is nothing to be done, and (despite having TB open and internet-connected pretty much 100% of the time) it's ~guaranteed that it'll want to sync 10s of thousands of messages in some folders. It's the sort of thing that I do only rarely, and usually via the set-it-running-and-go-to-bed strategy...

Don't know if that is useful. The n*10k message mis-syncs suggest to me that syncing unviewed mailboxes is *not* usually running as a background process. Whereas the Activity Manager tells me that there is constantly some indexing going on (although without any increment on the progress bar).
(In reply to Andy Buckley from comment #81)
> Don't know if that is useful. The n*10k message mis-syncs suggest to me that
> syncing unviewed mailboxes is *not* usually running as a background process.
> Whereas the Activity Manager tells me that there is constantly some indexing
> going on (although without any increment on the progress bar).

Hey, we asked about indexing in the past and nobody confirmed it. Can you paste the whole message that you see in the Activity manager?
Sure... but it just says "Indexing messages", then an empty status bar and "Indexing 1 of 20 messages (0% complete)". It's said the same thing for the last 12 hours at least...
OK, then you may try turning off the indexing in Options->Advanced->General->Global search.
This is used only for the global search in TB (not the quick filter and also not the Advanced search in Edit). If you do not use that you can safely turn it off. It is only indexing the local contents of the messages for quick search, it does not have anything with syncing between server and local messages.
I can confirm, that the work around in comment 73 worked for me. After deleting all *.msf files tb is running smooth again.
Disable indexing didn't help.
I reduced the mail folder size from 3.8GB to 2.5GB by deleting mails and attachments, that doesn't help too.
I don't have any IMAP folders, only POP3.
The work around by deleting all *.msf files doesn't last too long. After about 2 weeks TB 38.3.0 on Ubuntu 14.04 is as slow as before.
This is a recent problem for me as well. I have never had problems with Thunderbird freezing before, but it's frozen on me twice this morning. Prior to today I've been noticing Thunderbird using a fair number of cpu, so I followed some of the ideas in this post... cutting the number of emails in my folders and turning off the indexer. As evidenced by the freezing, this must not have fixed some/all of Thunderbird's problems. I can delete the *.msf files, but according to comment 86, that may not be a permanent solution.
I come to realize that it depends on the TB runtime, my PC and TB are running 24/7.

I tried to collect data with Gecko profiler, but after installation of TB nightly and Gecko add on, TB runs fluently. A few days and it becomes slower and slower again. Restarting TB without further action makes it responding well again.
I'm seeing this slowness as described on recent fedora's (currently on f22 with latest updates);
running TB 38.3.0 right now.

Gecko profiler shows 80% of the time is spent in __poll_nocancel() and strace confirms
similar.

There are also periods where TB decides to open a msf and spends a lot of time reading from
it (some msf's are rather huge on large folders).

I'm unsure why TB would need to keep polling .msf's while running; surely it should only need
to open each once at startup and if anything gets added there should be in-core flags set
to tell various bits of TB about that; it shouldn't need to keep re-reading them.

I've tried the mail.db.idle_limit tweak from 300k to 3M briefly; it didn't seem to help any.

I've got about 400 .msf's all up, 380 folders, 13GB of email 
(16 years of mail with many per-month folders), and most of the top-level folders (88 of) have 
"when getting new messages, always check this folder" enabled as I use server-side filtering/filing
(cyrus IMAP + sieve)

This slowness is really bad.  Even when composing messages I often get 30s delay before
I see what I've typed; I can type a whole sentence before I see it.

Scroll bars freeze for long periods.   

100% CPU when this is all happening.  I have a Q8400 which is quad-core; not the latest
and greatest but it _should_ cope with this.

I use a NFS server for my mail which may exacerbate the slowness but the server is pretty
idle as far as iostat/top on it says.
If you have that many folders and do not mind some increased memory usage, try increasing mail.db.max_open pref. It contains the max. number of .msfs that TB tries to keep open. If you have more than this number of active folders, it may happen that TB closes them after a while and then reopens by the costly read operation you notice.
Sounds like a good idea; st I've upped it from 30 to 500 and restarted TB.

However lsof doesn't show  any more than about 25 .msf's open at any one time, and that
number goes up and down continuously, so its not keeping any open for any length of time.

strace shows TB still opening .msf's all the time, about one a second.

But why is TB polling them?  Does TB expect that something other than TB would be playing with them?
It's kind of sureal to think that we may still have a problem in this area.  I'm glad aceman is here! :)
Perhaps we'll need to be filing a new bug report.

(In reply to Ian Donaldson from comment #89)
>...
> I've tried the mail.db.idle_limit tweak from 300k to 3M briefly; it didn't
> seem to help any.

aceman, is there any danger to picking a large value?
(In reply to Ian Donaldson from comment #91)
> Sounds like a good idea; st I've upped it from 30 to 500 and restarted TB.
> 
> However lsof doesn't show  any more than about 25 .msf's open at any one
> time, and that
> number goes up and down continuously, so its not keeping any open for any
> length of time.

Ian, 
based on the number of a) virtual folders that iterate over real folders, b) filters that might send email to different folders, and c) folders that explicitly click on ... how many folders do you expect should be opened after several hours or a day?
Flags: needinfo?(iand)
I have no TB based mail filters set up; all my filters are on the server side and deliver
to the top level IMAP folders; TB only fetches from IMAP; nothing else
that I know of should be touching the .msf's other than TB

88 or so top level folders; probably only about 20 of those get regular traffic during
the day.

TB is however opening lots of very old folders regularly; those that have had no traffic in years.
Flags: needinfo?(iand)
For imap folders receiving server side filtered mail, are they set in thunderbird to "check this folder when getting new messages for this account", or have you set any hidden pref of xxxxx.check_all_folders_for_new to "true"?
Flags: needinfo?(iand)
I have "check this folder when getting new messages for this account" only on the folders
that are set up for server side filtering; about 80 of those.

mail.server.default.check_all_folders_for_new is at its default, false value

There are no other check_all_folders settings in my config.
Flags: needinfo?(iand)
(In reply to Ian Donaldson from comment #96)
> I have "check this folder when getting new messages for this account" only
> on the folders
> that are set up for server side filtering; about 80 of those.

(oh my, I missed that before. holy crap)

> I use a NFS server for my mail which may exacerbate the slowness but the server is pretty
idle as far as iostat/top on it says.

yes NFS involved will affect speed - write IO from Thunderbird is unbuffered and therefore slow. Being addressing in a different bug report.
(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #92)
> (In reply to Ian Donaldson from comment #89)
> >...
> > I've tried the mail.db.idle_limit tweak from 300k to 3M briefly; it didn't
> > seem to help any.
> 
> aceman, is there any danger to picking a large value?

I think large numbers should work, unless we hit some overflow somewhere. I can check that.

In this specific case, I suggest setting to 3M permanently and ALSO increase mail.db.max_open to the number of folders.
The current algorithm is so that we first close all idle DBs (determined by mail.db.idle_limit) and only then see if we need to close some more (even non-idle ones) by consulting mail.db.max_open .
Been running with idle_limit=3M and max_open=500 for a coupla days; doesn't seem to help any.
I'm still only seeing 16 or so .msf's open right now.
On my windows system the freezing is occurring while the status bar shows something abt downloading messages. i forgot the exact message, but that was the gist. I'm using imap of course. On linux system I'm not seeing that same message in the status bar during freezing. 

The obvious thing to do(at least in the eyes of a c++ developer) is profile the application and see if any red flags show up. Unfortunately I don't know the Java ecosystem. Can anyone suggest a profiler to use? I'd be glad to upload it's output.
Any ideas on the very high __poll_nocancel() usage?

I attach an image of gecko profiler output.
Attached image gecko.png — 

    
    

    
  
(In reply to Ian Donaldson from comment #99)
> Been running with idle_limit=3M and max_open=500 for a coupla days; doesn't
> seem to help any.
> I'm still only seeing 16 or so .msf's open right now.

Set the preference mailnews.database.dbcache.logging.console to a value of "Debug" . Then restart TB and observe tools->error console. There should be info about cached databases (folders) and if any are discarded from memory.

    
    

    
  
(In reply to Alex Lindsay from comment #87)
> This is a recent problem for me as well. I have never had problems with
> Thunderbird freezing before, but it's frozen on me twice this morning. 

For anyone** who has done all the updates for the past year and is on windows, the first thing to suspect is antivirus software interference. If you prove to yourself that AV is not the problem (by running Windows in safe mode) then it will be helpful to do what is suggested in comment 103.

** eg Armin, burlen, Andy, ...

> Prior
> to today I've been noticing Thunderbird using a fair number of cpu, so I
> followed some of the ideas in this post... cutting the number of emails in
> my folders and turning off the indexer. As evidenced by the freezing, this
> must not have fixed some/all of Thunderbird's problems. I can delete the
> *.msf files, but according to comment 86, that may not be a permanent
> solution.

no it is no way a solution

    
    

    
  
I've set  mailnews.database.dbcache.logging.console to a value of "Debug"... it shows
pretty much what lsof is showing... not many DB's are staying open.

sequence is this... 

17:17:26 open db count 3
17:18:26 periodic check of cached dbs, count=16
17:18:26 skipping cachedDB not open in folder: xxxx
17:18:26 folder open in window, name Inbox
17:18:26 skipping cachedDB not open in folder: Trash
17:18:26 (another 7 lines like that; different folders)
17:18:26 open db count 7

17:19:26 periodic check of cached dbs, count=15
17:19:26 (bunch of similar stuff to prior minute)

This seems to repeat every minute.

    
    

    
  
Yes, but have you seen any occasion when the number of DBs went down?

I also enabled the logging and could see very few DBs cached. E.g. if folders were only opened and msgs displayed in TB for a moment, the relevant DB did not get cached. I don't know yet, if that is bad and if caching it would improve perf.

However, as DBs got added into the cache eventually, the number is steadily increasing because none are purged with the high pref values. So the cache purging seems to work correctly here.

    
    

    
  
Sorry for the delay... the DB's open went up and down over time; never many open at a time but.

I've given up debugging this; yesterday I created a new profile, re-set up everything I used previously
and now it is a lot faster and usable again.  (took over a day to re-download all my email and re-index)

Clearly something was messed up in the old profile (which I still have handy for a while) but no idea
what.

I diffed the pref.js's of the two profiles; lotsa differences there but the only
ones that stood out are:

< user_pref("mail.server.server1.ageLimit", 1);
< user_pref("mail.server.server1.daysToKeepBodies", 1);
< user_pref("mail.server.server1.daysToKeepHdrs", 1);
---
> user_pref("mail.server.server1.ageLimit", 30);
> user_pref("mail.server.server1.daysToKeepBodies", 30);
> user_pref("mail.server.server1.daysToKeepHdrs", 30);

http://kb.mozillazine.org/Mail_and_news_settings  doesn't document what they mean though.

One thing I did notice was the downloaded ImapMail was a lot smaller than the old one, so I 
compared the message counts in each file; there were a bunch of folders mostly created in the last
year or so that had close to double the number of messages in them in the old profile.
I ran 'Compact Folders' again but it didn't clear that up; I had to run 'Compact Folder' manually
on each one that was an issue to shrink them, so clearly the Compact Folders was stopping before processing all of them for some reason.   Anyway that's probably a different issue.

Pity the compact ops don't show in the activity manager; its a bit hard to see what its up to.

    
    

    
  
Since the profile reset thunderbird has been quite responsive and usable; unlike
before where I would write half an email before it would even echo what I had typed.

I've noticed though the CPU usage is fairly constant at 106% now, whereas prior it was pegged
at 100%.   Using the H option in top, one thread is sitting at 97% and another at about 7%.
(Quad-core processor)

Presumably more threads are working than previously.   A ps shows about 50 threads are running.

An strace shows one of the threads is still running around all my folders opening all the msf's
for some reason.  Folders that are archive folders that haven't had recent email in years; ...
why is it bothering with that?   Surely it shouldn't be interested in those if I don't have
the check for new mail flag enabled, and I don't have client side filtering enabled.

    
    

    
  
It may be indexing all the messages if you have the Global search enabled. See tools-> Activity manager.

    
    

    
  
Indeed I have it enabled; its great.   However I would have thought that it shouldn't
bother opening .msf's that haven't changed.

    
    

    
  
If you reset your profile the indexer database was removed so he must rebuild it anew. The database must contain all messages regardless of their date. Once that is done, only the new messages will be added as they come in.

    
    

    
  
Yep I watched global- from empty and eventually got to the size the old profile had, give or take a bit.
Well the new one was a bunch smaller, presumably related to messages that were deleted over time
and the space not reclaimed in the old one.

 
$ ls -l */global* |cut -c24-200
 732299264 Jan 15 17:29 t4grsgse.Default User/global-messages-db.sqlite
 945291264 Jan 12 12:39 xyfi225g.default/global-messages-db.sqlite

    
    

    
  
(In reply to Federico from comment #66)
> No: no IMAP, no crash reports, nothing ABRT caught. I have no idea if that's
> related... I also never understood what had fixed the problem for me
> earlier, might have been the config or the directories repair or who knows
> what.

Perhaps Federico's initial issue was fixed by bug 793455  - DB cache is closing databases too frequently - in in version 16/17.  

There was still another yet undiscovered Bug 1135310 - Algorithm from bug 72324 for closing idle folder databases is broken. Too many/all databases can be closed.  This was eventually fixed in version 38. 

bug 1135310 comment 28 notes there may still be a problem, for which mail.db.idle_limit = 30000000 is a workaround.  And comments in this bug from the past year also cite problems.


To properly address open issue it's time to end comments here in Federico's bug and open one or more new bug reports.  Please set mail.db.idle_limit = 30000000 (7 zeros):
1. If problem is solved then please file a new bug report with "folder databases closed too frequently" in the bug summary, including when you first noticed the problem.
2. If problem is not solved, then please file a new bug report with the details, including when you first noticed the problem.

Filing a new bug is a bit of a pain if you've already commented here, but it's the best way to get the right attention on your issue. And we want to fix them. Thanks.

          You need to log in
          before you can comment on or make changes to this bug.
        






  

    
  







  

    

      
Attachment

      

      
      
      
      
    

    

      

        

          

            
General

            

              Creator: 



            

            
Created: 

            
Updated: 

            
Size: