1135310 - Algorithm from bug 723248 for closing idle folder databases is broken. Too many/all databases can be closed.

Status: RESOLVED FIXED

(MailNews Core :: Backend, defect)

Description

[:aceman]

I think the algorithm for closing inactive folder databases (from bug 723248) does not really work correctly and in some cases may cause closing of all databases every minute, which could have some perf impact.

I'll attach a patch showing the reasons for this suspicion.

Comment 1

[:aceman]

Attachment: 1135310.patch

I attach a patch with comments showing the problems. They are not fixed in the patch, but there is debugging output to show the algorithm is bad (it is seen it tries to close all dbs and does not leave maxOpenDBs open). I think I can fix all of the problems, if we can decide on how to fix problem 5.

To test this, set mail.db.max_open to a low number and then click more than that folders in the folder pane.

Comment 2

[:aceman]

I have noticed the databases are really removed from the cache (and DB destructor is called) when the focus changes to another window, be it even Preferences, Account manager or Compose. It always leaves only 4 DBs in the cache (not the value of mail.db.max_open). And no, it is not late purging of the DBs that were marked for closing. It happens also if the cache manager didn't try to close any DBs. That is a phenomenon I absolutely do not understand. You can see all this by running with the patch and watching the console.
Anyway, that event may never happen in a TB session, the user may have no need to open those windows ever.

So my Problem 5 still holds, we need a way to see if we already set folder.msgDatabase = null in a previous run so that the counting how many more to close is right.

Comment 3

[Wayne Mery (:wsmwk)]

Interesting analysis aceman! Matt has perhaps seen symptoms of this and is more clued in to user reports, so I'm copying him. 

Obviously the focus of this bug is to fix the issue. But I'd be curious what you all think of the time line below, and your sense of the potential conclusions listed.

I'm not sure I've got all the time line perfect here but 
- broken bug 723248 patch landed 2012-04-26 for TB15 trunk, and TB15 shipped ~25-Aug-2012 
- excessively low mail.db.idle_limit setting from bug 793455 uplifted/fixed in version 16 and 17.
- problem first reported afaict) by me 2013-10-21/bug 929281 - because of user reports about version 24, not from my own experence. Matt, I don't recollect this being reported by users prior to version 24.

Possible conclusions?
- Did bug 939462 (which landed at the tail end of TB24 development) really help bring this to light, as I speculate in bug 929281?
- Did problem really go undetected in dailies, earlybird and beta from version 18 until 24?
-- Is this yet another example where our non-release tester population so poorly represents either in hardware or mail configuration, our average or our extreme users, or both?  Or, are they so toleratant of ratty performance that they don't report it?

Comment 4

[:aceman]

I think it is because problems like bug 929281 need specific conditions and are not seen in normal usage and conditions, even testing. If the bug here is true, then you need to meet this specific condition: having a high number of folders (30+) opened by TB simultaneously. E.g. by having so many accounts, that you at startup download into more than 30 folders simultaneously. Also the folders must be so big (number of msgs, size of msf) that closing and opening them is computationally non-trivial.

Even if you hit the problem described in this bug in normal usage, you do not notice it. Even if all folders are forced closed (except the currently active one), what does the user see? Nothing. The current folder works fine. If he clicks another one, yes, it is reopened. But that takes split of a second on normal folder. I can only feel some slowness when revisiting a 50000 msgs folder. Is that common size? Maybe for the users that reported the problem. If they have 50 of such big folders and those are accessed by TB in parallel (downloading mail on startup), then yes, just opening them may take 1 minute. But as that happens at startup, any purging of the cache hasn't happened yet. Opening so many folders will take 1 minute regardless of what we do here. The only current workaround is to not automatically check msgs into 50 inboxes IF they are that big.

That is why I filed this bug separately. It may help bug 929281 and others partly, or not really.
E.g. if there are reports that while using TB it is slow. That may happen if all folders are closed periodically due to this bug, but need to be reopened often due to computations of the count of messages. Like bug 939462.

So what you can recommend to users to test is to increase those 2 prefs to really large values that the cache purging never kicks in.

Comment 5

[:aceman]

Attachment: WIP fix

So this patch shows how it would work if the worst problem are fixed.
If you observe the output, the list of folders is stable and no folders are forcefully purged if they are below the limit.

I have observed that folder that are empty, or that get no use (just clicked in folder pane, but no change of data) are "disconnected" from DB automatically (probably more precise that they never open their db so .msgDatabase is null).

Another observation. DBs for folders that have their .msgDatabase at null (either set manually in this cache manager) or automatically, ARE destroyed and removed from the cache at some later time.

The observation that opening Prefs or AM destroys the DBs still holds, however it only destroys those DBs that had .msgDatabase at null. The previous observation that only 4 DBs remain isn't true with this patch. So that seems like correct behaviour to me.

Comment 6

[[e-mail]]

If setting the msgDatabase to null doesn't remove the database from the cache then something else must be keeping a reference to the database. However this does make the database cheap to reopen, so that shouldn't be a problem, except that we keep trying to close databases and they don't go away when we expect. I'm not sure how we would find out what is keeping the database open though.

Comment 7

[WADA:World Anti-bad-Duping Agency]

> Bug summary : closing idle folder databases potentially broken

FYI.
Bug 905576(and  bug  624806) is for actual "outdated msf comdition" which is perhaps relevant to "closing idle folder database", because problem started to occur from Tb 3 in which "closing idle folder database" was introduced.
Bug 905576(and  bug  624806) is report for .msf file on file share., but I don't think "file share" is essentially irrelevant to problem. I think "file share" merely expands timing hole by slowness in file access than local HDD.

Comment 8

[:aceman]

(In reply to [e-mail] from comment #6)
> If setting the msgDatabase to null doesn't remove the database from the
> cache then something else must be keeping a reference to the database.
> However this does make the database cheap to reopen, so that shouldn't be a
> problem, except that we keep trying to close databases and they don't go
> away when we expect. I'm not sure how we would find out what is keeping the
> database open though.
Yes, but if we set msgDatabase to null then the DBs can be destroyed later on. So wrongly settings databases to null as the current code does is still a problem. Once removed from the cache, loading a DB seems expensive.

Maybe the fact that opening some new window makes the databases go destroyed is some clue. That causes the links (references) to the DBs to be released. Maybe the folderpane is holding onto the DBs while it has focus in an active window? Maybe bug 455972 comment 10?
Can you see how Seamonkey behaves? I think it has a different implementation of the folder pane.

Comment 9

[Wayne Mery (:wsmwk)]

Thanks for comment 4. Makes sense.  But we do need to be careful because:
- although the majority of our users might not have lots of accounts such that that they might need ~30 folders simultaneously opened, I personally am certain the "other population" that can be affected is a non-trivial percentage**
- even the average user consider their use of virtual folders, and unified folders, which will need more folders open
- the average folder may be quick to open on a mythical average computer, but it won't always be quite so fast on the average laptop which has slower disk speed, and especially on windows where all kinds of other crap software is running

And relevant in context of this bug but perhaps a seperate bug needed, we should expose high water mark of # open folders (we already have it, no?) in about:support aka Troubleshooting. And when mail.db.max_open and mail.db.idle_limit are not at default values, we should also expose those values in Troubleshooting.  

** You asked in comment 4 "Is that [50000 msgs folder] common size?". We currently have no way to know.  So we should discuss, are there other useful metrics worth considering for Troubleshooting and/or telemetry, and ultimately in our test methods?

Comment 10

[Wayne Mery (:wsmwk)]

perhaps not related, but want to track Bug 1087403 - TB 24.6.0 intermittantly forgets imap synchronization flag. Sent folder sometimes "not special"

Comment 11

[Wayne Mery (:wsmwk)]

Typically we do not place high priority on very old problems, like this one being 2+ years old. 

But in this case I agree with aceman below that we should get something into TB38 - the real problem is when we have structural defects (old or not old) that cause performance issues, it makes it very much harder to find and diagnose new issues/regressions when we roll out major new releases - remember we have no automated performance tests to help us avoid new undetected issues in our development branches accumulating since TB31 branched to TB32

aceman  wsmwk: we do not strictly need a complete solution for all problems
aceman  wsmwk: rkent: I think even landing my patch as is would be a partial win.

Comment 12

[WADA:World Anti-bad-Duping Agency]

Major objective of "closing inactive folder databases" which was introduced from Tb 3 :
(1) Avoid "outdated msf" due to "no close of msgDatabase until shutdown".
(2) Avoid file handle shortage.

"outdated msf" is local mail folder only problem, so, different algorithm can be applied to local folder and imap folder, newsgroup, rss feeds, ....
If "no outdated msf condition" can be guaranteed by other than msgDatabase close, "required frequency of msgDB close of local mail folder" can be reduced.

Penalty in imap folder open is :
  (a) "entire msf file reading and re-constructing msgDB in memory" + (b) "full resync of folder by uid fetch 1:* Flags".
  Even when msgDB is kept open, because  max cached connection is limited(default=5),
  if connection is stolen by other folder, "SELECT + uid fetch 1:* Flags" is needed until QRESYNC+proper CONDSTORE support will be available.
Because auto-sync is used in many imap environments, this "re-sync due to steeling connection by other folder" frequently occurs.
Folder opened by auto-sync should be closed immediately, and folder needed for new mail check should be kept open.
So, different algorithm is needed even in imap folders.

If virtual folder, "open of Virtual folder" opens all search target folders of the Virtual Folder. Because re-aquision of all messageKey is executed for all search target folder is executed, for example, SEARCH UNDELETED ... if imap and  online search is requested with Body condition, and because "Virtul Folder"=="Serch Folder"==Result of an executed search, search target folders of Virtual folder can be closed immediaately after Virtual Folder open.
So, different algorithm can be applied to Virtual Folder and its associated real search target folders.

Most recently used, Most frequently used, Folder attributes, Folder type, etc. is better counted upon choosing DB to close.
   Previously selected folder at folder pane is better kept open for long time.
   Search Target Folder of Virtual Folder, folder opened by auto-sync, is better closed as quickly as possible.
A bad in current design/implementation : No easy way/API to force msgDB close, force de-select at connection.
Problem in imap : usable method to de-select  == "SELECT other Mbox" only

Comment 13

[:aceman]

(In reply to Wayne Mery (:wsmwk) from comment #11)
> Typically we do not place high priority on very old problems, like this one
> being 2+ years old. 
> 
> But in this case I agree with aceman below that we should get something into
> TB38 - the real problem is when we have structural defects (old or not old)
> that cause performance issues, it makes it very much harder to find and
> diagnose new issues/regressions when we roll out major new releases -
> remember we have no automated performance tests to help us avoid new
> undetected issues in our development branches accumulating since TB31
> branched to TB32
The problem is we have not yet any confirmation from users that if they avoid this bug (by setting max_open very high) it wins them any perf. Can you get any proofs?

I only claim the algorithm seems broken (because the db closing loop does not stop when it should, due to some logic errors), by reading the code. I can't claim it wins any visible performance, because I have not the right test environment (on small folders, where the performance win must theoretically be there, but are not visible). I just claim it COULD win big if a user has 30 active folders that take 60 seconds to open and are immediately closed after those 60 seconds again, due to this bug.

> aceman  wsmwk: we do not strictly need a complete solution for all problems
> aceman  wsmwk: rkent: I think even landing my patch as is would be a partial
> win.

Yes, fixing the algorithm should be a partial win. It should stop improperly marking DBs for closing (via .msgdatabase = null). That the DBs are not closed immediately, but only later at some event is a second problem. It is not necessary to fix it together with the marking algorithm. It may also be very hard to find the culprit that is holding onto the DBs so they are not closed immediately. So that part can be spun off to a new bug. That is Neil's comment 6.

My theory is this:
1. WITH this bug, we have e.g. 60 open DBs and we mark all for closing. When the event comes, all of them are really closed and memory released.
2. WITHOUT this bug (with my patch), we have e.g. 60 open DBs and we correctly mark only 30 for closing (30 should stay open). When the event comes, the 30 of them are really closed and memory released.

So BEFORE the event comes, the memory consumption is the same in both cases.
AFTER the event, in case 1. memory is lower than in case 2. But also perf may be lower (even pathologically) because all DBs are closed and some need reopen. In case 2, memory usage is higher, but that is actually the intended state, because the whole feature is about keeping max. 30 DBs open. If keeping 30 DBs open is too much memory-wise, then the limit (max_open) should be revised, but is a separate bug. But in case 2 the feature would work as intended.

I have identified that mythical "event" to be e.g. focusing/opening some other window. Maybe even the "slow script" dialog is enough of a trigger? There may of course be other triggers not yet found, but they would only make things worse (while the bug is there).

So I say landing this partial fix should NOT regress us perf-wise and NOT memory-wise. Yes, TB may take more memory at some moments, but only at the levels we always intended. This bug caused it artificially to be below those levels.

Can anybody find a problem in my logic?

And can anybody actually look at the patches? :)

Comment 14

[Wayne Mery (:wsmwk)]

Your logic has merit, but I've held off because 
a) there is no question in my mind  this has a perf hit for a certain class of user, but I believe the population is larger than what you are thinking...
b) I'm not convinced that just increase the limit is a sufficient for all users to test 
c) I'd rather have a user test a fix that we know will fix X, Y, Z and that we can tell the user to look for fixing the behavior we expect to be fixed.
d) getting user feedback is painful because it is time consuming, typically lacks hard data, and is highly dependen on user enviroment and lack of user technical expertise

If we really need hard data, we either must replicate a user environment, or find a highly educated user (good luck to us).

Comment 15

[:aceman]

(In reply to Wayne Mery (:wsmwk) from comment #14)
> Your logic has merit, but I've held off because 
> a) there is no question in my mind  this has a perf hit for a certain class
> of user, but I believe the population is larger than what you are thinking...
I am not sure what this relates to.

> b) I'm not convinced that just increase the limit is a sufficient for all
> users to test 
I am convinced increasing the pref to above the total number of folders the user has will remove this particular bug for him (as the buggy count-based cache purging never kicks in, only the age-based will, which can again be removed using the second pref).

I think this bug is a bug even standalone and should be fixed in any case. I just wanted to try if it also helps some of the particular bug reports you have (slow start up).

> c) I'd rather have a user test a fix that we know will fix X, Y, Z and that
> we can tell the user to look for fixing the behavior we expect to be fixed.
Sure, go ahead. The "WIP fix" should be usable and should solve the problem I analyse. If you get anybody to test it I will be happy.

> d) getting user feedback is painful because it is time consuming, typically
> lacks hard data, and is highly dependen on user enviroment and lack of user
> technical expertise
> 
> If we really need hard data, we either must replicate a user environment, or
> find a highly educated user (good luck to us).

I think increasing the pref should be doable for a large number of users.
That is why I do not request you to get users to apply the patch, just increase the pref which should have the same effect.

Comment 16

[Patrick Cloke [:clokep]]

Wayne asked me to push this to try:
https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=ebf84511166e

Comment 17

[Kent James (:rkent)]

Attachment: Use log4moz

Let's make logging permanent. I'd like to land this ASAP so that we can get in the interface change.

Comment 18

[:aceman]

Comment on attachment 8577441 [details] [diff] [review]
Use log4moz

Review of attachment 8577441 [details] [diff] [review]:
-----------------------------------------------------------------

So this is basically my patch with dump() changed to log.info() or log.debug(). I can't really complain. You have removed the debug output from the destructor of nsMsgDatabase. But that is fine, it is the other part of the problem (why is DB not really destroyed when folder.msgDatabase is set to null).

Please just make the output texts more consistent and readable as I comment below.

::: mailnews/base/public/nsIMsgFolder.idl
@@ +29,5 @@
>  
>  // enumerated type for determining if a message has been replied to, forwarded, etc.
>  typedef long nsMsgDispositionState;
>  
> +[scriptable, uuid(E6CC0770-D0D8-4E67-8A2E-02C82C32C99A)]

Yeah, I forgot this :)

::: mailnews/base/util/msgDBCacheManager.js
@@ +116,2 @@
>      let cachedDBs = gDbService.openDBs;
> +    log.info("periodic check of cached dbs count " + cachedDBs.length);

Something is missing here so the sentence is strange. Maybe add a comma: "periodic check of cached dbs, count "

@@ +118,5 @@
>      let numOpenDBs = 0;
>      for (let i = 0; i < cachedDBs.length; i++) {
>        let db = cachedDBs.queryElementAt(i, Ci.nsIMsgDatabase);
> +      if (!db.folder.databaseOpen) {
> +        log.debug("skipping cachedDB not open in folder " + db.folder.name);

"folder: " (otherwise the name flows into the sentence)

@@ +129,5 @@
>          continue;
>        }
> +      if (db.lastUseTime < closeThreshold)
> +      {
> +        log.debug("closing expired msgDatabase for " + db.folder.name);

"closing expired msgDatabase for folder: "

@@ +138,2 @@
>      }
>      let openDBs = gDbService.openDBs;

This variable should be "cachedDBs", otherwise it conflicts with numOpenDBs which contains the number of really open dbs (with db.folder.databaseOpen == true).

@@ +152,3 @@
>        dbs.sort(function(a, b) a.lastUseTime > b.lastUseTime);
> +      let dbsToClose = dbs.length - maxOpenDBs;
> +      if (dbsToClose > 0)

So if dbsToClose can be <= 0 then we must exit here and not enter the loop below as it will close at least one DB.

@@ +152,4 @@
>        dbs.sort(function(a, b) a.lastUseTime > b.lastUseTime);
> +      let dbsToClose = dbs.length - maxOpenDBs;
> +      if (dbsToClose > 0)
> +        log.info("Trying to close " + dbsToClose + " databases");

Lowercase "Trying".

@@ +157,3 @@
>          if (MailServices.mailSession.IsFolderOpenInWindow(db.folder))
> +        {
> +          log.debug("not closing db open in window, name: " + db.folder.name);

"not closing db open in window, folder: "

@@ +160,3 @@
>            continue;
> +        }
> +        log.debug("closing db for " + db.folder.name);

"closing db for folder: "

::: mailnews/mailnews.js
@@ +810,5 @@
>  pref("mailnews.database.summary.dontPreserveOnMove",
>    "account msgOffset threadParent msgThreadId statusOfset flags size numLines ProtoThreadFlags label storeToken");
> +// Should we output dbcache warnings and errors to the "error console"?
> +pref("mailnews.database.dbcache.logging.dump", "None");
> +// Should we output dbcache warnings and errors via dump?

Please add allowed values for there prefs, or a link to log4moz docs where it is explained, as it is not obvious here these prefs are used for log4moz logging.

Comment 19

[Kent James (:rkent)]

Attachment: incorporating Aceman comments

This is Aceman's patch, with my review and suggested changes. r+=me

Comment 20

[Kent James (:rkent)]

https://hg.mozilla.org/comm-central/rev/ec7886c51421

I'll also push this to TB38 after a nightly cycle.

Comment 21

[Kent James (:rkent)]

Comment on attachment 8577480 [details] [diff] [review]
incorporating Aceman comments

[Approval Request Comment]
Regression caused by (bug #): 
User impact if declined: 
Testing completed (on c-c, etc.): 
Risk to taking this patch (and alternatives if risky):

Comment 22

[APraxis]

Re: comment that no one reported issue between v17 and v24...
I reported the issue very soon after upgrading to v24 FROM v17. I was upgrading manually and chose to stay with v17 until one day I thought that maybe I should upgrade to the latest version.
Does that mean that I was the only one seeing the issue? I'm not sure.
I did provide aceman with a copy of my TB v28 profile and DB in hopes that it would help but I'm not sure if it did.
In any event, nothing seemed to be progressing so I reverted to v17 and have merely watched from the side lines.
If there's a new version (v39) with a patch that needs testing then I'll be happy to download it and test. I'll need a specific URL to get the version. My basic TB profile and DB of mail hasn't changed significantly from Nov. 2013 when I posted bug 939462 so I should be able to prove or disprove whether the patch fixes the issue.

Comment 23

[Wayne Mery (:wsmwk)]

(In reply to APraxis from comment #22)
> Re: comment that no one reported issue between v17 and v24...
> I reported the issue very soon after upgrading to v24 FROM v17. I was
> upgrading manually and chose to stay with v17 until one day I thought that
> maybe I should upgrade to the latest version.
> Does that mean that I was the only one seeing the issue? 

definitely not. 

> I'm not sure.
> I did provide aceman with a copy of my TB v28 profile and DB in hopes that
> it would help but I'm not sure if it did.
> In any event, nothing seemed to be progressing so I reverted to v17 and have
> merely watched from the side lines.

we could have used your feedback to question in bug 939462 comment 66. but instead ...

> If there's a new version (v39) with a patch that needs testing then I'll be
> happy to download it and test. I'll need a specific URL to get the version.
> My basic TB profile and DB of mail hasn't changed significantly from Nov.
> 2013 when I posted bug 939462 so I should be able to prove or disprove
> whether the patch fixes the issue.

Please test the build at https://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-comm-central/

Comment 24

[APraxis]

(In reply to Kent James (:rkent) from comment #20)
> https://hg.mozilla.org/comm-central/rev/ec7886c51421
> 
> I'll also push this to TB38 after a nightly cycle.

I can confirm that it's fixed, at least on my system.
I just downloaded v 39.0a1.en-US.win32.installer.exe  	2015-03-14 11:52
and installed it with the Standard setup type.
There is now only a very very very small delay, about less than a second but noticeable, but definitely not an issue.

I'd also done a regular update to TB which gave me v 31.5.0 which is a log way behind v 39 daily and obviously still had the issue. My options are to revert to V 17 or continue with the Daily versions. What dangers are there to using Daily that would possibly damage my real TB DB of e-mails?

Is it likely that v 39 will replace v31.5 in the near future? I ask because I would think that any v 32 to 38 would have replaced v 31 it if the newer version were safe to release to the public. However there may be other issues in v 39 that mean that it won't be released and we'll have to wait for v 40 or higher.

I am glad to see that you folks were able to track the issue down and get a fix  :)  Thanks,
J

Comment 25

[Kent James (:rkent)]

The patch that was added in this bug will move to the aurora (TB 38) branch in a few days, assuming there are no major issues discovered. Then in about two weeks, there should be a beta of TB 38 that will also have this patch.

So assuming nothing changes, this code will be in aurora in about a week, in beta in about 2 weeks, and in release (esr38) in about 9 weeks. There is more risk to newer code of course, but also more fixes.

Thanks for the report, BTW, that it helped you.

Comment 26

[Kent James (:rkent)]

Comment on attachment 8577480 [details] [diff] [review]
incorporating Aceman comments

https://hg.mozilla.org/releases/comm-aurora/rev/cbbdc97adf02

Comment 27

[Wayne Mery (:wsmwk)]

APraxis, 
is your issue totally gone?

Comment 28

[Wayne Mery (:wsmwk)]

In http://rainbow.chard.org/2013/02/19/thunderbird-high-cpu/ - a blog about changing mail.db.idle_limit as a means to helping performance issues - many users report being helped by raising the value of mail.db.idle_limit  (Leho Kraav was one of them)

However, starting at http://rainbow.chard.org/2013/02/19/thunderbird-high-cpu/#comment-37299 several using version 38 still report that raising the value greatly helps their performance, which argues against this bug helping them.  It's unclear whether this bug helps prior posters in the blog - I haven't yet asked them.

Aceman, are you aware of any other bugs currently open for this area?

Comment 29

[APraxis]

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #27)
> APraxis, 
> is your issue totally gone?

No Wayne, not completely. There's still a small 'dead space' of about 10 seconds, down from 19-20 seconds before.

1) open TB
2) click on [Work online]
3) basic skeleton window opens immediately
4) basic window goes blank with the spinning wheel for about 10 seconds
5) password prompt appears, which responds quickly to entry & click [OK]

Comment 30

[APraxis]

sorry, I should have said that after updating to v 38.5 from 17 and then to 38.6 from 38.5
apparently 38.6 is the latest released version.
should I download the v 39.x to test that? if so, URL please

Comment 31

[APraxis]

Sorry, it just is too much trouble so I've reverted to V 17 once more.

I don't like the loss of the number of e-mails read and unread and I don't use any of the new features (don't know what those may be). V 17 does everything that I need.

Comment 32

[:aceman]

(In reply to APraxis from comment #31)
> I don't like the loss of the number of e-mails read and unread and I don't

Which numbers do you mean here?

Comment 33

[:aceman]

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #28)
> Aceman, are you aware of any other bugs currently open for this area?

I am not aware of any new problems here. I think this bug can stay closed, it has a great patch that should improve many things. APraxis has a separate problem about slowness of opening the folder pane, where we are not yet sure what is the cause. It may be the summarizing of subfolder values (unread/size) in the folder pane (bug 939462), or it could be this bad closing of msfs in case of many folders so I asked for his test here.

Comment 34

[APraxis]

(In reply to :aceman from comment #32)
> (In reply to APraxis from comment #31)
> > I don't like the loss of the number of e-mails read and unread and I don't
> 
> Which numbers do you mean here?

in the left pane there's a count on the right side of unread and total e-mails in a folder

Comment 35

[APraxis]

I updated to V 38, through the normal TB Check for Updates and the problem still existed, see my comments in bug 939462.

I then looked at this bug and downloaded the latest build 
thunderbird-47.0a1.en-US.win32.installer

I backed up my TB folders just in case formats in the files were changed, then installed V 47

I opened and closed TB several times and then started taking screen shots which have been attached, note that there's a 'not responding' time of about 13 seconds, this hasn't changed.

Comment 36

[APraxis]

Attachment: TB 01 work online.png

Comment 37

[APraxis]

Attachment: TB 02 initial window.png

Comment 38

[APraxis]

Attachment: TB 03 not responding.png

Comment 39

[APraxis]

Attachment: TB 04 enter pwd.png

Comment 40

[APraxis]

Attachment: TB 05 ready to go.png

Comment 41

[APraxis]

I'll assume that V47 hasn't broken my e-mail data and I'll leave the Daily build on my system for further testing if you ask me to.

Wow, just got a request from aceman re: bug 939462 as follows:
===============================================================================
--- Comment #88 from :aceman <[e-mail]> 2016-03-02 12:38:00 PST ---
OK, in that case it seems your problem is not fully caused by the summing of
unread/total msg counts across subfolders. So the bug here as filed is fixed by
bug 1135310 and bug 464973.

Can you agree that the performance has improved between TB24-38?

But you say it is not yet at the level of TB17. In that case I'd ask if you
could file a new bug please to allow for proper tracking of the issues.

Also please try TB45 first (should be out in a few weeks now). If the problem
is still there, please create the new bug and we can tell you how to run a
profiler to identify the code that is causing the slowness.
==============================================================================
but if V 47 still has the delay I don't see how V 45 wouldn't, what do you suggest?

Comment 42

[Wayne Mery (:wsmwk)]

(In reply to :aceman from comment #33)
> (In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment
> #28)
> > Aceman, are you aware of any other bugs currently open for this area?
> 
> I am not aware of any new problems here. I think this bug can stay closed,
> it has a great patch that should improve many things. 

Yeah. By no means am I suggesting to reopen this bug. Just a heads up that there may be another issue.

Comment 43

[Wayne Mery (:wsmwk)]

(In reply to APraxis from comment #41)
> ...
> but if V 47 still has the delay I don't see how V 45 wouldn't, what do you
> suggest?

Please file a new bug report

Comment 44

[APraxis]

I opened Daily, there was a delay, was told I was up-to-date with 47.0a1 (2016-03-01)
I added screen shots from V 47

Comment 45

[APraxis]

Attachment: 02 TB Daily V 47 initial screen.png

Comment 46

[APraxis]

Attachment: 03 TB Daily V 47 not responding.png

Comment 47

[APraxis]

Attachment: 04 TB Daily V 47 enter pwd.png

Comment 48

[APraxis]

Attachment: 05 TB Daily V 47 ready to go.png

Comment 49

[APraxis]

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #43)
> (In reply to APraxis from comment #41)
> > ...
> > but if V 47 still has the delay I don't see how V 45 wouldn't, what do you
> > suggest?
> 
> Please file a new bug report

Do you mean a new bug report for V 47?

Comment 50

[APraxis]

OK, new bug 1253235