528810 - get all new messages causes filters to not run - messages could not be filtered to folder "Inbox" because another operation is in progress.

Status: RESOLVED DUPLICATE of bug 168648

(Thunderbird :: General, defect)

Description

[David Taber]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR 3.5.30729)
Build Identifier: Thunderbird 3a3 + Eudora 8.0b7

I have 10 email accounts configured to download into 4 Thunderbird mailboxes.  

When I select "get new messages from all accounts" from the File menu, the downloads occur more or less in parallel.  Almost inevitably, I get an error message from Thunderbird indicating that the filtering action couldn't be done because other processes were using the mailboxes at the time.  This is made worse when TB indexing is on, as the indexing thread starts as soon as one of the downloaded accounts is complete, even though other accounts may still be working on the same TB mailbox.

Suggest:
  - "get all messages" work sequentially on the downloads, rather than in 
     parallel
  - or make the filters wait until the last of the download threads is complete
     before starting
  - or make the indexing thread wait the same way, and wait until the filtering
     is complete

Possibly, I'm doing something wrong or haven't configured TB correctly.  But I don't see an option relating to this topic.


Reproducible: Always

Steps to Reproduce:
1.  Have a bunch of un-downloaded emails strewn across several accounts
2.  Push the "download all messages" button
3.  Watch the "filters can't do their thing" error message pop up.
Actual Results:  
Filters don't always run.
Error messages pop up.

Expected Results:  
Filters would run after all downloads are complete, rather than in parallel with each other.

This bug has been here a loooong time.

Comment 1

[Ludovic Hirlimann [:Usul]]

If you are really running Thunderbird 3a3 please update to rc1 and let us know if it works or not.

Comment 2

[Wayne Mery (:wsmwk)]

(In reply to comment #1)
> If you are really running Thunderbird 3a3 please update to rc1 and let us know
> if it works or not.

still need your response (based on newest release of course) and exact text of the error message.

Comment 3

[David Taber]

Sorry, I had somehow missed your 17 Nov 2009 message.

I'm now on Eudora 8B9, which I guess is based on T'Bird 3.0.1

This symptom still occurs whenever there are a lot of mailboxes with new contents.

Comment 4

[David Taber]

The error messages that come up depend on the specific sequence of activities within Eudora.

One of them is:
   The messages could not be filtered to folder "Inbox" because another 
   operation is in progress.

That one can show up 8 times or more in a single mail download, as two feeder threads are trying to grab mail from different sources and put them in the same Eudora mailbox. While the error message is up, the downloads appear to block (or at least, the GUI blocks.

The other one is:
   This folder is being processed.  Please wait until processing is complete
   to get messages.

Sometimes, this is caused by the background indexer blocking the mail download.  But other times, this message will persist until I stop/restart Eudora...and when that occurs, one or more of my email accounts will not be downloading anything at all.  Has made for some embarrassing situations where I *thought* I was current, but am many hours behind the flow of email traffic.

Comment 5

[john]

There is another impact, related to the fact that the indexing thread takes on a life of its own:

If an indexing operation is being undertaken, it seems like there is no way to stop it so that mail can be accessed.   For example, if a few messages are being indexed, and one "gets mail", using the button, the request is impeded by the indexing.  I expect that the original poster's bug report's problem would occur when TBird undertakes an automatic "get mail" on intervals.

This suggestion, perhaps:

Establish a listener within the indexing threads, that recognize another thread's attempt at user operations, such as getting mail, or derivatives of those actions [such as the filtering] -- have that listener stop the indexing in those cases.

Yet more times when indexing blocks user activity:  When indexing is going on [such as when TBird status reports that 4 messages are being indexed], and for some reason that operation is occurring just when I am interested in moving to a message to read it.  Rather than stopping the index operation, pending my request, it makes me wait.

Another: when I reindex an entire mail folder [when, for unknown reasons, the index get tangled].  I can't stop the reindex, and on large folders, this means I have to wait a while to undertake a simple task like reading a message.  

In that context, I wonder what happens if I were to send a new message while indexing is ongoing...?   Does indexing stop the send?  I assume that the new message will get sent, saved and indexed -- is that true?

Comment 6

[Wayne Mery (:wsmwk)]

my understanding is indexing reads messages in a non-locking way, such that it could not be causing what you are seeing.

what is your check new mail interval?
are these imap accounts or pop?
  if pop, are you using global inbox, or separate inbox for each account?
what is your nnKB setting for auto compacting folders at Options|Advanced|Network & Disk space?
does the problem lessen or go away if you disable auto compact?

It looks like there is some consolidation to be done with the open bugs [1] that mention this error message and probably make it depend on some general but about the root cause.

[1] https://bugzilla.mozilla.org/buglist.cgi?type1-0-0=anywordssubstr&short_desc=%20&field0-0-0=short_desc&type0-0-1=substring&field0-0-1=component&classification=Client%20Software&classification=Components&query_format=advanced&longdesc=could%20not%20be%20filtered%20&short_desc_type=allwordssubstr&value0-0-1=filter&type0-0-0=anywordssubstr&value0-0-0=filter%20inbox%20get%20new&field1-0-0=short_desc&product=MailNews%20Core&product=Thunderbird&longdesc_type=substring

Comment 7

[[:Aureliano Buendía]]

It seems closely a dupe of bug #168648...

Comment 8

[john]

Occurs to me that this is entwined in how the index works, so I thought I would mention it:  

Sometimes, the status bars shows a messages akin to this:  "indexing 2 messages" -- that status line doesn't disappear, after a time lag where one might expect it to have completed.  Instead, it only disappears when I undertake another operation [say, for example, selecting a message that is within some folder -- thereupon the status line goes blank]

Comment 9

[David Taber]

Answering a series of related questions:
  - I've deliberately set my email auto-download intervals to be different prime numbers, so the automatic stuff shouldn't be colliding very often.
  - The effect is the worst when I hit the "get all messages" button, particularly if I've been off line for a while and have a bunch of stuff in each mail server (I'm downloading from 8 different accounts).
  - All of this is exclusively in the land of POP.  No IMAP allowed ;-)
  - Reading through bug #168648 it does seem to be a dupe and OMG it's been in the code line for 8 years!!  Nice.

The indexing is generally non-blocking for *reads*.  But obviously, downloads are writes and the filtering mechanism is blocking (or is causing a pop-up message which is blocking somewhere).

I believe the error message is a generic one, so it's a red herring IMHO.  The real issue is that several writing threads are trying to access the mailboxes (there are only 4 of those--each of them is being fed by two email servers).

Comment 10

[David Taber]

Actually, maybe there's a simple work-around here:  is there a way to disable the stupid pop-up message entirely?  If it didn't come up, the downloading wouldn't be blocked and T'Bird would be much more useful.

I couldn't find any magic parameters in config.edit...but maybe it's hidden...