260373 - When Global Inbox, Message Filters will not filter incoming mail, and/or "Run Filter on folder" does nothing [Read Thunderbird FAQ, please!]

Status: RESOLVED WORKSFORME

(MailNews Core :: Filters, defect)

Description

[Tom Lane]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; rv:1.7.3) Gecko/20040913 Firefox/0.10
Build Identifier: Thunderbird version 0.8 (20040913)

I have tried running the menu option to run filters against folder with no luck
and the filters do not work when inbound mail is received from POP server.  I
can find no other settings which would affect this.  After looking through
previous bug reports, it appears it works for some but I have not been able to
make it work at all, even with only one filter.  Filter was configured as:
Sender   Is   <email address>  ;  Move To  <folder name>.

Reproducible: Always
Steps to Reproduce:
1. Set-up Filter
2. Run Filter 
3.

Actual Results:  
None

Expected Results:  
Moved messages from specified email address to folder selected.

No Crash.

Comment 1

[JC Jones]

Confirmed under one absolutely fresh installation of Windows XP SP 2. 

Comment 2

[David :Bienvenu]

did you try sender contains instead of sender is? "is" does an exact match...

Comment 3

[David :Bienvenu]

Also, are you using the global inbox or per-account Inbox?

Comment 4

[Thommie Rother]

Confirmed for version 0.8 (20040913) on Win 2000 SP4 and filtering from an IMAP
account (cyrus imap). If I copy a running msgFilterRules.dat from moz. 1.73 to
tb 0.8, it is not used at all, although it is visible and editable in Tools,
Message Filters. New messages just remain in the standard inbox. My folder
structure is

Account |
        |--- Inbox
        |--- Drafts etc.
        |--- Other Folders (Targets for the Filter process)

Maybe tb does not detect the correct inbox? I also tested various server
settings (server directory, personal namespace), but no change. Now, I reverted
back to the standard settings (no entry for server dir, personal namespace
"INBOX.").

Comment 5

[deleteaccount]

I have a POP account, I think it's called, and I also cannot get filters to work
on my incoming email.  I did get them to work as run on folder once someone told
me about selecting them and using the dropdown menu of folders (I had assumed
enabling them and clicking run would be sufficient.)  I have the log enabled,
but it shows zip.  I am using contains, not is, and do not have a global inbox.
 Running prerelease 1.0.  windows xp.

Comment 6

[David :Bienvenu]

Can you attach your msgFilterRules.dat, and also tell me which directory you
found it in? It should be in the pop3 server sub-directory of your user profile
directory.

Comment 7

[deleteaccount]

From looking at the inbox file in each directory, cox.net appears to be the one
I am trying to filter.  cox-1.net is another username.

...Thunderbird\Profiles\*stuff.default\Mail\pop.east.cox-1.net has
version="8"
logging="no"
--------------------------------
and
...Thunderbird\Profiles\*stuff.default\Mail\pop.east.cox.net has
version="8"
logging="yes"
name="subscribe and maybe unsubscribe requests"
enabled="yes"
type="1"
action="Move to folder"
actionValue="mailbox://[e-mail]/Inbox"
condition="AND (to,contains,feline-owner) AND (from,contains,yahoogroups)"
name="spam to feline-owner"
enabled="yes"
type="1"
action="Move to folder"
actionValue="mailbox://[e-mail]/Junk"
condition="OR (to,contains,feline-owner)"

Comment 8

[David :Bienvenu]

I wonder if we're getting confused about what filters file to use...what happens
if you copy the filters file into the cox-1.net into the cox.net directory?

Comment 9

[deleteaccount]

Instead I tried setting up filters in the other user name and seeing if they
punted into the wrong user mailbox, etc.  Several simple ones I tried worked
properly - from my username, with subject containing, and one with two conditions.

Simple ones I created in the original username also worked.

I am now thinking that the next time I get such spam, I will post the heading
and perhaps someone can tell me why it is slipping thru, because I am setting my
catches for the stuff that is displayed, but perhaps those are not the fields I
think they are.  Unfortunately I deleted the spam I already received.

Comment 10

[Guillermo Rodriguez Garcia]

I am also seeing this same problem with tb 0.8. I have tb configured to use a
Global Inbox, and there's only one e-mail account. I setup the filters but when
I select Run Filters on Mailbox, nothing happens.

Playing around a bit I found the Mail subdirectory in the default profile's data
directory. Inside Mail/ there were two subdirs, Local Folders and one named like
my pop3 server. Each had a msgFilterRules.dat file. The file under the pop3
server's subdir had all the filter information, whereas the file under Local
Folders only had this:

version="8"
logging="yes"

I just copied the msgFilterRules.dat file from the pop3 server's subdir to the
Local Folders dir, and now everything works fine.

I assume filters don't currently play nice when using a Global Inbox ?

Comment 11

[E. M. Ryan]

I can confirm this bug on the released 0.9, and on the 20041105 nightly build. 
It appears to me that this bug ONLY applies to setups using the Global Inbox. 
If you use individual account inboxes, the filters work perfectly.  In all cases
I do have the filters set to apply to the "Local Folders" in the Filter dialog.
 For good measure, I also applied them to the individual account, but they still
don't work.

A few notes:

1. This is not limited to Windows.  I reproduced the bug on OSX (10.3.5).
2. It affects both the manually defined filters, and the built-in Junk Mail filters.
3. The filters to not automatically run as new mail is recieved.
4. The filters DO work if you run them manually (by selecting "Run filter on
folder" or by un-marking a message as Junk and then re-marking it).

Comment 12

[David :Bienvenu]

Message filters for incoming mail have to be defined for the actual incoming
account, not the local folders account.  That works fine for me...

Comment 13

[Milen A. Radev]

(In reply to comment #12)
> Message filters for incoming mail have to be defined for the actual incoming
> account, not the local folders account.  That works fine for me...

Was that a conscious decision? I have several filters that should work on all
incoming mail - should I "replicate" them to all (>8) mail accounts?

Comment 14

[Andre Meyer]

Of course, you can move or copy the msgFilterRules.dat file from Local Folders 
to the appropriate pop account folder(s), but that is very unhandy. When 
creating a new filter using the "Create filter from Message..." option the 
filter is created in the Local Folders again!

Comment 15

[E. M. Ryan]

One problem with using individually defined (per-email-account) Filters is that
the Junk Mail controls still don't work.  If you use a Global Inbox, spam is
correctly marked as Junk, but isn't moved according to what you have selected in
the junk mail controls.

If you unmark/remark the messages as Junk Mail, then they are correctly moved
(or whatever you've selected).  I suspect that this is related to this bug ...

I haven't tried it, but possibly you could work around it by defining a
per-email filter that searches for Junk status and moves/deletes/etc for you.

Comment 16

[Sean Vickery]

(Also in reply to comment #12..)

Thanks for the words of explanation. I will try following the advice and see if
it fixes the problem for me. (I'm using 0.9.)

If one's using the global inbox (very nice feature in my opinion), to have to
set the filters on the (now hidden in the folder pane) accounts is non-intuitive.

Perhaps someone could have a look at the filters UI with a view to improving it
for the global inbox. Perhaps the filters UI could be split into two parts:
1. A UI for defining the filters
2. A UI for specifying which filters apply to which folders/accounts.

Comment 17

[xave]

linux/tb .9 confirmed...

I tend to think that a an account using the Global inbox should either ignore
its own filters set or adds them to the Local Folders' ones. But it should use
the Local Folders filters anyway.

Comment 18

[Andre Meyer]

Using "Create Filter from Message..." filters are associated with the folder
where the message is received. By using the Global Inbox that folder is Local
Folders.

Simply put, filters associated with the Local Folders need to be run every time
that new mail is received in Local Folder's Inbox. That is the same behaviour
for all other accounts. Please, add this behaviour to the Global Inbox, too.

Comment 19

[Mike Cowperthwaite]

*** Bug 260471 has been marked as a duplicate of this bug. ***

Comment 20

[Mark Jansen]

My filters have been working for a few days, but now nothing happens unless I
manually activate the filters. 

This is exactly as is being described at the page
http://kb.mozillazine.org/index.phtml?title=Thunderbird_:_Issues#Miscellaneous

But.... there's no link to a solution for this problem!! Why??

Comment 21

[David A. Cobb]

I think this is the same major breakage in filtering I keep tripping over.
I'm using TB "version 0.6+ (20041121)" mostly, but I have a "releases-0.9"
version installed also.

There is no obvious difference between filters that work and those that do not.
For me, the majority of both test "\"List-Id\" ends-in <name.at.place.dom>";
some of each are old [pre-disaster], some are new [post-likewise].

I'm greatly saddened that the breakage is also post David's wonderful fix to
another long-standing gripe of mine.

Regarding David's Comment#8: I think you're onto something here.  
Relevant(?) -- I put all my mail including the Inbox in Local%20Folders.
As a result, I have two msgFilterRules.dat files of aproximately the same size.
 They also seem to track by modTime -- closely but not identically.
Somewhere along the way, TBird (nightly) decided it couldn't read my
msgFilterRules.dat and clobbered it.  With over 200 filters, I'm happy I had
some backup.  The clobbered one was in "pop.east.cox.net" -- the one in Local
wasn't hit.  

During this period, TBird-nightly would show me an empty pane in MessageFilters,
however it would apply the filters despite not being able to show them.  After
recovering the Rules file, TBird[release0.9] would correctly show the same
filters the nightly could not.  This debacle was solved by identifying one
filter that the prog was choking on and deleting it (with a text editor).  

It does seem that, since the (VeryGoodThing) change that made it possible to
have my Inbox in Local Folders, there is some confusion about which place to look. 

The major fact to note is: I can fetch a batch of mail, see from the content of
my Inbox that a lot of filters didn't work, then select "Apply Filters to
Folder" on the Inbox and see the same set of filters do just what they should.  
Somehow, the logic paths for post-fetch filtering and for "during-fetch"
filtering are not consistant.

Anyway, 

Comment 22

[David :Bienvenu]

I suspect there's just a bit of confusion about the fact that there are filters
for local folders (which don't get applied to incoming mail, but do get applied
when you run filters on folders on the global inbox) and filters for each of
your pop3 accounts. You have to see which set of folders you're looking at when
you edit filters...compounding the confusion, some of the commands to create
filters from a message would create a local folders filter, which doesn't get
applied to incoming mail. I'm trying to clean up the UI so it edits the incoming
server filters as much as possible, until the time that local folder filters can
get applied to incoming mail.

Comment 23

[David A. Cobb]

The biggest question, of course, is: if I select my "Inbox" then do
&Tools+&Message&Filters, which msgFilterRules.dat am I editing?  
My recent bad experience suggests the one in the server directory (pop.east...).

Does it get copied into Local%20Folders at some point?  Because newly-created
filters do get processed by "Run Filters on Folder."  

Confusing is hardly the word.  

How can I help?

PS - this should at least be confirmed, since a few folk have now written about it.

Comment 24

[David :Bienvenu]

> My recent bad experience suggests the one in the server directory (pop.east...).

No, until tomorrow, it usually would be the local folders account. Tomorrow, it
will almost always be the server's filters, the ones that get run when you get
new mail on the server. If you want to create filters for the local folders to
run later, you will explicitly need to pick the local folders account from the
drop down in the filter editor UI. The rationale is that filters run on incoming
mail is the usually scenario. Running filters after the fact is more of a power
user feature.

When you are editing filters, it does tell you which account's filters are
getting edited, unless you're creating a new filter from an exisiting message.

Comment 25

[WADA:World Anti-bad-Duping Agency]

See Thunderbird FAQ.
http://kb.mozillazine.org/index.phtml?title=Thunderbird_:_FAQs_:_Filters
This is the best description we can get currently, I think.

Comment 26

[WADA:World Anti-bad-Duping Agency]

*** Bug 271472 has been marked as a duplicate of this bug. ***

Comment 27

[WADA:World Anti-bad-Duping Agency]

Change summary for ease of search.

Comment 28

[Dubravko Radiæ]

Read thru this thread and I can CONFIRM that both for versions 0.9 AND 1.0 (0.8-
cannot confirm):

-Message Filters for the Global Inbox do not filter incoming mail.

-Junk Mail Auto mark and filtering does not work also.


  It is obtained easily as I've recently upgraded from ISDN to 384/64 DSL,thus 
I am having Thunderbird always opened,checking for incoming mail every ten 
minutes. 

Reproductibility: million percent...define msg rules ,then click "get mail" 
icon to collect mails from accounts to your global inbox.

Severity: hm...It is annoying for the user as the user must manually 
filter,THEN navigate to the filter-dest folder. Two steps instead of 
one,million times a day.

Greetings to the engineering/debugging community

Comment 29

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #28)
> Read thru this thread and I can CONFIRM
Dubravko, have you read thru FAQ I mentioned in comment #25 too?

Comment 30

[David :Bienvenu]

you need to define filters on the actual incoming server, not the global inbox.
Prior to 1.0, the default create new filter behavior created a filter for the
local folders account; in 1.0, it creates a filter for the pop3 server, if you
have a message selected...or the default server...

Comment 31

[Josh Gerdes]

I have notice this problem in Thunderbird 1.0, and it should definitely be
fixed. Yes, it is possible to use individual filters for the same tasks, but
global message filters would be much more convenient in many cases.

Comment 32

[Phil Paradis]

(In reply to comment #30)
> you need to define filters on the actual incoming server, not the global inbox.
> Prior to 1.0, the default create new filter behavior created a filter for the
> local folders account; in 1.0, it creates a filter for the pop3 server, if you
> have a message selected...or the default server...

Using 1.0 (20041206); freshly installed on XP Pro SP-2. Clicking an email
address in the headers of an existing message and selecing Create Filter from
Message creates the rule in Local Folders, not the account that received the
message. Further, the UI for creating the filter does not allow this to be
overridden, and there is no UI to move a rule from one filter set to another.
I'm stuck copying the address to the clipboard, cancelling the new rule, and
manually creating a new one in the correct filter set. The junk mail filter
doesn't seem to be learning much either--I'm constantly marking nearly identical
spam as junk each day. I wonder if the Junk button is doing something similar,
i.e. adding the new filter data to the Local Folders ruleset, while incoming
mail is filtered using an account-specific file that isn't being updated.

Comment 33

[WADA:World Anti-bad-Duping Agency]

Following is open bug list which contains "Global" & "Filter" in Summary.
(as of today)

Bug  34973 	2005-01-08 	NEW  	MailNews 
	Would like global message filter option
Bug 242665 	2004-11-23 	NEW  	MailNews 
	Create Filter From Message is disabled in local folders/global inbox
Bug 257979 	Thu 21:25 	NEW 	MailNews 
	pop3 deferred accounts (global inbox) should have ability to run
	 filters based on the deferred server and the deferred to server
Bug 263541 	2004-12-16 	NEW  	MailNews 
	New Thunderbird 0.8 install, POP, defaulted to global inbox,
	 can't run filters after the fact
Bug 271729 	2004-12-20 	UNCO 	Preferen 
	No Automatic Filter For Global Inbox
Bug 275132 	2004-12-23 	UNCO 	MailNews 
	Can't filter/move messages to Inbox that is busy, results in loss
	 of moved messages when Global Inbox(mail is not moved to it)

Comment 34

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #32)
Phil Paradis, search bugzilla for open bugs with "Product=Core &
Component=MailNews: Filters".
Enhancements/improvements in this area are already requested(many of them are
"Resolution=Enhancement".)

Comment 35

[Hrvoje Hladnik]

In version 1.0.2 this bug is still here. I have couple of email accounts 
created in Account Settings and all of them store Inbox emails into general 
account Inbox folder. Also, I have filters for general account, but filters are 
never automaticaly started. I must run filter manualy from menu.

Comment 36

[David :Bienvenu]

this is only fixed in trunk builds...

Comment 37

[Henrik Skupin [:whimboo][⌚️UTC+1]]

David, this bug still happens for current trunk builds and I'm thinking that I know why it happens.

If you are using a Global Inbox to store messages from different POP3 accounts, filters which were created for these POP3 accounts don't work. The filter has to be created for the target account. Do these steps to reproduce the issue:

1. Create a POP3 account
2. Create a second POP3 account and set "Inbox for a different account": first POP3
3. Receive a message from account2
4. Create a filter by right clicking on the address in the header pane
5. Create the same filter for account1
6. Open Tools | Message Filter and run filter for account2 => nothing happens
7. Choose account1 in the dropdown and run this filter => Filter works

The filter you have created for the received message is stored for account2. Which seems ok after reading this bug. But running the filter doesn't seem to work on the correct folder? 

I was able to track it down until StartTimer() in SearchWOUrls:
http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/mailnews/base/search/src/nsMsgSearchSession.cpp&rev=1.65&#557

Where does the real search happen?

Comment 38

[RC]

This happens to me currently with 2.0.0.12. No global inbox, using pop. My older filters are still working, but any new filters I create are completely ignored and do nothing regardless of how I setup the filter - be it contains or is - nothing.
It's very annoying.
It fails automatically and it fails manually.
OS X 10.5.2

Comment 39

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #37)
> 6. Open Tools | Message Filter and run filter for account2 => nothing happens
> 7. Choose account1 in the dropdown and run this filter => Filter works

This is problem when (a) Message filter for "hidden account" of Global Inbox use, and (b) "Run selected filters on:" on folder of "Global Inbox owner account".

When message filter for "hidden account(account2 in your case)" is chosen at Tools/Message Filters, folder listed at "Run selected filters on:" is folder of "Global Inbox Owner(account1 in your case)", because owner account of the filter is "hidden account of Global Inbox use".
I think this mismatch is not properly handled yet. The phenomenon could be observed with Seamonkey 1.1.9.

To Henrik Skupin:
Open separate bug please, in order to avoid confusion/misleading, for ease of problem analysis.

Comment 40

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #38)
> No global inbox,

To RC:
As clearly written in bug summary, this bug is only for "Global Inbox use" case. Please don't add comment to bug which has no relation to your phenomenon/issue.

> older filters are still working, but any new filters I create are completely ignored
Sounds phenomenon of Bug 362539.
Read all of many bugs closed as Dup of Bug 362539 first. 

Comment 41

[Prateek]

I have been using Thunderbird for 5 years or so, and recently I have been having this problem. I have one mail account with global inbox enabled (so all mails come to Local Folders). I have several filters set up which sort out mails into various folders. These have been working for years and without any change on my part, have stopped working. I recently upgraded from Kubuntu 11.04 to Kubuntu 11.10, that might be related.

In the Filters window, the filters appeared under the individual mail account, not under Local Folders. Running them manually through the Filters window or Tools menu does nothing. I copied the file msgFilterRules.dat from the folder pop.gmail.com/ to the folder Local Folders/, and then manually ran the filters on Local Folders. Some old mails to which the filters somehow had not been applied were moved where they should have been, but recent mails to which the filters should have been applied were untouched. I have enabled filter logging (enabled recently, just to debug this issue), and I found out the above information (that old mails were moved) only from the filter log.

Comment 42

[Wayne Mery (:wsmwk)]

Giacomo says this is gone. THanks for testing