1215933 - FIlter loop moving messages using post-processing filters(infinite loop of filter move when "Filter after Junk Classification" is used. caused by bug 770137 and bug 1320676)

Status: NEW

(MailNews Core :: Filters, defect)

Description

[mrbi11]

User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36

Steps to reproduce:

I updated to the current version. I do that rarely, because it usually causes problems.





Actual results:

My preexisting and working filters stopped working.
Rules of the form "move to folder X, stop filter execution"
and "Move to folder Y, stop filter execution" would constantly move emails back and forth between X and Y folders.
They were set correctly, and previously worked. I had to disable one of them to get it to stop oscillating.

In this case it was moving emails so often it created duplicates sometimes. It would oscillate for hours till i read email and intervened.


Expected results:

Whichever is first in the list should be the location of the email (X in above case).

It may be relevant that the form of my email folders has all
folders i create as a sub-folder of Inbox. ie inbox/X and inbox/Y.

possibly the movement of the email appears to be a new inbox entry.

email acct is IMAP accessed at inmotionhosting.com

Comment 1

[mrbi11]

It may not have been clear that I jumped to current release version from version 1 years back.
So the behavior change I cannot narrow down much more than in the past year or so.

Comment 2

[Björn Victor]

This looks like bug #1216197. It is extremely annoying.

Comment 3

[Wayne Mery (:wsmwk)]

(In reply to Björn Victor from comment #2)
> This looks like bug #1216197. It is extremely annoying.

mrbi11, do you agree?


> It may not have been clear that I jumped to current release version from
> version 1 years back.

Like version 31?

Comment 4

[mrbi11]

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #3)
> (In reply to Björn Victor from comment #2)
> > This looks like bug #1216197. It is extremely annoying.
> 
> mrbi11, do you agree?
Yes. Rather that bug is like this one. I assume the numbers are assigned in order, and this bug was the earlier report.
> 
> 
> > It may not have been clear that I jumped to current release version from
> > version 1 years back.
> 
> Like version 31?
Oddly, I can confirm version 31 does not have the bug. I have tbird on several computers
(aside, having filters on multiple computers can also cause this behavior sort of, so i only have active filters
on one computer. Some genius out there that can figure out how to make tbird more multi-computer friendly and
have the filters work as they are written. Perhaps a header flag can be added to the email that says I've been filtered?)

But back to this problem, I have my filters moved to a computer running version 31. They work correctly there.
So some time after version 31 they broke.

Comment 5

[Kent James (:rkent)]

Please give details on the failing filter. Like, account type, pre- or post- junk filtering, type of folders move from and to.

I'm also confused by "would constantly move emails back and forth between X and Y folders". At least for IMAP, by default incoming filters are only processed on the INBOX folders.

There are many different code paths for filters, and we need more details to figure out which one is being used here.

Comment 6

[mrbi11]

(In reply to Kent James (:rkent) from comment #5)
> Please give details on the failing filter. Like, account type, pre- or post-
> junk filtering, type of folders move from and to.

It happens an all filter types. The bug is that the "stop filter execution" fails.
The email account is accessed only via imap, because i access it from several locations.

> I'm also confused by "would constantly move emails back and forth between X
> and Y folders". At least for IMAP, by default incoming filters are only
> processed on the INBOX folders.

I think it may be an interaction with my isp. As noted above,
all folders are subfolders of inbox. (not my idea, my isp is set up that way)
Clearly it is applying the filters again as a side effect of moving the email. 

The email is moved between inbox/x and inbox/y and back.
So either the isp is flagging it as a new mail (i am unfamiliar with the details of the inner workings)
or tbird is treating moved mail as new and re-filtering.
> 
> There are many different code paths for filters, and we need more details to
> figure out which one is being used here.

My suggestion would be to visit relevant code changes (filters, new mail handling)
between version 31 and 38. Its often more fruitful to identify when a bug was introduced,
and simply make a close review of the changed code.

Thanks for investigating. This makes filters (and hence tbird) unusable for me.
Having an older version running gave me a stop gap, but I cant use the release version at all.
I depend on some sorting being done by the filters.

Comment 7

[Kent James (:rkent)]

"It happens an all filter types."  Please answer my questions. Even better, starting from a clean profile, give detailed steps to create a filter that shows this behavior.

"Stop Filter Execution" has never worked fully correctly in all filter types, and is known to not work correctly now in all filter types. This is related to the issue that filters are not processed async, while some of the filter actions ARE async. The best that could be hoped for in the short run, short of a fairly extensive rework of filter code, is to mitigate any regressions that are keeping you from getting your work done.

Comment 8

[mrbi11]

Generally, the filter is on from, to, cc or bcc, or on subject.
I have probably 30 of these.

They all work in version 31, not in version 38.
I would get emals bouncing around, deduced the issue and started disabling one of the relevant filters.
Then I removed all the filters from tbird 38, and moved them to tbird 31 on another pc.

bug 1216197 also has a reproduction, with 2 filters.
If that reproduction doesnt work i can try to package one up for you.
In the event its ISP specific i can also set up a test email account on one of my domains for you.

Trying to generate a reproduction involves me effectively disabling my working email filters for some hours while i work,
so i'd like confirmation the reproduction in 1216197 is not working for some reason.
I do appreciate the help, and i'm willing to work at it for you/with you, but if the other reproduction
is working I thought you had what you needed. Can you confirm one way or the other?

Comment 9

[Kent James (:rkent)]

The other reproduction uses post-processing filters. I've now asked twice for details on your filters from comment 5, and I still cannot figure out what you are using.

Let's assume you are using post-processing, if so why? Try using a normal pre-processing filter instead. The code paths are completely different, and will have different bugs and issues.

Comment 10

[mrbi11]

I dont know what you mean by pre and post processing.
If that refers to before and after spam filtering, I use both. Where The choice is based on what actually works.
And they did work when I set them up. By work I dont mean tbird, I mean if some legit messages were going to spam folder
i set the filter to run first, and if spam was sneaking in thru my filters i ran them after spam filtering.

I dumped my filters, which are only in my current email fuelscience.org.
Previously i had my main email at me.com.
When I moved emails, I exported the me.com filters and imported them into the fuelscince.org tbird account.
Below is the dumpe of FUELSCIENCE.ORG I double checked.
It has as actions things like actionValue="imap://[e-mail]/INBOX/ebay"
but it is NOT in fact moving email to the wrong account, I double checked that too.
So although the dump has the wrong email account throughout, the filters are in fact sending the email to the right folder.

So next I created filter named NEWFilterTest, under account Fuelscience.org, which moves email to "ebay on Fuelscience"
and experted again, and the new filter just created still says its moving to the wrong email account, although it is in fact working
and moving to the right account.

I surmise that export filters is also broken. (yeah! in sad voice).
Anyway, I think you can glean everything from the filters listed below.
Again, these are all working on version 31. This is an export from version 31.
I have reviewed it and no personal information that isnt already public is there.

Here is the export of the filters.
RootFolderUri=imap://[e-mail]
mailnews.customHeaders=reply-to:     from:     to:     cc:     bcc
version="9"
logging="yes"
name="NEWFilterTest"
enabled="yes"
type="17"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/ebay"
action="Stop execution"
condition="OR (all addresses,contains,ebay) OR (all addresses,contains,paypal)"
name="craigs"
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/craig"
action="Stop execution"
condition="OR (all addresses,contains,craigs) OR (body,contains,craigslist) OR (subject,contains,massage) OR (all addresses,contains,scaldas) OR (all addresses,contains,backpage) OR (all addresses,contains,411.com) OR (all addresses,contains,eccie) OR (all addresses,contains,dallas13) OR (all addresses,contains,liannfb) OR (all addresses,contains,let327) OR (all addresses,contains,spencer ponce)"
name="From is: [e-mail]"
enabled="yes"
type="17"
action="Delete"
action="Stop execution"
condition="AND (from,is,[e-mail]) AND (subject,contains,update billing)"
name="zipspam"
enabled="no"
type="17"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/Junk"
action="Stop execution"
condition="OR (from,contains,zipcloud.com)"
name="[e-mail]"
enabled="yes"
type="17"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/fuel"
action="Mark flagged"
action="Stop execution"

condition="AND (from,is,[e-mail]) AND (subject,contains,statement)"
name="ebay2"
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/ebay"
action="Stop execution"
condition="OR (all addresses,contains,ebay) OR (all addresses,contains,paypal) OR (all addresses,contains,me3d.us) OR (all addresses,contains,bonanza)"
name="ebay"
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/ebay"
action="Stop execution"
condition="OR (all addresses,contains,ebay) OR (all addresses,contains,auctiva) OR (all addresses,contains,paypal) OR (all addresses,contains,me3d.us) OR (all addresses,contains,bonanza) OR (all addresses,contains,etsy)"
name="Match All Messages"
enabled="no"
type="17"
action="Stop execution"
condition="ALL"
name="spiral"
enabled="yes"
type="17"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/alibaba/nylon"
action="Stop execution"
condition="OR (all addresses,contains,wishsino) OR (all addresses,contains,vdaytech) OR (all addresses,contains,rapid-mfg) OR (all addresses,contains,gyrmodel) OR (all addresses,contains,smartprototype) OR (all addresses,contains,yuanhaimould) OR (all addresses,contains,scienake) OR (all addresses,contains,beokey) OR (all addresses,contains,dademin) OR (all addresses,contains,d4e-sunfly) OR (all addresses,contains,dshmould) OR (all addresses,contains,wt-industry) OR (all addresses,contains,melody hung) OR (all addresses,contains,cnc-prototype)"
name="Subject contains: bearing"
enabled="yes"
type="17"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/alibaba/bearing"
action="Stop execution"
condition="OR (subject,contains,bearing)"
name="fastener"
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/alibaba/fastener"
action="Stop execution"
condition="AND (all addresses,contains,cancofasteners.com)"
name="cnc quotes"
enabled="yes"
type="17"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/alibaba/cnc"
action="Stop execution"
condition="OR (all addresses,contains,wyhardare) OR (all addresses,contains,szkingmetal) OR (all addresses,contains,xy-global) OR (all addresses,contains,[e-mail]) OR (all addresses,contains,[e-mail]) OR (all addresses,contains,hnc888) OR (all addresses,contains,imission.com) OR (all addresses,contains,szkele) OR (all addresses,contains,[e-mail]) OR (all addresses,contains,starzx) OR (all addresses,contains,pitrukrupa) OR (all addresses,contains,zhang angelina) OR (all addresses,contains,qdystech) OR (all addresses,contains,kevinlee84624) OR (all addresses,contains,uni-industry) OR (all addresses,contains,vivian.shiyang) OR (all addresses,contains,sylue machine) OR (all addresses,contains,starzx@123) OR (all addresses,contains,custom-processing) OR (all addresses,contains,cindymachinery) OR (all addresses,contains,khmetal) OR (subject,contains,huayi alum) OR (subject,contains,cnc) OR (all addresses,contains,sddcly)"
name="howto "
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/howto"
action="Stop execution"
condition="OR (all addresses,contains,insight) OR (all addresses,contains,support) OR (all addresses,contains,customer) OR (all addresses,contains,second copy) OR (all addresses,contains,mozilla) OR (all addresses,contains,cnet) OR (all addresses,contains,pingtest) OR (all addresses,contains,speedtest) OR (all addresses,contains,disqus.net) OR (all addresses,contains,practicalmachinist) OR (all addresses,contains,synthesis) OR (all addresses,contains,mobileme) OR (all addresses,contains,portforward) OR (all addresses,contains,getsatisfaction) OR (all addresses,contains,sorenson) OR (all addresses,contains,forum) OR (all addresses,contains,droid) OR (all addresses,contains,alkservice) OR (all addresses,contains,trigem) OR (all addresses,contains,dkszone) OR (all addresses,contains,MascoCabinetry.com) OR (all addresses,contains,facebook) OR (all addresses,contains,ieee) OR (all addresses,contains,sinful) OR (all addresses,contains,desmos.com) OR (all addresses,contains,magellan) OR (all addresses,contains,vgirls) OR (all addresses,contains,tuxtools) OR (all addresses,contains,wikipedia) OR (all addresses,contains,gary felder) OR (all addresses,contains,codeproject) OR (all addresses,contains,playstation) OR (all addresses,contains,samsung.com) OR (all addresses,contains,samsungusa.com) OR (all addresses,contains,hp.com) OR (all addresses,contains,straighttalk) OR (all addresses,contains,yilicasting) OR (all addresses,contains,gastleads) OR (all addresses,contains,126.com) OR (all addresses,contains,linearmotion) OR (all addresses,contains,sygic) OR (all addresses,contains,htgaustin) OR (all addresses,contains,pump) OR (all addresses,contains,trustpilot) OR (all addresses,contains,icloud) OR (all addresses,contains,wikimedia) OR (all addresses,contains,sourceforge)"
name="github"
enabled="yes"
type="17"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/github"
action="Stop execution"
condition="OR (all addresses,contains,[e-mail])"
name="nema"
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/alibaba/motor"
action="Stop execution"
condition="OR (all addresses,contains,hzmgdj.com) OR (all addresses,contains,smoothmotor) OR (all addresses,contains,casun.mobi)"
name="SLS "
enabled="yes"
type="17"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/alibaba/nylon"
action="Stop execution"
condition="OR (subject,contains,\" SLS \") OR (all addresses,contains,ebyton)"
name="ball screw"
enabled="yes"
type="17"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/alibaba/ballscrew"
action="Stop execution"
condition="OR (all addresses,contains,michelle liu) OR (all addresses,contains,[e-mail]) OR (all addresses,contains,[e-mail])"
name="ceramic"
enabled="no"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/alibaba/ceramic"
action="Stop execution"
condition="OR (all addresses,contains,ceranj) OR (all addresses,contains,jinyuco) OR (all addresses,contains,ceramic) OR (all addresses,contains,chinajiuchen) OR (all addresses,contains,suindustry) OR (all addresses,contains,aliyun) OR (all addresses,contains,dsindustries) OR (all addresses,contains,yeso) OR (all addresses,contains,doubleegret) OR (all addresses,contains,gongtaoceramics.com) OR (subject,contains,zirconia) OR (body,contains,Zirconia) OR (subject,contains,ceramic) OR (subject,contains,camille wu) OR (subject,contains,helen zhang) OR (subject,contains,Lianyungang) OR (subject,contains,Shenglongpan) OR (subject,contains,Changsha Zhonglong ) OR (subject,contains,Huayuexin Precise)"
name="thermistor"
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/alibaba/thermistor"
action="Stop execution"
condition="OR (all addresses,contains,fu john) OR (all addresses,contains,honest-well) OR (all addresses,contains,f-power) OR (all addresses,contains,evalit.tc)"
name="turbine"
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/alibaba"
action="Stop execution"
condition="OR (all addresses,contains,casting)"
name="repetier"
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/repetier"
action="Stop execution"
condition="OR (all addresses,contains,repetier) OR (all addresses,contains,henning)"
name="newegg"
enabled="yes"
type="17"
action="Delete"
action="Stop execution"
condition="OR (from,contains,[e-mail])"
name="hotmail spam"
enabled="yes"
type="17"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/Junk"
action="Stop execution"
condition="AND (from,contains,hotmail) AND (from,isn't in ab,moz-abmdbdirectory://abook.mab) AND (from,isn't in ab,moz-abmdbdirectory://history.mab) AND (from,isn't in ab,moz-abmdbdirectory://abook-3.mab)"
name="trafficgate delete"
enabled="yes"
type="17"
action="Delete"
condition="OR (from,contains,trafficgate) OR (from,contains,stellarrecovery)"
name="parts"
enabled="yes"
type="17"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/parts"
action="Stop execution"
condition="OR (all addresses,contains,smcusa) OR (all addresses,contains,pneumatic) OR (all addresses,contains,precisionrings) OR (all addresses,contains,topcatengines) OR (all addresses,contains,shapeways) OR (all addresses,contains,mcm elect) OR (all addresses,contains,superpcb) OR (all addresses,contains,pcb) OR (all addresses,contains,keyence) OR (all addresses,contains,solidworks) OR (all addresses,contains,unixsurplus) OR (all addresses,contains,3d) OR (all addresses,contains,netfabb) OR (all addresses,contains,solidconcepts) OR (all addresses,contains,aar@cpan) OR (all addresses,contains,bensonengineering)"
name="To contains: bill9@windhome"
enabled="yes"
type="17"
action="JunkScore"
actionValue="100"
action="Mark read"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/Junk"
condition="AND (to,contains,bill9@windhome) AND (from,isn't in ab,moz-abmdbdirectory://abook-3.mab) AND (from,isn't in ab,moz-abmdbdirectory://history.mab) AND (from,isn't in ab,moz-abmdbdirectory://abook.mab)"
name="frazells"
enabled="yes"
type="17"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/zbarb"
action="Stop execution"
condition="AND (all addresses,contains,[e-mail])"
name="friends"
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/friends"
action="Stop execution"
condition="OR (all addresses,contains,winford) OR (all addresses,contains,mzt8ken) OR (all addresses,contains,john chan) OR (all addresses,contains,carley) OR (all addresses,contains,fowlkes) OR (all addresses,contains,sub fox) OR (all addresses,contains,subfox) OR (all addresses,contains,mliberty) OR (all addresses,contains,krista.net) OR (all addresses,contains,nancy) OR (subject,contains,sierra) OR (all addresses,contains,jim kelley) OR (all addresses,contains,beck) OR (all addresses,contains,hagood) OR (all addresses,contains,flajnik) OR (all addresses,contains,d_webster) OR (all addresses,contains,mpaige) OR (all addresses,contains,linda_redman) OR (all addresses,contains,simonsdana) OR (all addresses,contains,buffyvsangel) OR (all addresses,contains,melanie anne) OR (all addresses,contains,malanie anne) OR (all addresses,contains,melanie manning) OR (all addresses,contains,curt hern) OR (all addresses,contains,masry) OR (all addresses,contains,zander_k) OR (all addresses,contains,roth) OR (all addresses,contains,connieadk) OR (all addresses,contains,eclecticangler)"
name="web"
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/web"
action="Stop execution"
condition="OR (from,contains,sedo.co) OR (all addresses,contains,dotster) OR (all addresses,contains,website) OR (all addresses,contains,hosting) OR (all addresses,contains,hostgator) OR (all addresses,contains,norton) OR (all addresses,contains,google.com) OR (all addresses,contains,icann)"
name="high"
enabled="yes"
type="17"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/alibaba/high"
action="Stop execution"
condition="OR (all addresses,contains,vivasd) OR (all addresses,contains,imission.com.cn) OR (all addresses,contains,waboncast)"
name="m"
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/med"
action="Stop execution"
condition="OR (all addresses,contains,drugstore.com) OR (all addresses,contains,aetna) OR (all addresses,contains,sonobello) OR (all addresses,contains,tmb.state.tx.us) OR (all addresses,contains,cherpack)"
name="To doesn't contain: craigslist"
enabled="yes"
type="17"
action="JunkScore"
actionValue="100"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/Junk"
action="Mark read"
action="Stop execution"
condition="AND (to,doesn't contain,craigslist) AND (body,contains,craigslist) AND (body,contains,free) AND (body,contains,apple) AND (body,contains,teak)"
name="mod"
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/mod"
action="Stop execution"
condition="OR (all addresses,contains,schaubhut) OR (body,contains,schaubhut) OR (all addresses,contains,almanza) OR (all addresses,contains,page mi) OR (all addresses,contains,walkerlaw) OR (all addresses,contains,incharge) OR (all addresses,contains,rachel stroud)"
name="software"
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/software"
action="Stop execution"
condition="OR (all addresses,contains,nilesh) OR (all addresses,contains,thandadude) OR (all addresses,contains,Fitzgerald silva) OR (all addresses,contains,scriptlance) OR (subject,contains,scriptlance) OR (all addresses,contains,basabi) OR (all addresses,contains,e-software-license) OR (all addresses,contains,basabi) OR (all addresses,contains,ave-nir) OR (all addresses,contains,sathish) OR (all addresses,contains,treat24) OR (all addresses,contains,webhealer) OR (all addresses,contains,steelman) OR (all addresses,contains,freelancer) OR (all addresses,contains,nelly) OR (all addresses,contains,php.net) OR (all addresses,contains,klovadis)"
name="med"
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/med"
action="Stop execution"
condition="OR (all addresses,contains,mayo) OR (all addresses,contains,restora) OR (all addresses,contains,walgreens) OR (all addresses,contains,prescriptionsolutions) OR (all addresses,contains,medicare) OR (all addresses,contains,balconespain) OR (all addresses,contains,aclepsa) OR (all addresses,contains,gology) OR (all addresses,contains,seton) OR (all addresses,contains,drug) OR (all addresses,contains,kabani) OR (all addresses,contains,zenni) OR (all addresses,contains,tria beauty) OR (all addresses,contains,health)"
name="insurance"
enabled="yes"
type="17"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/bank/insurance"
action="Stop execution"
condition="OR (all addresses,contains,travelers) OR (all addresses,contains,progressive) OR (all addresses,contains,esurance)"
name="bank"
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/bank"
action="Stop execution"
condition="OR (all addresses,contains,fargo) OR (all addresses,contains,chase.com) OR (all addresses,contains,owb.com) OR (all addresses,contains,dpscu) OR (all addresses,contains,genworth) OR (all addresses,contains,quicken) OR (all addresses,contains,thinkorswim) OR (subject,contains,loan) OR (subject,contains,mortgage) OR (all addresses,contains,surepoint) OR (all addresses,contains,downsinc) OR (all addresses,contains,lender)"
name="kids2"
enabled="yes"
type="17"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/kids"
action="Stop execution"
condition="OR (all addresses,contains,leanderisd) OR (\"reply-to\",contains,leanderisd) OR (all addresses,contains,naviance)"
name="groups"
enabled="yes"
type="48"
action="Move to folder"
actionValue="mailbox://nobody@Local%20Folders/groups"
action="Mark read"
action="Stop execution"
condition="OR (all addresses,contains,groups) OR (all addresses,contains,poly-austin) OR (all addresses,contains,yahoogroups) OR (subject,contains,austin_voyagers) OR (all addresses,contains,gwnn)"
name="kids"
enabled="yes"
type="17"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/kids"
action="Stop execution"
condition="OR (all addresses,contains,leander) OR (subject,contains,cphs) OR (subject,contains,cpms) OR (body,contains,zander) OR (\"reply-to\",contains,leander) OR (all addresses,contains,leanderisd) OR (all addresses,contains,schoolmessenger)"
name="nozzle"
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/alibaba/nozzle"
action="Stop execution"
condition="OR (all addresses,contains,eastwins) OR (all addresses,contains,kennethfang2013) OR (subject,contains,nozzle)"
name="spring"
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/alibaba/spring"
action="Stop execution"
condition="OR (all addresses,contains,hessp) OR (all addresses,contains,xmhsth) OR (all addresses,contains,tinfin) OR (all addresses,contains,pdmindustry) OR (all addresses,contains,xmtcjh)"
name="rambo"
enabled="no"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/alibaba/rambo"
action="Stop execution"
condition="OR (all addresses,contains,mokotechnology) OR (subject,contains,rambo) OR (all addresses,contains,szeastwin) OR (all addresses,contains,)"
name="spam before howto"
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/Junk"
action="JunkScore"
actionValue="100"
action="Mark read"
action="Stop execution"
condition="OR (all addresses,contains,@support.com) OR (all addresses,contains,trafficgate) OR (subject,contains,\" pills \") OR (all addresses,contains,laketravis.com) OR (all addresses,contains,flag) OR (all addresses,contains,sleepapnea) OR (all addresses,contains,bill taylor) OR (subject,contains,tax problem) OR (from,contains,phonesystem) OR (from,contains,doctorate) OR (subject,contains,gerber) OR (all addresses,contains,menslocker) OR (all addresses,contains,Gerber) OR (all addresses,contains,googlereply)"
name="spam"
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/Junk"
action="JunkScore"
actionValue="100"
action="Mark read"
action="Stop execution"
condition="OR (subject,contains,pharmacy) OR (subject,contains,viagra) OR (subject,contains,cialis) OR (subject,contains,levitra) OR (subject,contains,walk in tub) OR (subject,contains,walk-in tub) OR (subject,contains,online doctorate) OR (subject,contains,flag design) OR (subject,contains,casino) OR (all addresses,contains,scott.reed) OR (all addresses,contains,scottreed) OR (all addresses,contains,linkedin) OR (all addresses,contains,yourpolicyshop) OR (all addresses,contains,laketravis) OR (all addresses,contains,insideapple) OR (to,contains,dixon_salv)"
name="To contains: undisclosed"
enabled="yes"
type="17"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/Junk"
action="Stop execution"
condition="OR (to,contains,undisclosed)"
name="fuel"
enabled="no"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/fuel"
action="Stop execution"
condition="OR (all addresses,contains,plugin) OR (all addresses,contains,ford.com) OR (all addresses,contains,jlumptx) OR (subject,contains,patent) OR (all addresses,contains,tease) OR (all addresses,contains,uspto) OR (subject,contains,engine) OR (all addresses,contains,mikedavis) OR (all addresses,contains,inovair) OR (all addresses,contains,bertoninc) OR (all addresses,contains,elsevier) OR (all addresses,contains,pacificbearing) OR (all addresses,contains,pneumatic) OR (all addresses,contains,smcusa) OR (all addresses,contains,precisionrings) OR (all addresses,contains,martin.andreas.abel) OR (all addresses,contains,martin.abel) OR (all addresses,contains,ferrotec) OR (all addresses,contains,lhdean) OR (all addresses,contains,glosser) OR (all addresses,contains,fuelscience)"
name="alibaba aliexpress"
enabled="yes"
type="17"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/alibaba"
action="Stop execution"
condition="OR (all addresses,contains,alibaba) OR (all addresses,contains,aliexpress) OR (all addresses,contains,made-in-china) OR (all addresses,contains,swcpu) OR (all addresses,contains,liliansh98) OR (all addresses,contains,extramoretech.com) OR (all addresses,contains,lu-lighting) OR (all addresses,contains,pcyangguangban) OR (all addresses,contains,hynnex) OR (all addresses,contains,163.com) OR (all addresses,contains,bala-tech) OR (all addresses,contains,zc-sensor) OR (all addresses,contains,lgvictory) OR (all addresses,contains,heatle) OR (all addresses,contains,baide-cn) OR (all addresses,contains,dazaocn) OR (all addresses,contains,chipego) OR (all addresses,contains,lindas)"
name="receipts"
enabled="yes"
type="48"
action="Move to folder"
actionValue="imap://[e-mail]/INBOX/reciepts"
action="Stop execution"
condition="OR (all addresses,contains,amazon) OR (subject,contains,your statement) OR (all addresses,contains,netflix) OR (all addresses,contains,newfrog) OR (all addresses,contains,antonline) OR (all addresses,contains,papajohns) OR (all addresses,contains,aa.com) OR (all addresses,contains,orbitz) OR (all addresses,contains,expansys) OR (all addresses,contains,extremerest) OR (all addresses,contains,customer.service) OR (all addresses,contains,canon.com) OR (all addresses,contains,coautilities) OR (all addresses,contains,nbmpub) OR (all addresses,contains,itunes) OR (all addresses,contains,directv) OR (all addresses,contains,tbtc) OR (all addresses,contains,vitacost) OR (all addresses,contains,dishnetwork) OR (all addresses,contains,sales) OR (all addresses,contains,bhphoto) OR (all addresses,contains,sprint) OR (all addresses,contains,htc) OR (all addresses,contains,myfax) OR (all addresses,contains,shipping) OR (all addresses,contains,orbitz) OR (all addresses,contains,ups quantum) OR (all addresses,contains,customerservice) OR (all addresses,contains,hertz) OR (all addresses,contains,performing arts) OR (all addresses,contains,att) OR (all addresses,contains,beachaudio) OR (all addresses,contains,dominos) OR (all addresses,contains,depot) OR (all addresses,contains,newegg) OR (all addresses,contains,phonepower) OR (all addresses,contains,buy.com) OR (all addresses,contains,dealextreme) OR (all addresses,contains,adultsextoy) OR (all addresses,contains,t-mobile) OR (all addresses,contains,maine) OR (all addresses,contains,buydig) OR (all addresses,contains,intuit) OR (all addresses,contains,goodvalue) OR (all addresses,contains,automationdirect) OR (all addresses,contains,onlinemetals) OR (all addresses,contains,mcmelect) OR (all addresses,contains,mcm-email) OR (all addresses,contains,receipts) OR (all addresses,contains,smarthome) OR (all addresses,contains,shipping) OR (all addresses,contains,hilton) OR (subject,contains,purchase) OR (all addresses,contains,twcable) OR (all addresses,contains,store.net) OR (all addresses,contains,timewarner) OR (all addresses,contains,dx.com) OR (all addresses,contains,kensington) OR (all addresses,contains,bbb.org) OR (all addresses,contains,freight) OR (all addresses,contains,statelinetack) OR (all addresses,contains,zorotools) OR (all addresses,contains,edsci.com) OR (all addresses,contains,lehmans.com) OR (all addresses,contains,snapfitness.com) OR (all addresses,contains,orders) OR (all addresses,contains,shipped) OR (all addresses,contains,fandango) OR (all addresses,contains,arbortech) OR (subject,contains,order) OR (all addresses,contains,aarp) OR (all addresses,contains,thinkgeek) OR (all addresses,contains,e.scien) OR (all addresses,contains,fedex) OR (subject,contains,shipping) OR (subject,contains,payment) OR (all addresses,contains,oemheaters) OR (all addresses,contains,tigerdirect) OR (all addresses,contains,sony)"

Comment 11

[Kent James (:rkent)]

OK I can confirm that a message loop can be created, and that it is new in Thunderbird 38. But it has nothing to do with Stop Execution, as that action is redundant after a Move filter. "Post-processing" is the internal term, "Filter after Junk Classification" is the UI decription.

STR:

Using IMAP, create two incoming/post-processing filter.

Filter #1: IF (subject contains "moveme") THEN move to folder "test21"
Filter #2: IF (subject contains "moveme") THEN move to folder "test22"

Sending a message that hits the filter will result in a move loop between folders test21 and test22.

I have not been as close to filters lately, and I did not realize the post-processing filters are applying to filters other than the inbox. In general, we do not want users to be able to create filter loops without taking extraordinary actions (such as using FiltaQuilla and its "apply incoming filters" option). It probably would not be difficult to locate where the post-processing filter is getting activated, and make sure it does not apply to non-inbox folders, but that might break other uses that relied on this behavior.

Comment 12

[Björn Victor]

Do you mean that filters "before junk classification" will not create loops, due to not being applied on non-inbox folders? That would be a work-around, at least.

Comment 13

[Kent James (:rkent)]

(In reply to Björn Victor from comment #12)
> Do you mean that filters "before junk classification" will not create loops,
> due to not being applied on non-inbox folders? That would be a work-around,
> at least.

That is my expectation, yes, but I have not tested this.

To use complex filters that might cause loops, you need some way of breaking the loop. One way is to use a search term that looks at the folder name. Unfortunately this term is not available in core, but only in my FiltaQuilla extension. (Also FiltaQuilla lets you specify which folders will be processed with "before junk" filters, so it is possible with that to create filter loops with "before junk" filters.)

Comment 14

[mrbi11]

So I gather from discussion, that filters are being applied to folders other than inbox on purpose, which is the bug.

Filters are a simple linear program, and cannot be run asynchronously at all. Incoming mail only ever arrives at inbox, so automatic filtering (as opposed to manual), should only occur there.

The before/after spam alters the order of processing, which could and should be reflected in the display order of the filters.
Display all the pre spam filters, then display all the post spam filters. A visual divider would be useful, that is two sub windows in the filter display.

Processing follows the display order then exactly, when new mail arrives, run all pre spam filters, run the spam filter, then run all the post spam filters.

You are having these bugs by perhaps trying to maintain the illusion that a post spam filter will run first because it appears befor a pre spam filter, but thats a logical impossibility. Stop trying to do the impossible, and do the possible.

A single pass through all the filters will fix all these problems as well as all the stop filter execution bugs.
Thanks for your hard work, as always.

Comment 15

[Björn Victor]

I agree that running filters on other than Inbox content seems like a bug (except when run explicitly on a particular folder), and fixing that behaviour would likely solve the problem.

Visualisation may also be important, but perhaps a separate issue.

(I have changed all my filters to "before junk classification" and have so far not had the filter loop behaviour.)

Comment 17

[Kent James (:rkent)]

Interesting that TB 31 does not have this issue, since post-processing filters were done much earlier. I was surprised that post-processing filters operated on non-inbox folders. So it does seem like we should figure out what caused the regression, and fix it. That would not be difficult, anybody feel like trying it?

Comment 18

[Wayne Mery (:wsmwk)]

We need your help to find the regression date - one person will do. :)

There is a well documented tool at http://mozilla.github.io/mozregression/documentation/usage.html to help you.

Comment 19

[u552782]

I tried to to this, but i have got the following error:

platform: Windows-7-6.1.7601-SP1
python: 2.7.11 FROZEN (32bit)
mozregui: 0.9.1
mozregression: 2.3.1
message: KeyError: 'build_url'
traceback:   File ".\mozregui\bisection.py", line 59, in bisect
  File "..\mozregression\bisector.py", line 548, in bisect
  File ".\mozregui\bisection.py", line 98, in _bisect
  File ".\mozregui\bisection.py", line 132, in _bisect_next
  File "..\mozregression\bisector.py", line 347, in init_handler
  File "..\mozregression\bisector.py", line 146, in initialize
  File "..\mozregression\bisector.py", line 78, in initialize
  File "..\mozregression\build_range.py", line 94, in __getitem__
  File "..\mozregression\build_range.py", line 41, in build_info
  File "..\mozregression\build_range.py", line 35, in _fetch
  File "..\mozregression\fetch_build_info.py", line 267, in find_build_info

I have installed mozregression-gui (on Windows 7) and after that Python 2.7.11.
Do I need to install mozregression too? Unfortunately it doesn't work. I'm not an expert...

Comment 20

[Wayne Mery (:wsmwk)]

Perhaps alice help with comment 19

Comment 21

[Alice0775 White]

Sorry, I never used neither mozregression and mozregression-gui.

You can download common-central nightly and tinderbox from the following links.

http://archive.mozilla.org/pub/thunderbird/nightly/
http://archive.mozilla.org/pub/thunderbird/tinderbox-builds/comm-central-win32/

Comment 22

[u552782]

I have tested it manually:
The bug would have occurred since the version 37.0a2.
I have testet this version - this doesn't have this bug:
http://archive.mozilla.org/pub/thunderbird/nightly/2015/01/2015-01-12-00-40-01-comm-aurora/thunderbird-36.0a2.en-US.win32.installer.exe
And I have testet this version - this one have this bug:
http://archive.mozilla.org/pub/thunderbird/nightly/2015/01/2015-01-13-00-40-08-comm-aurora/thunderbird-37.0a2.en-US.win32.installer.exe


I hope this was helpful.

Comment 24

[WADA:World Anti-bad-Duping Agency]

(In reply to Kent James (mostly AFK 2016-11-19 to 2016-11-26) (:rkent) from comment #11)
> OK I can confirm that a message loop can be created, and that it is new in
> Thunderbird 38. But it has nothing to do with Stop Execution, as that action
> is redundant after a Move filter. "Post-processing" is the internal term,
> "Filter after Junk Classification" is the UI description.
> 
> STR:
> Using IMAP, create two incoming/post-processing filter.
> Filter #1: IF (subject contains "moveme") THEN move to folder "test21"
> Filter #2: IF (subject contains "moveme") THEN move to folder "test22"

Move loop by following is not IMAP only problem, although it's not automatic loop if local mail folder.
> Filter #1: IF (subject contains "moveme") THEN move to folder "test21"
> Filter #2: IF (subject contains "moveme") THEN move to folder "test22"
It's old bug 393668, and it still occurred in Thunderbird 45.5.0 on Win.
(1) If mail is held in test21, the mail is moved to test22 by "Run filter on folder test21".
(2) If mail is held in test22, the mail is moved to test21 by "Run filter on folder test22".

(2) is normal, because Filter-#1 is applied and mail is moved to test21 and filter stops because of "Move to folder".
(1) is problem.
Filter #1 hits, then Tb tries to move the mail, but the mail is already held in test21,
then Thunderbird does do nothing even though action is "Move to folder", and doesn't stop here.
After it, because the mail is still held in test21, Filter #2 is applied and the mail is moved to test22.

IIUC, "Post-processing" is identical to "Run filter on folder".
But I don't know bug 393668 is "Run filter on folder" type only issue or not.

If (a) user sets folder property of "Include this folder when new mail check" in all mbox under Inbox,
and if (b) user enables hidden "apply incoming filters" option,
and if (c) bug 393668 happens at same time,
automatic message loop can occur.
However, it doesn't look user did (a) nor (b).
 
Problem in IMAP is:
It looks that "new mail check" is always executed when mbox under Inbox(IIRC, I saw bug for this).
It looks that "filter on new mail" is always applied to mbox under Inbox.
It may be Namespace=INBOX/(or INBOX.) case only phenomenon.

To Kent James, does it occur?

Other concerns.

(i) (i-a) Filter before Junk Classification and (i-b) Filter after Junk Classification.
IIRC, filter on new mail was (i-a) only, and (i-b) was added, and (i-a)/(i-b) was defined as "per filter rule attribute" instead of "account wide(per msgFilterRules.dat file) attribute". So, both can be mixed in a msgFilterRules.dat.
This change may be relevant.
(according to bug opener, it looks that problem doesn't occur with "Filter before Junk Classification".)
(according to your comment, it was done before Tb 31.)

(ii) (body,contains,craigslist)
If Body condition is used, Thunderbird issues "uid fetch a:b,...,y:z body.peek[]" for all new mails after header fetch of all new mails, because entire mail data is required for Body condition.
And, when "Before Junk Classification", Tb applies message filter & Junk filter on a mail when fetch of a mail is completed.
When "After Junk Classification", Tb applies Junk filter after fetch of entire mail data of all new mails, then applies filter. 
This may be relevant, because timing of filter application may be different from ordinal case.

Sorry but I don't know above changes were done in which Thunderbird release.

Problem looks to occur when all three conditions are satisfied.
- new mails of all mbox under Inbox are fetched by new mail check
- "filter on new mail" is applied to all mbox under Inbox
- bug 393668 happens at same time
And root cause of "Loop" is bug 393668.
So, avoiding bug 393668 is simplest workaround of "Loop".

Simple workaround, although modification is tough work in many cases, because "current conditions" is OR conditions in many cases.
(A) If mail is not marked as Starred && current conditions,
    mark Starred, then move to folder. Starred can be tag etc.
    => condition won't hit with filter moved mail, so bug 393668 can't occur.
Best way :
(B) Filter rules definition of "any mail hits only one condition for move" => bug 393668 can't occur.

Problem of this bug was reported to a forum Japan, and user said he will try (A) "Starred method", but no response after it.

Comment 25

[WADA:World Anti-bad-Duping Agency]

(In reply to Kanzler from comment #22)
> I have tested it manually:
> The bug would have occurred since the version 37.0a2.
> I have testet this version - this doesn't have this bug:
> http://archive.mozilla.org/pub/thunderbird/nightly/2015/01/2015-01-12-00-40-
> 01-comm-aurora/thunderbird-36.0a2.en-US.win32.installer.exe
> And I have testet this version - this one have this bug:
> http://archive.mozilla.org/pub/thunderbird/nightly/2015/01/2015-01-13-00-40-
> 08-comm-aurora/thunderbird-37.0a2.en-US.win32.installer.exe
> 
> I hope this was helpful.

Thanks. Without regression window discovery, no one can usually find change produced problem.

Pushlog of comm-aurora between 2015-01-10 00:00 and 2015-01-13 23:59
> https://hg.mozilla.org/releases/comm-aurora/pushloghtml?startdate=2015-01-10+00%3A00&enddate=2015-01-13+23%3A59
Because of comm-aurora, patch application is bulk mode, so single big change only.
Bug 1116561 is a change relevant to "filter after the fact".
"filter after the fact" is not limited to Inbox only upon new mail check after some change?

Comment 26

[Wayne Mery (:wsmwk)]

To narrow the regression even further, please test a few comm-central nightlies
https://archive.mozilla.org/pub/thunderbird/nightly/2015/01/2015-01-02-03-02-00-comm-central/
https://archive.mozilla.org/pub/thunderbird/nightly/2015/01/2015-01-01-03-02-00-comm-central/
https://archive.mozilla.org/pub/thunderbird/nightly/2014/12/2014-12-31-03-02-00-comm-central/
https://archive.mozilla.org/pub/thunderbird/nightly/2014/12/2014-12-30-03-02-00-comm-central/
https://archive.mozilla.org/pub/thunderbird/nightly/2014/12/2014-12-29-03-02-01-comm-central/

Comment 27

[WADA:World Anti-bad-Duping Agency]

Sorry, it was not bug 393668. It was bug 232569.
1. bug 232569 occurred => bug 232569 disappered and phenomenon was changed to bug 393668. So I closed bug 232569 as WORKSFOME.
2. But bug 232569 is observed in Thunderbird 45.5.0.
It looks for me:
  problem of bug 232569 was masked by bug 393668 once, but bug 393668 was corrected, then problem of bug 232569 was exposed again.
I've reopened bug 232569.

Comment 28

[WADA:World Anti-bad-Duping Agency]

About next in bug opener's environment:
  New mails are fetched at mbox under Inbox, even though "include this folder when new mail check" is not set.

To bug opener.

What is your "Allow immediate server notification when new messages arrive" setting of Server Settings?
("Use IDLE command" in Server Settings/Advanced for long time since initial of IDLE support, and moved to Server Settings and term of IDLE was removed.)

If server supports IDLE command, and if this option is checked(==Tb uses IDLE command), new mails in selected mbox at a cached connection is notified from imap server to Thunderbird via. IDLE.
This is identical to "new mail check at the mobox" for Thunderbird.
And, if "move to folderA" and "move to folderB" is invoked in message filter, the folderA and folderB is usually selected at a cached connection. 

Do you see your problem with the option Unchecked, with "Filter After Junk Classification"?

Comment 29

[mrbi11]

Honestly I deleted almost all filters as they cause infinite loops of movement.
So my ability to answer is limited.
I used both before/after junk classification flags. I had accumulated about 50 filters
to sort incoming mail before the infinite movement bug showed up. I tried at first to disable
the filters in the folders i saw movement but would later find a new email causing different loop and
as I cannot monitor my email 24/7 eventually disabled all the move to folder/stop filter execution rules.

I think I have mentioned it is a solution to this bug to ONLY filter new messages coming into the inbox.
While apparently some people want to filter "incoming" messages to other folders, it would clearly
fix this bug to allow that choice to be either part of the message filter or the higher level dialog.

What folders should be filtered? 
  Inbox
  Inbox/FolderX
  FolderY
etc, and once could specifically consider only messages to Inbox, not Inbox subfolders or sibling folders.

My provider only allows subfolders of inbox to be created for example.

I would encourage though be given to people with multiple Thunderbird email programs running from separate
computers, such as home, laptop and office. 
I beleive the concept is called idempotent, meaning a set of rules that can be applied many times or just once,
but will always have the same outcome.

Allowing the user to choose to filter only the folder(s) they know they want filtered would be a method
of allowing a stateless description of what should happen.

You already have a partial implementation of that, as rules dont apply to all accounts, just one.

I hope my comments are helpful, and only loosely disguise my total ignorance of answering anything related to "IDLE" :-)

A universal of all control systems is that they be controllable and observable, and the same applies
to human interfaces. People want to be able to say what they mean. In My case, a pretty common case,
I only every want to move folders from "Inbox" to somewhere else. 
Once moved I dont want any filters moving it again, or even looking at it again.

The state of the communications between thunderbird and server are in my view not the question.
The Question is can I filter only new mail to the "Inbox" folder. That solves the problem,
...perhaps add a checkbox for this filter applies only to "Inbox", or NOT only to "inbox" or a more general way to list the folders to apply it to, although Id never use that. Make the road most traveled the default, and allow tricky stuff by the more tricky description.

2 cents worth, completely free:-)

Comment 30

[WADA:World Anti-bad-Duping Agency]

(In reply to mrbi11 from comment #29)
> Honestly I deleted almost all filters as they cause infinite loops of movement.

Why you need to delete almost all filters?
You said that problem didn't occur if "Filter before Junk Classification" was used, didn't you?
Because you have many filters, it's perhaps tough work but change to "before" is better than delete filters.
And, there is a way of "Edit msgFilterRuls.dat by Text Editor and restart Thunderbird".
  remove action="Stop execution" line, modify type="nn" value(change bit for before/after)
Further, there is check box for enable/disable on each filter rule row in Message Filter panel.
So, you can disable almost all filters, instead of delete almost all filters.

Because infinite loop, if filter log is enabled, filter log file becomes huge. Please stop fillter logging and clear it.

> My provider only allows subfolders of inbox to be created for example.

It's because server uses Personal Namespace=INBOX/(or INBOX.). See Server Settings/Advanced.
However, Namespace was not relevant. As known by comment #11 by Kent James, problem occurs even when top level folder.

Conditions were:
(1) New mail check is enabled.
(2) IDLE use is irrelevant
(3) "Filter after Junk Classification" is used(checked with both is "after" as done by Kent James)
(4) Phenomenon of bug 232569 occurs because a mail can hits multiple filter rules which has "Move to folder" action.
    For example,
    Mail : From=abc@x.y.z, Subject=pqr
    Filter#1 : If From begins with abc,    move to FolderA 
    Filter#2 : If Subject begins with pqr, move to FolderB 
(5) Folder Property, "When getting ..., always check this folder" is irrelevant,
    if "Filter after Junk Classification" is used.
(6) Following is irrelevant, if "Filter after Junk Classification" is used.
    mail.server.server#.applyIncomingFilters = true/false (usually this setting doesn't exist)
    mail.server.default.applyIncomingFilters = true/false (usually this setting doesn't exist)

As Kent James says in comment #11, phenomenon of "endless loop of message move between FolderA and FolderB" is reproducible and easily observed on Thunderbird 45.

> I used both before/after junk classification flags.

If so, in above filter definition example, following can occur.
(i)   Filter#1 = before, Filter#2 = before => "Loop" didn't occur
(ii)  Filter#1 = before, Filter#2 = after  => I don't know
(iii) Filter#1 = after,  Filter#2 = before => I don't know
(iv)  Filter#1 = after,  Filter#2 = after  => As Kent James said in comment #11, problem is reproduced.

Mechanism is as follows:
(1) After "Before" message filter application and Junk filter application,
    a new mail-X in Inbox is moved to FolderA by Filter#1(After).
If "Filter after Junk Classification", following occurs.
(2) Filter(After) is applied to the FolderA. <= this is newly known problem
(3) mail-X kicks Filter#1, but because here is FolderA, bug 232569 occurs, <= this is old bug
    then Filter#2 fires and mail-X is moved to FolderB
(4) Filter(After) is applied to the FolderB. <= this is newly known problem
(5) mail-X kicks Filter#1 because defined first, and mail-X is moved to FolderA.
(6) Go to (2) => Endless loop

(2)/(4) may be following, although I'm not sure.
After application of "Filter before Junk Classification", Junk filter is invoked. In this step, Junk filter is applied to mails moved from Inbox to other FolderX.
"Filter after Junk Classification" is always invoked at the FolderX(folder where new mail is moved by message filter) after above Junk filter application.
Or it may be following.
 "Junk Classification" in "Filter before Junk Classification" ==
    Junk filter application + application of filters of "Filter after Junk Classification"
i.e.
Filters of "Filter after Junk Classification" is considered as a part of Junk filter by Thunderbird.

Simplest/easiest workaround currently available:
  Don't use "Filter after Junk Classification",
  Use "Filter before Junk Classification".

Comment 31

[WADA:World Anti-bad-Duping Agency]

(In addition to my comment #30)
> > My provider only allows subfolders of inbox to be created for example.
> It's because server uses Personal Namespace=INBOX/(or INBOX.). See Server Settings/Advanced.
> However, Namespace was not relevant. As known by comment #11 by Kent James,
> problem occurs even when top level folder.

Sorry but "Namespace was not relevant" was not accurate.
When Namespace="INBOX/"(or "INBOX."), it's worse than usual, because all mbox is held under Inbox.

(Case-1 : move target folder = top level folder)
  Mail-X : From=abc@x.y.z, Subject=pqr
    FilterRule#X1(After) : If From begins with abc,    move to FolderA 
    FilterRule#X2(After) : If Subject begins with pqr, move to FolderB 
(Case-2 : move target folder = folder under Inbox)
  Mail-Y : From=def@x.y.z, Subject=stu
    FilterRule#Y1(After) : If From begins with def,    move to Inbox/FolderC 
    FilterRule#Y2(After) : If Subject begins with stu, move to Inbox/FolderD 

(0) New mail check is enabled. IDLE command use is disabled.
(1) Mail-X and Mail-Y arrive at Inbox
(2) Following problem occurs due to this bug.
(2-1) Mail-X : Endless loop of move between FolderA and FolderB 
(2-2) Mail-Y : Endless loop of move between Inbox/FolderC and Inbox/FolderD 
(3) Go Work Offline => Endless loop stops
(4) Go Work Online, without doing any other action than Go Work Offline/Go Work Online.
(4-1) Mail-X / FolderA / FolderB : Nothing happens. 
(4-2) Mail-Y : Endless loop of move between Inbox/FolderC and Inbox/FolderD 

(4-2) indicates that "New mail check" is executed on Inbox/FolderC and Inbox/FolderD, regardless of Folder Property/General/"When getting ..., always check this folder" setting of Inbox/FolderC and Inbox/FolderD.
(i.e. My remeber was correct.)

To bug opener, please stop using "Filter after Junk Classification" and use "Filter before Junk Classification" if you want to use message filter of Thunderbird.

I can understand that this bug is intolerable for user especially when all folders are held under Inbox.
I tested with "move target folder = Inbox/FolderA and Inbox/FolderB" first using Gmail IMAP, because I suspected "new mail check of folder under Inbox".
And, I tried to stop the "Endless loop" then I did "Go Work Offline" in order to kill connection with server.
But, when I did "Go Work Online" to do next test or other job, the "Endless loop" automatically/silently started again.
Why? What can I do when such mysterious phenomenon happened?
Because of Gmail IMAP, it's merely excess Gmail's server CPU consumption and excess network-bandwidth consumption, but it'll consume all HDD space of my PC because I enable filter logging of Thunderbird.  
If I were you, killed Thunderbird and forced Thunderbird out from my valuable PC :-)

But, comment of complaint only won't help resolving bug by developer.
Many conditions are relevant to this bug, but conditions are:
 if( "After" filter && bug 232569 occurs due to characteristics of filter definition && ... && ... )
    this bug occurs;
So, if a condition is changed to false, this bug doesn't occur.
Do available workaround before adding comment of complaint to this bug, please.

Comment 32

[WADA:World Anti-bad-Duping Agency]

Oh, this bug was opened on 2015-10-18 for Tb 38 instead of opened on 2016-10-18 for Tb 45.
I thought this bug was new issue in Thunderbird 45, because I saw a report for phenomenon of this bug at a forum
in this autum, and because I thought "2016-10-18".
i.e. Bug opener had been sufferred from this bug for more than one year....

Comment 33

[WADA:World Anti-bad-Duping Agency]

Following comment #11 was posted on 2015-12-10,
by "Kent James (mostly AFK 2016-11-19 to 2016-11-26) (:rkent)"  
> OK I can confirm that a message loop can be created, (snip)

I thought it was posted in 2016, because two "2016" are seen in Kent James's name part :-)

Kent James, what kind of log data is needed for problem analysis?
NSPR imap log? NSPR log with xxxx:5?
If some trap code is needed in JavaScript code, I can write JavaScript code to insert such logic in Filter related object.
  Backup FilterObject.A_JavaScriptFunction in global variable or singleton object,
  and replace it by modified function code.
  After end of test, restore it from backup.
This can be relatively easily done by utilizing button script of button created by "Custom Buttons" addon.

Comment 34

[WADA:World Anti-bad-Duping Agency]

FYI.

New regression window of bug 232569 was following.
> Problem doesn't occur:
> https://archive.mozilla.org/pub/thunderbird/nightly/2014/12/2014-12-20-03-02-03-comm-central/
> Problem occurs:
> https://archive.mozilla.org/pub/thunderbird/nightly/2014/12/2014-12-21-03-02-07-comm-central/
I've opened bug 1320676 for the new regression which still remains in Thunderbird 45.

Comment 35

[WADA:World Anti-bad-Duping Agency]

FYI.

Two problems are involved in this bug.
(a) "Filter after Junk Classification" on new message is executed on "filter moved new message",
    as done by Junk Filter.
(b) Bug 232569 came back in Thunderbird 37(Bug 1320676)

I think (a) is behavior since initial of "after Junk Classification", and it was exposed by Bug 1320676.
If (b) doesn't occur, nothing usually happens by (a) on "filter moved new message" at "filter move target folder".

(a) may be a cause of "phantom mail".
1. New MailX is moved from Inbox to FolderX by message filter.
2. At FolderX, Junk Filter runs, because Junk filter is applied on "filter moved new message".
3. "After Junk Classification" on new message also invoked at FolderX by (a).
4. When action of "move to FolderX" is fired by message filter,
   even if nothing is done by DoCopy because MailX is already held in FilderX,
   contention can occur between 3 and 2 because both action is on same message in same folder.
   And, if Junk move happens on the filter moved new message, Bug 1110583 may occur.

Comment 36

[mrbi11]

Hi,
I very much appreciate developers work on Thunderbird.
I posted answer because one of the developers asked me a question.
Not trying to rabble rouse, but I was once a decent programmer, and I dont see
in the discussion clear awareness that the bug has components from design as well as
from interaction with server, specifically one cannot add a test for which folder the message
is currently in. One cannot say do this if the bug is in folder X, where X is likely always Inbox.
Also my email provider's requirement that all folders be under inbox makes it more complex to test that.
I see there is discussion and awareness of this wrinkle now.
I have been advised to always filter before junk, and always filter after :-)...
But until the bug has definitive fix, my workaround is to avoid using filters. 
So please take your time and design a nice bullet proof fix for the general case.
And be very certain I appreciate your efforts.

Comment 37

[WADA:World Anti-bad-Duping Agency]

(In reply to mrbi11 from comment #36)
> I have been advised to always filter before junk, and always filter after :-)...

To bypass a bug, (a) use "Filter after Junk Classification" was needed in some cases, but (b) use "Filter before Junk Classification" was needed in other cases.
If you don't have much severe problem than this bug due to other bug(s) which requires bypass of (a),
and if you need to bypass problem of this bug,
please do (b) as I already wrote sometimes in this bug,
because this bug is a member of bugs which needs (b) to bypass problem.

> But until the bug has definitive fix, my workaround is to avoid using filters.

As you already know "when Filter before Junk Classification, loop problem didn't occur".
and as I already wrote sometimes in this bug,
"use Filter before Junk Classification instead of Filter after Junk Classification" is a workaround of problem of this bug.
And, you are free to stop using message filter of Thunderbird, and you are free to use "Filter before Junk classification".

I'm not interested in what workaround you choose or you do nothing to workaround.
All is up to you.
Here is Bugzilla.Mozilla.Org for developers to resolve bugs. Here is not support forum or Customer Help center.

Comment 38

[WADA:World Anti-bad-Duping Agency]

(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #26)
> To narrow the regression even further, please test a few comm-central　nightlies

Regression window was between 2014-12-20 and 2014-12-21.
So, you can get consolation prize only -:)

Comment 39

[Wayne Mery (:wsmwk)]

(In reply to WADA from comment #34)
> FYI.
> 
> New regression window of bug 232569 was following.
> > Problem doesn't occur:
> > https://archive.mozilla.org/pub/thunderbird/nightly/2014/12/2014-12-20-03-02-03-comm-central/
> > Problem occurs:
> > https://archive.mozilla.org/pub/thunderbird/nightly/2014/12/2014-12-21-03-02-07-comm-central/
> I've opened bug 1320676 for the new regression which still remains in
> Thunderbird 45.

So no further regression range hunting needed? 
(you didn't mark bug 1320676 with "regression")

Apologies - my earlier request for nightly range should have been addressed to kanzler ...
And big kudos to kanzer for getting the aurora range. Deserves more than a consolation prize IMO.

Comment 40

[WADA:World Anti-bad-Duping Agency]

(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #39)
> So no further regression range hunting needed? 

"Loop" won't occur in this bug's case, unless Bug 232569(currently Bug 1320676) occurs.
However, other causes(or relevant components) of problem of this bug, for example, "Filter after Junk Classification is executed at folder where new mail is moved by message filter", has different regression window.
Although detectipon of such thing is far harder than "unwanted move mails in Bug 232569/Bug 1320676" and "Loop in this bug",
I believe regression window of such things is better provided by someone for developers.

> (you didn't mark bug 1320676 with "regression")

Do it silently if you care about it. I don't mind such field. Regression or not, bug is bug :-)

Comment 41

[WADA:World Anti-bad-Duping Agency]

FYI.

I saw following mysterious phenomenon with Gmail IMAP.
(0) Gmail IMAP. Auto-expunge=enabled.
    Max cached connections=5. IDLE command use is disabled.
    IMAP delete model=Just mark it as deleted.
    New mail check is disabled. Auto-Sync=Off. Gloda=disabled.
    "When new mail check ..., include this folder" is off.
    Hidden ...applyIncomingFilters=false or is Reset.
(1) Filter: ("Filter after Junk Classification" only)
    FilterRule#1(After) : if subject begins with xxx, move to Folder#1
    FilterRule#2(After) : if subject begins with xxx, move to Folder#2
(2) mails in Folder#1 :
    Mail#1: Subject=xxxxx.... (no \Seen flag==Unread)
    Mail#2: Subject=xxxxx.... (no \Seen flag==Unread)
    (Because these mails are used several times,      )
    (original copy is already held in [Gmail]/All Mail)
(3) Restart Thunderbird 45.
(4) Click Folder#1 at Folder Pane(open Folder#1. "uid fetch 1:* Flags" is issued)
    => "message move loop between Folder#1 and Folder#2" starts.

It's merely ordinal IMAP folder open when "no \Seen flag==Unread" mails are contained in an IMAP folder.
Why "Filter after Junk Classification" is invoked at Folder#1/Folder#2?
Something is wrong in my Thunderbird settings?

When Gmail IMAP(move command, UIDPLUS etc. is supported), same UID may be returned after :
  "uid move ### Folder#2" at Folder#1 + "uid move ??? Folder#1" at Folder#2
  + "uid fetch $$$:* Flags" at Folder#1
So, above phenomenon may not occur if ordinal IMAP server.
However, such phenomenon with Gmail IMAP is usually indicator of something wrong in Thunderbird.

Comment 42

[WADA:World Anti-bad-Duping Agency]

(In reply to mrbi11 from comment #36)
> I have been advised to always filter before junk, and always filter after :-)...

Were you recommended "filter after junk" by someone at somewhere when you saw phenomenon such as Bug 931303?

Comment 43

[mrbi11]

Sorry I dont recall. Its been a while.
If the feature cannot be reliably implemented perhaps it is best to remove it till it is reliable?

Comment 44

[WADA:World Anti-bad-Duping Agency]

(In reply to mrbi11 from comment #43)

I'm one of peoples who said following in some bugs.
(i)  Use "Filter after Junk Classification", in bug(s) which was caused by "Filter before Junk Classification".
(ii) Use "Filter before Junk Classification", in bug(s) which was caused by "Filter after Junk Classification".
Which should we do when both problems exist? :-)
So you did both? :-)

IIRC and IIUC, current biggest problen in message filter for user of "Filter after Junk Classification" is this bug only,
and IIRC and IIUC, crical problem with "Filter before Junk Classification" currently doesn't exist.
I can understand that this bug is intolerable for you, but I can say (ii) only in this bug.
Please do one of known workarounds and wait for bug analysis/resolving of this bug by developer(s).

Comment 45

[WADA:World Anti-bad-Duping Agency]

Sorry for typo, forgot to use Spell Check : problen => problem, crical => critical

Comment 46

[WADA:World Anti-bad-Duping Agency]

Message filter was "Apply selected filter rules on selected messages".

Following may be reason why After filter is applied to "filter moved new message".
(0) MsgDBHdr of new mails in Inbox are saved in msgDBHdrList.
(1) Enabled Before filter rules are extracted to FilterListBefore,
    then FilterListBefore is applied to msgDBHdrList.
    And msgDBHdr of "filter moved message in move target" is stored in msgDBHdrList.
    This is to support "Junk filter application on filter moved message",
(2) Junk filter is applied to the msgDBHdrList.
    "Junk moved message to Junk folder" is removed from msgDBHdrList.
(3) Enabled After filter rules are extracted to FilterListAfter,
    then FilterListAfter is applied to same msgDBHdrList.
(3-1) MailX in Inbox is moved to Folder#1 by FilterRule#1
      and msgDBHdr of MailX in Inbox is removed from msgDBHdrList,
      (*) and msgDHdr of MailX in Folder#1 is stored in msgDBHdrList,
          as done by "Filter before Junk Classification".
(3-2) Due to (*), After filter is applied to the moved MailX in Folder#1,
      and due to Bug 1320676(Bug 232569 came back),
      MailX in Folder#1 is moved to Folder#2 by FilterRule#2,
      and msgDBHdr of MailX in Folder#1 is removed from msgDBHdrList,
      (*) and msgDBHdr of MailX in Folder#2 is stored in msgDBHdrList,
          as done by "Filter before Junk Classification".
(3-3) Due to (*), After filter is applied to the moved MailX in Folder#2,
      and MailX in Folder#2 is normally moved to Folder#1 by FilterRule#1,
      and msgDBHdr of MailX in Folder#2 is removed from msgDBHdrList,
      (*) and msgDHdr of MailX in Folder#1 is stored in msgDBHdrList,
          as done by "Filter before Junk Classification".
(3-4) Due to (*), go to step (3-2) => Infinite loop

Kent James, is my guess wrong? Or partially right?

Comment 47

[WADA:World Anti-bad-Duping Agency]

My guess of comment #46 can't explain following phenomenon(tested with Gmail IMAP).
(0) New mail check is disabeled both startup and every NN minutes. IDLE command use=Disabled.
    Auto-sync is not effective(all folders is Offline-use=Off folder), Gloda=Disabled 
(1) Two rules of "Filter After Junk Classification"
    FilterRule#1: if subject begins with abc, move to Folder#1
    FilterRule#2: if subject begins with abc, move to Folder#2
    Folder#1, Folder#2 is empty, or some mails are held, and Thunderbird knows it
    FolderX/S1/S2/S3 is empty at Thunderbird or some mails are held. Folder is not opened at Thunderbird
(2) New mail(unread mail) is copied or moved to FolderX/S1/S2/S3 by other client. (SeaMonkey is used)
    This is emulation of "Filter move at server".
    Mail in FolderX/S1/S2/S3 : Subject = abcXXXXXX,  change to Unread(no \Seen flag)
(3) At Thunderbird, click FolderX/S1/S2/S3 at folder pane
    => Endless loop of "move between Folder#1 <-> Folder#2" starts.

This indicates that "any mail of new UID && no \Seen flag in any folder of IMAP" is "New message to be filtered upon getting new mail" for "Getting New Mail : Filter after Junk Classification".

Comment 48

[WADA:World Anti-bad-Duping Agency]

Problem of following in my previous comment was Bug 770137 which was opened on 2012-07-02 for Thunderbird 13.
 Filter after Junk Classification is invoked on any new mail detected by any process at any imap folder.

Comment 49

[WADA:World Anti-bad-Duping Agency]

This bug is produced by combination of (a) Bug 770137 and (b) Bug 1320676.
Regression window of Bug 1320676 is already found.
Currently known regression window of Bug 770137 is merely "between Tb 12 and Tb 13".

To Wayne Mery: Do you want regression window of Bug 770137 in trunk nightly builds?

Comment 50

[WADA:World Anti-bad-Duping Agency]

FYI.

Same loop as this bug is easily reproduced by following pretty simple test.
i.e. Infinite loop can occur with bug 770137 only. No need of Bug 1320676.
IIUC. this kind of loop is the reason why mail.server.default.applyIncomingFilters is hidden prefs setting in Config Editor(not held in mailnews.js).

imap accountA  : Getting New Mail: Filter after Junk Classification
  FilterRule#1 : if subject contains ����, copy(or move) to imap accoutB/FFFDmail
imap accountB  : Getting New Mail: Filter after Junk Classification
  FilterRule#1 : if subject contains ����, copy(or move) to imap accoutA/FFFDmail
This may be funny definition, but is never abnormal definition.
  If such junk mail newly arrived, notify to another imap account by copying the mail.

"Copy mail to folder of different imap account" is "fetch body[] + select FolderX + append", and in order to know UID of the copied mail, "uid fetch HighestUID:* Flags" is needed. This is same as new mail check, folder open.
It may be different if UIDPLUS(APPENDUID,COPYUID) is supported, because UID of copied mail is known by response to append/copy.

Comment 51

[WADA:World Anti-bad-Duping Agency]

It looks for me that "Filter after Junk Classification" is blindly applied to any new mail in any folder.
If so, code like following may be a solution.

// For filter rules of Getting New Mail:Filter after Junk Classification
When new mail is detected at a folder, regardless of "how new mail is detected",
if(Inbox) Apply=true;
else if(Special folder such as trash, junk, sent, draft, templates, allmail) Apply=false;
else if(local mail folder of POP3/Local Folders) Apply=false;
else if(imap && true=="include in new mail check" && true==applyIncomingFilters) Apply=true;
else Apply=false;
if(!true==Apply) return;
Do application of "Filter after Junk Classification" to newly detected mails in the folder.

I think regulations in "Getting New Mail:Filter before Junk Classification" on new mail via. IDLE or folder open is similar to above, or it's executed upon "explicit new mail check by Server Settings" only.

Comment 52

[WADA:World Anti-bad-Duping Agency]

"Getting New Mail:Filter after Junk Classification" seems a sub part of "Junk Filtering" which is inserted by "Getting New Mail:Filter after Junk Classification" support.
  Junk Classification -> in passing, before go to Junk move, do "Filter after" -> Junk move

(1) When new mail check, after "Filter before Junk Classification", following looks executed for Junk filtering.
> https://dxr.mozilla.org/comm-central/source/mailnews/base/util/nsMsgDBFolder.cpp#2495
> nsMsgDBFolder::CallFilterPlugins(nsIMsgWindow *aMsgWindow, bool *aFiltersRun)
> {
>   Do Junk Classification jobs.
>   When Junk Classification completed,
>   if (filterPostPlugin)  <= filter after junk classification. "Junk filter" is called "plugin"
>     OnMessageClassified(nullptr, nsIJunkMailPlugin::UNCLASSIFIED, 0);
> }

(2) OnMessageClassified is called for filterPostPlugin.
> https://dxr.mozilla.org/comm-central/source/mailnews/base/util/nsMsgDBFolder.cpp#2322
> nsMsgDBFolder::OnMessageClassified(const char *aMsgURI,nsMsgJunkStatus aClassification,uint32_t aJunkPercent)
> {
>   // Call ApplyFilters() for Filter after Junk Classification
>   rv = filterService->ApplyFilters(nsMsgFilterType::PostPlugin,mPostBayesMessagesToFilter,
>   // Notify for final Junk Move process
>   notifier->NotifyMsgsClassified(classifiedMsgHdrs,mBayesJunkClassifying,mBayesTraitClassifying);                                        this, nullptr, nullptr);
> }

(3) ApplyFilters invokes filterExecutor named nsMsgApplyFiltersToMessages.
> https://dxr.mozilla.org/comm-central/source/mailnews/base/search/src/nsMsgFilterService.cpp#1097
> NS_IMETHODIMP nsMsgFilterService::ApplyFilters(nsMsgFilterTypeType aFilterType,
>   nsIArray *aMsgHdrList,nsIMsgFolder *aFolder,nsIMsgWindow *aMsgWindow,nsIMsgOperationListener *aCallback)
> {
>   // Create our nsMsgApplyFiltersToMessages object which will be called when ApplyFiltersToHdr
>   // finds one or more filters that hit.
>   RefPtr<nsMsgApplyFiltersToMessages> filterExecutor =
>     new nsMsgApplyFiltersToMessages(aMsgWindow, filterList, folderList,
>                                     aMsgHdrList, aFilterType, aCallback);
>   if (filterExecutor)
>     return filterExecutor->AdvanceToNextFolder();
> }

(4) nsMsgFilterAfterTheFact     : for "Run filters on folder"  of Tools menu
    nsMsgApplyFiltersToMessages : for "Run filters on message" of Tools menu
    nsMsgApplyFiltersToMessages is perhaps also used by "Run selected filters on: / Run Now" of Message Filter.
> https://dxr.mozilla.org/comm-central/source/mailnews/base/search/src/nsMsgFilterService.cpp#276
> // this class holds the list of filters and folders, and applies them in turn, first iterating
> // over all the filters on one folder, and then advancing to the next folder and repeating.
> // For each filter,we take the filter criteria and create a search term list. Then, we execute the search.
> // We are a search listener so that we can build up the list of search hits.
> // Then, when the search is done, we will apply the filter action(s) en-masse, so, for example, if the action is a move,
> // we calls one method to move all the messages to the destination folder. Or, mark all the messages read.
> // In the case of imap operations, or imap/local  moves, the action will be asynchronous, so we'll need to be a url listener
> // as well, and kick off the next filter when the action completes.
> class nsMsgFilterAfterTheFact : public nsIUrlListener, public nsIMsgSearchNotify, public nsIMsgCopyServiceListener
> {
> public:
>   nsMsgFilterAfterTheFact(nsIMsgWindow *aMsgWindow,
>                           nsIMsgFilterList *aFilterList, nsIArray *aFolderList,
>                           nsIMsgOperationListener *aCallback);
> 
> https://dxr.mozilla.org/comm-central/source/mailnews/base/search/src/nsMsgFilterService.cpp#990
> // nsMsgApplyFiltersToMessages overrides nsMsgFilterAfterTheFact in order to
> // apply filters to a list of messages, rather than an entire folder
> class nsMsgApplyFiltersToMessages : public nsMsgFilterAfterTheFact
> {
> public:
>   nsMsgApplyFiltersToMessages(nsIMsgWindow *aMsgWindow,
>                               nsIMsgFilterList *aFilterList,
>                               nsIArray *aFolderList, nsIArray *aMsgHdrList,
>                               nsMsgFilterTypeType aFilterType,
>                               nsIMsgOperationListener *aCallback);

It looks for me ;
  When "Filter after Junk Classification" is invoked upon new mail check after Junk Classification ,
  folder is included in new mail check or not, applyIncomingFilters, Inbox or not, Special Folder or not, imap or not,
  and so on, are not cared
I thought that same msgDBHdr list as one passed to Junk Classification is used by "Filter after", but it doesn't look so. 

I can't find reason why "Filter after Junk Classification" is invoked upon folder open.
Someone calls nsMsgDBFolder::CallFilterPlugins() upon folder open of any folder to execute Junk filtering on newly detected mails?

Comment 53

[WADA:World Anti-bad-Duping Agency]

CallFilterPlugins() is invoked upon folder open by following root?

1. Someone calls EndMsgDownload() upon folder open when new mail is detected. (Unknown UID, no \Seen flag)
> https://dxr.mozilla.org/comm-central/search?q=EndMsgDownload&redirect=false

2. EndMsgDownload() is called by someone.
> https://dxr.mozilla.org/comm-central/source/mailnews/local/src/nsParseMailbox.cpp#2386
>   nsresult nsParseNewMailState::EndMsgDownload()

3. EndMsgDownload() calls CallFilterPlugins()
> https://dxr.mozilla.org/comm-central/source/mailnews/local/src/nsParseMailbox.cpp#2409
>   m_filterTargetFolders[index]->CallFilterPlugins(nullptr, &filtersRun);

Comment 54

[WADA:World Anti-bad-Duping Agency]

Above code is for local mail folder, so it's not applicable to imap...
Phenomenon is following?
1. Message filter + Junk filter is always invoked upon any of folder open, new mail via. IDLE, new mail check.
2. "Filter before Junk Classification" and "Junk filtering" is skipped when some conditions are satisfied. For example,
   Not new mail check, not "included in new mail check", !Inbox && true!=applyIncomingFilters, Trash or Junk, ...
3. But "Filter after Junk Classification" is not skipped in any condition.

Comment 55

[WADA:World Anti-bad-Duping Agency]

CallFilterPlugins() is called at two places in nsImapMailFolder.cpp.
> https://dxr.mozilla.org/comm-central/search?q=CallFilterPlugins&redirect=false
#4693 is in nsImapMailFolder::NormalEndMsgWriteStream() and CallFilterPlugins() is called when "if (m_filterListRequiresBody)".
#5889 is in nsImapMailFolder::HeaderFetchCompleted() and CallFilterPlugins() is called when "if (!m_filterListRequiresBody)".

In my test, Body filter is not used. So false==m_filterListRequiresBody.
Phenomenon may be following.
1. After header fetch for newly detected mail, CallFilterPlugins() is always called.
   (at least when newly known UID. I don't know "no \Seen flag" is required or not)
2. "Filter before Junk Classification" part is skipped when some conditions are satisfied,
    or nothing is done by "Filter before Junk Classification" part when some conditions are satisfied.
3. Nothing is done by "Junk Classification of Junk Filter" part when some conditions are satisfied.
4. "Filter before Junk Classification" part is always executed regardless of conditions.

Comment 56

[WADA:World Anti-bad-Duping Agency]

Sorry, wrong paste.
4. "Filter AFTER Classification" part is always executed regardless of conditions.

If this guess is right, phenomenon can be explained.

Comment 57

[WADA:World Anti-bad-Duping Agency]

If following is cause of problem,
> // if Body condition is not used,
> // message filter(and Junk Classification,PostPlugin filter,Junk move) is done immediately after header fetch of new mail
> HeaderFetchCompleted    : if (!m_filterListRequiresBody){ ...; CallFilterPlugins(); ... }
> // if Body condition is used,
> // message filter(and Junk Classification,PostPlugin filter,Junk move) is done after fetch body[] of mnew ail
> NormalEndMsgWriteStream : if  (m_filterListRequiresBody){ ...; CallFilterPlugins(); ... }
Following may be a possible solution.
> HeaderFetchCompleted    : if (!m_filterListRequiresBody && FilterNeeded()){ ... ; CallFilterPlugins(); ... }
> NormalEndMsgWriteStream : if  (m_filterListRequiresBody && FilterNeeded()){ ... ; CallFilterPlugins(); ... }
> function FilterNeeded(){ // Irresponsible && never-usable sample code in psuedo JavaScript 
>   if(Inbox && ( New mail by New mail check || New mail via. IDLE) ) return true;
>   else if(Inbox) return false; // folder open, ...
>   else if(Special Folder of Trash,Junk,Drafts,Template) return false;
>   else if( (!Inbox&&!Sent&&!Archives) && IncludeThisFolderInNewMailCheck && applyIncomingFilters ) return true;
>   else if( (!Inbox&&!Sent&&!Archives) && IncludeThisFolderInNewMailCheck && !applyIncomingFilters ) return false;
>   else if( (!Inbox&&!Sent&&!Archives) && !IncludeThisFolderInNewMailCheck  ) return false;
>   if(Archives && filter for Archiving) return true;
>   else if(Sent && filter for AfterSending) return true;
>   else ...
> }
This kind of function can be enhanced for request such as "Junk Filter only, without Message Filter", or "Message Filter only, without Junk Filter".

Hoping comment by Kent James.

Comment 58

[WADA:World Anti-bad-Duping Agency]

Following is copy of my bug 770137 comment #10.
If my guess on "invoking CallFilterPlugins() in imap code" is right, it's fault in imap code, instead of fault in nsMsgApplyFiltersToMessages which is used by "Filter after Junk Classification". 

*** copy of bug 770137 comment #10 ***

It looks for me that Bug 517849 is a report of same issue as this bug in imap on Junk Cllassification&&Junk Move.
Bug 517849 was opened on 2009-09-21, for "Gecko/20090915 Shredder/3.0b4pre".
> Bug 517849 - Only scan NEW emails for junk
> When thunderbird is setup with an IMAP account,
> it looks to be scanning all unread emails in all folders for junk mail.
It looks for me that Bug 189970 is one of oldest bug reports on same phenomenon in imap && Junk Filter.

Comment 59

[WADA:World Anti-bad-Duping Agency]

About whose fault, who is culprit.
Special folder flag check, server type check etc. was done in nsMsgDBFolder::CallFilterPlugins().
So, major culprit was nsMsgDBFolder::CallFilterPlugins() who acts between nsImapMailFolder.cpp/nsLocalMailFolder.cpp etc.) and Message Filter/Junk Filter.
I found interesting things.
In nsMsgDBFolder::CallFilterPlugins(), major code is for Junk Classification, and module started from code for Junk Classification.
It's perhaps because of follewing history. 
  Junk filter is implemented first(Junk Classification is called ClassifyJunk, filterForJunk, JunkMailPlugin etc.).
  It's perhaps implemented as plugin initially.
  -> message filter(Filter before, called ClassifyTrais, filterForOther) is implemented, and executed after Junk Classification
  -> changed to "message filter first, junk filter second"
  -> "Junk filter application on moved mail by message filter" is added
  -> "Filter after Junk Classification"(called filterPostPlugin) is inserted between Junk Classification and Junk move(by Tb 3)

I think "apply Junk filter on any new mail in any imap folder" is behavior since initial of Junk filtering in Thunderbird, because nsImapMailFolder.cpp simply calls CallFilterPlugins() after header fetch for new mail(no Body condition) or after fetch body[] for new mail(Body condition is used).
And, I believe that "Filter After Junk Classification(filterPostPlugin)" is also applied on any new mail in any imap folder as Junk Classification(filterForJunk) is done.

Anyway, hoping comment by Kent James who is Message filter/Junk filter professional.

Comment 61

[Arthur Goldberg]

Triggered this bug in TB 102.6.1 (64-bit).

Used the "Filter before Junk Classification" workaround to stop TB's infinite loop and CPU use.

Comment 62

[Günter Gersdorf]

This is still an issue with TB 115 and TB 121b6

STR:
Filter A moves message to folder1
Filter B moves message to folder2
Both filters have 'Stop Filter Execution' and 'Filter after Junk classification'

Result:
Filter A fires two times, than Filter B one time ad infinito

Expected result (as is with 'Filter before Junk classification'):
Filter A fires one time, stop

Comment 63

[GoldenBugatti]

Yep. Same issue here. TB115. TB should not fire the same filter over and over again infinitely on the same email.

What's worse, I have filter terms "from [e-mail]" AND "body contains XYZ" in filter A, and "from [e-mail]" AND "body does not contain XYZ" in filter B, and they're BOTH firing over and over again in an endless loop on the same email (which does contain XYZ in the body) anyway.

Comment 64

[GoldenBugatti]

Incidentally, the above poster who says this is related to "filter after junk filtering" setting is wrong. I have tried flipping that back and forth from before to after and back to before again, and no matter what, the same email, which doesn't even match one of the filters, is being filtered by both of them, over and over and over again forever.

This is a showstopper bug. Email filtering via rules is an extremely basic functionality and needs to work without requiring the users to waste hours of their time debugging and looking for workaround, otherwise the email client is not finished or ready for use, as is evidenced by the fact that, in my own experience, every single other email client since I first used email in 1986, whether standalone app or web-based UI, has had working filtering.