94968 - Mail filters are randomly disabled

Status: VERIFIED DUPLICATE of bug 115228

(MailNews Core :: Filters, defect, P1)

Description

[Sebastian Bergmann]

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.3) Gecko/20010801
BuildID:    2001080110

Mail filters are randomly disabled at startup time of Mozilla's mail/
news reader and have to be activated manually. This happens often, but
not always.


Reproducible: Sometimes
Steps to Reproduce:

Comment 1

[Håkan Waara]

Can you attach your 'rules.dat' file here?

Comment 2

[Charles Hixson]

It's happening to me, too.
N.B.:  All of my filters are of the "transfer to subfolder" variety.

It doesn't really seem random, but I sure can't place the pattern.  Some folders
seem to continue working.  The ones that stop tend to be the more recent ones.
This seems similar to a bug that happened a few months ago, though I can't place
the exact release. 

Comment 3

[RFLazaro]

I am using buil 2001083003 on WinNT

This is also happening to me with the current build I am using, the build before
this one was working fine. It seems to be random though but maybe you can find a
pattern. This is very annoying since most of my filters are set up to move mail
to certain directories and when one is 'randomly' disabled I have to manually
filter my inbox.

Comment 4

[laurel]

This can happen in many cases if the folder is renamed or otherwise not
available when filter fires.  Is the folder name the same as in the filter rule?
 Try resetting the folder in the action field of the filter rules dialog.

One example scenario is covered in bug 97530, and main original problem was bug
41720.

Comment 5

[Sebastian Bergmann]

Folders were not renamed, names of the folders are exactly the same as in the 
filter rules.

Comment 6

[laurel]

Well, I haven't seen this except in cases with bad folder info in the rules. In
the absence of any other clues, we may be able to uncover something if you
attach your rules.dat file.

Comment 7

[Sebastian Bergmann]

Attachment: my rules.dat, as requested

Comment 8

[laurel]

Is it a particular one/few that are disabling?

Comment 9

[D Dulay]

This is happening to me as well. I have only two or three filters of the
'transfer to subfolder' type. Only a single filter is affected, but there does
not appear be a pattern to its spontaneous disabling. The one that is affected
is - of course - a high traffic mail list (bugtraq).

Comment 10

[D Dulay]

Attachment: Rules.dat as described earlier. Failing rule is Bugtraq2

Comment 11

[Håkan Waara]

Wild guess: the folders that are used are Local Folders. I think that is the
culprit. We've seen filter bugs involving those before.

Naving might know more about this.

Comment 12

[John DeCarlo]

Attachment: Another 'rules.dat' to help debugging

Comment 13

[John DeCarlo]

Hello,

I have 162 mail filters.  Yet only a few ever seem to have this problem.  And it
only happens when I start the mail system.  I have tried starting in OffLine
mode and checking the Edit -> Message Filters - everything is OK.  I have tried
starting in OnLine mode and canceling the login to the mail server.  Every
filter is enabled.  As soon as I go online and start reading mail from my IMAP
server, some of the filters get disabled.  As already mentioned, these are
filters that Move the message from the IMAP Inbox to a Local Folder on my hard
drive.

I have attached my rules.dat file.

One more note.  I have another profile that goes against a POP email server, and
it has fewer rules that move messages to local folders.  I have not seen any
filters get disabled on that profile yet.  It only has 67 rules.  Also, while
the folders being moved to are local, they aren't in Local Folders, but in the
folder with the name of the POP server.

Thanks.

For me, the ARS List filter invariably becomes disabled and I have to manually
handle the 50+ messages that came in overnight.

Comment 14

[John DeCarlo]

I have been experimenting since I posted this morning.  One additional data
point. If I quit and restart and no mail came in during that time, all my
message filters are still enabled.  But when mail *does* come in after that, one
or more message filters are disabled.

And, the only ones that are disabled are ones that leave messages in my IMAP inbox.

Though it is difficult to prove, it seems that it may disable the first few (it
has been 1-5 for me) filters that it tries to apply (for the first delivery of
mail since Mozilla was started). 

It has been clear today that if I redownload mail, then restart Mozilla, and
just one message comes in that would fire a filter, that filter is disabled and
that message remains in my IMAP inbox.   

Thanks.

Comment 15

[Mike Shore]

I have been experiencing this same problem (or perhaps a similar one) in the
last several milestones of Mozilla. I'm fairly sure 0.9.1 had it, and I know
0.9.3 and 0.9.4 still contain it.

I'm using a POP3 server, and the pattern that I've found (which took me a while
to figure out) was that the filters would work fine until *I* sent an email to a
listserv which was filtered. Upon receiving that email, the message would not be
filtered, and the filter would be disabled.

This is rather annoying, especially when several emails for that listserv come
in at the same time as the message I sent.

Hope this helps, and I hope the problem is resolved soon.

Comment 16

[Navin Gupta]

Mike, I have tried your steps and it works for me.

Comment 17

[Navin Gupta]

Can anybody come up with a list of reproducible steps. The only way i can see 
this happening is when it doesn't find the destination folder. 

Comment 18

[John DeCarlo]

[e-mail] asks for reproducible steps.

1.  Use an IMAP account.
2.  Have a set of rules to move mail from the IMAP account Inbox to a folder in
the Local Folders area.
3.  Log on and collect your mail.
4.  You will find that some of the filters that were enabled are now disabled. 
And the only ones that are disabled are ones that should have fired, leaving
those messages in the IMAP Inbox.  But some of the filters will have worked,
moving the messages.  And, once reenabled, the disabled filters will work fine
until the next time MailNews is started.

Now, I have not tried to create everything from scratch, but I have attached my
rules.dat file and it doesn't seem to have anything wrong in it.  I
automatically upgraded from Netscape 4.75 to 6.1, then to nightly builds to get
other bugs fixed.

Comment 19

[John DeCarlo]

The current question is whether this bug can be reproduced.

I created a new profile to my same IMAP server from scratch.  I then created
three mail filters from scratch.  I cannot get them to disable themselves yet. 
Some are identical to what they are on previous profiles.  Some are new.  

Unfortunately, I find I cannot reproduce the problem.  If I have time, I will
try and duplicate my entire 162 filters and who knows how many local folders in
a new profile.  Either it won't have the problem and I can relax, or I can show
how to reproduce it.

At this point, I am confused.  It could be related to having migrated all the
information from an old account.  I just don't know.  Sorry.

Comment 20

[dado feigenblatt]

I don't have to send messages to lists to trigger this bug.
I use filters to move messages to non local folders, but I believe since I use
POP3 this might be exactly the same as IMAP moving to local folders.
The problem happens mostly when I start up, but it has happened mid session.
The filters that might get turned off are always the same, but not always all of
them will be be affected. I can't figure any pattern or reproduce the bug.
Is this still a Windows2000 bug? I'm on RedHat 7.2, Mozilla 0.9.3

Comment 21

[Henning Schnoor]

Hi,

I have had problems with filters that may or may not be related.

I also experienced automatic disabling of filters. When I looked into it, these
were all filters which had several criteria and were set to AND. When I
re-edited the filters, the criteria was set to OR for some reason, I can't set
this to AND.

When the filters are used with OR, the filters collide, meaning several filters
apply to a mail (for example, if you filter based on the word "digest" in the
subject line when you're on sereval mailing lusts).

Colliding filters seem to be disabled automatically. 

It seems the rules.dat is OK (set to AND), but the mail client handles it as an
OR filter (maybe a parser problem with rules.dat).

If you are interested in more details, let me know.

Comment 22

[Kathy A. Meyer]

Attachment: rules.dat file as req

Comment 23

[Navin Gupta]

henning, your problem has been fixed on today's trunk build - this was 
covered as part of bug 100978

Comment 24

[Navin Gupta]

I looked at the code and the only place we disable the filter is when we
cannot file/move messages to a destination folder or the destination 
folder does not exist. Please make sure that your filters are pointing
to your destination folders correctly. 

Comment 25

[dado feigenblatt]

My filters are pointing to real, valid, existing directories.
The only thing I can think of is some sort of NFS latency
making those directories unavailable, even if for a nanosecond.Hmmmm.... we use
automount here. That would introduce some good latency.
Could that be the problem? I have to check with my SysAdm. on how automount is
configured for home accounts.

Comment 26

[Steve M. Mann]

Comment on attachment 50349 [details]
rules.dat file as req

>version="8"
>logging="yes"
>name="GRRoW Stuff"
>enabled="yes"
>type="1"
>action="Move to folder"
>actionValue="mailbox://[e-mail]/Inbox"
>condition="OR (to,is,[e-mail]) OR (to,contains,[e-mail])"
>name="Golden Listserves"
>enabled="yes"
>type="1"
>action="Move to folder"
>actionValue="mailbox://[e-mail]/Inbox/Subscriptions/Golden%20Lists"
>condition="OR (from,contains,[e-mail]) OR (subject,contains,[G@H] GOLDEN digest) OR (to,contains,[e-mail]) OR (to,contains,) OR (from,contains,) OR (subject,contains,)"
>name="Steve"
>enabled="yes"
>type="1"
>action="Move to folder"
>actionValue="mailbox://[e-mail]/Inbox/My%20Mann"
>condition="OR (from,contains,@steve-mann.com) OR (from,contains,[e-mail]) OR (from,contains,Steve Mann)"
>name="Ex-Files"
>enabled="yes"
>type="1"
>action="Move to folder"
>actionValue="mailbox://[e-mail]/Inbox/X-FILES"
>condition="OR (from,contains,[e-mail]) OR (from,contains,[e-mail]) OR (from,contains,) OR (from,contains,) OR (from,contains,Brian Shaw)"
>name="Copies_To_Me"
>enabled="yes"
>type="1"
>action="Move to folder"
>actionValue="mailbox://[e-mail]/TO_FILE/Copies_To_Me"
>condition="AND (to,is,kmeyer@) AND (from,is,kmeyer@)"
>name="Mail_Subsystem"
>enabled="yes"
>type="1"
>action="Move to folder"
>actionValue="mailbox://[e-mail]/TO_FILE/Mail_Subsystem"
>condition="OR (subject,contains,Return Receipt) OR (from,contains,[e-mail]) OR (from,contains,MAILER-DAEMON)"
>name="Kreations"
>enabled="yes"
>type="1"
>action="Move to folder"
>actionValue="mailbox://[e-mail]/Inbox/Kreations"
>condition="OR (to,contains,[e-mail])"
>name="Site"
>enabled="yes"
>type="1"
>action="Move to folder"
>actionValue="mailbox://[e-mail]/Inbox/Kreations"
>condition="AND (subject,contains,Site Comments  )"

Comment 27

[Navin Gupta]

Since many users are experiencing this problems and I have seen a number of
dupes, I want to alert the user when we disable the filter. There is also one
case for local folders where message cannot be filtered, if we do not get folder
lock. I have added alert for that as well. patch coming up..

Comment 28

[Navin Gupta]

Attachment: proposed fix for throwing alerts.

Comment 29

[Navin Gupta]

cc robinf for alert wording review. 
cc bienvenu for code review.

Comment 30

[John DeCarlo]

>I looked at the code and the only place we disable the filter is when we
>cannot file/move messages to a destination folder or the destination 
>folder does not exist.

How does the code determine this?  After all, my experience originally was that 
filters only got disabled when the MailNews was first processing incoming mail 
and the mail filters.  However, I just added a new mail filter, during which I 
created a new Local Folder (a new subfolder under an existing folder), and that 
filter always gets disabled whenever mail comes in that applies to it.  

Could there be some sort of delay on my computer in initially accessing some 
Local Folders, so that the code would time out?  There needs to be some 
explanation of why the filter would get disabled initially, and then never 
again.

I have noticed occasional delays on my computer when I fire up some programs 
that display a list of files or folders, especially in certain directories that 
have lots of subfolders.

Also, I apologize for not looking at your code in the new build yet (was on 
vacation and had to switch back to 4.7 to avoid having several thousand 
unfiltered emails in my inbox).  But could the warning about the filter have an 
option to retry, if it doesn't?  Then if it were simply a time-out problem on my 
computer, I could get around it.

Comment 31

[David :Bienvenu]

Are you adding this code to help debug the problem? If so, I would only do it
behind a hidden pref. 

Comment 32

[robinf]

Review of error message text added in attachment 51038 [details] [diff] [review]:

>+alertFilterDisabled=The folder '%S' could not be found, so filter(s)
associated >with this folder are disabled. Verify that filter(s) point to the
right >destination folder.

Suggested wording: "The folder '%S' could not be found, so filter(s) associated
with this folder will be disabled. Verify that the folder exists, and that
filters point to a valid destination folder."

>+alertFilterDestinationLocked=The messages could not be filtered to folder '%S'
>because another operation is in progress.

Before I can suggest alternate wording: What causes this error? What happens to
the messages in this case? Do they land in the inbox? Does the filter get
disabled and then have to be re-enabled manually by the user before it will work
again? Any corrective action users can take? Thanks.

Comment 33

[David :Bienvenu]

I think it would be better to spend effort on figuring out why this is
happening. It should not happen at all.

Comment 34

[David :Bienvenu]

Comment on attachment 51038 [details] [diff] [review]
proposed fix for throwing alerts.

I think this is going to turn up false positives (i.e., we're going to tell users that a filter was disabled even though the folder is still present), if I understand the comments in this bug correctly. But I hope this helps us track down the problem, and is also a legitimate error in some situations, e.g., the user deletes an imap folder on a different machine.

Comment 35

[Navin Gupta]

>we're going to tell users that a filter was disabled even though the folder is 
>still present.

I am not sure about this. Can you give me an example. The alert messages in the 
patch are pretty accurate. I have cleaned up some related compact string bundle 
code and will attach the patch. I will ask seth to review the new patch and 
you could just sr. 

Comment 36

[David :Bienvenu]

Everything I read in this bug says that we're randomly disabling filters even
though the target folder exists. If that's true, then we're going to put up an
error message saying a filter has been disabled because the destination folder
doesn't exist, even though it does exist. Am I misunderstanding the comments in
this bug? For those of you that see this problem, do you have to go into the
filter editor and manually re-enable a filter after this happens?

Comment 37

[Navin Gupta]

>If that's true, then we're going to put up an
>error message saying a filter has been disabled because the destination folder
>doesn't exist, even though it does exist

The code will not reach alert message if that were to happen. 

I agree that this bug requires more investigation to track down random 
disabling. I suspect when the people in this bug report say the filter is 
randomly disabled they mean that message were not filtered to destination folder
rather than the state of the filter being disabled as shown in the filter 
list dialog. reporter and others who are experiencing this problem, please
correct me if I am wrong. 

Comment 38

[Navin Gupta]

Attachment: same fix w/ more clean-up

Comment 39

[David :Bienvenu]

I'm really not being clear, I can tell.

My assertion is that this code will get hit

 if (!parentFolder || !canFileMessages)
       {
         filter->SetEnabled(PR_FALSE);
+        destIFolder->ThrowAlertMsg("filterDisabled",msgWindow);
         return NS_MSG_NOT_A_MAIL_FOLDER;
       }

this code will disable the filter, and put up a message. And I'm speculating
that for some reason, we're hitting this code when we shouldn't.

Comment 40

[Navin Gupta]

This code may create bogus folder because if it does not find a resource
it creates one, but still it will not be visible in the folder-pane so they 
would know something is wrong and the alert would guide them to correct 
this mistake by using the filter editor. 

  nsCOMPtr<nsIRDFService> rdf(do_GetService(kRDFServiceCID, &err)); 
  nsCOMPtr<nsIRDFResource> res;
  err = rdf->GetResource(destFolderUri, getter_AddRefs(res));
  if (NS_FAILED(err))
    return err;

Comment 41

[Kathy A. Meyer]

> For those of you that see this problem, do you have to go into the
> filter editor and manually re-enable a filter after this happens?

Yes.  All filters can be on/enabled and once a mail comes through that has a
filter for it... it's as if that mail turns off the filter when it comes in. And
you must manually go into the filter editor and re-enable it. Every time it happens.

Comment 42

[David :Bienvenu]

Navin, you're right, and I think that created folder would not have it's parent
set correctly, so we would hit your code. That lends weight to the idea that the
folder target really doesn't exist (or has a bad uri).

Comment 43

[(not reading, please use [e-mail] instead)]

Comment on attachment 51118 [details] [diff] [review]
same fix w/ more clean-up

sr=sspitzer

please change ApplyFilterHit to applyFilterHit in the .idl file

Comment 44

[Navin Gupta]

robin,

>+alertFilterDestinationLocked=The messages could not be filtered to folder '%S'
>because another operation is in progress.

Before I can suggest alternate wording: What causes this error? What happens to
the messages in this case? Do they land in the inbox? Does the filter get
disabled and then have to be re-enabled manually by the user before it will work
again? Any corrective action users can take? Thanks.

This error is caused for example if you were compacting the folder, ya they land 
in inbox, the filter does not get disabled, the user cannot take any corrective
action.

Comment 45

[robinf]

>+alertFilterDestinationLocked=The messages could not be filtered to folder '%S'
>because another operation is in progress.

Then I think this wording is OK.

Comment 46

[Navin Gupta]

fix checked in to throw alerts. I hope this will solve problems of atleast
some users.  

Comment 47

[Sven Krohlas]

Hi,
I have a similar problem:
my mail filters for a pop3 account aren't working, until I view the Inbox of
this account. After that everything works fine.

Comment 48

[laurel]

*** Bug 106415 has been marked as a duplicate of this bug. ***

Comment 49

[laurel]

*** Bug 104917 has been marked as a duplicate of this bug. ***

Comment 50

[David A. Cobb]

Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.5+) Gecko/20011022
Windows98se (4.10.2222)

1) I'm getting the error message about "Cannot find target folder" but then,
when I go to edit the filters it is still enabled and the message that triggered
the error is either filtered correctly or tossed into the blackhole.

2) I'm also getting occasional "Another operation is in progress" message. 
Again, the message that triggered the error is either filtered or tossed away --
it isn't in my inbox.

When either of these happens on folder "X" it continues to happen for that "X"
for the duration of the present POP3 login.

Possibly related: I discovered by accident when I had a crash that Mozilla has
an awful lot of .msf files open [and thus vulnerable].  Since Windows is such a
lame system could this be causing a delay for some resource?

Comment 51

[Navin Gupta]

>1) I'm getting the error message about "Cannot find target folder" but then,
>when I go to edit the filters it is still enabled and the message that 
>triggered the error is either filtered correctly or tossed into the blackhole.

So, did you verify that folder exists in the folder pane. The folder-picker
in the filter editor does not always show the right thing. You mave have 
to reselect the target using that picker. 

>2) I'm also getting occasional "Another operation is in progress" message. 
>Again, the message that triggered the error is either filtered or tossed away 
>-- it isn't in my inbox.
Can you describe the full alert wording ?

Comment 52

[laurel]

*** Bug 105671 has been marked as a duplicate of this bug. ***

Comment 53

[laurel]

*** Bug 106662 has been marked as a duplicate of this bug. ***

Comment 54

[Ian Hawkins]

I had a similar problem that seemed to be caused by spaces in the folder name.

The rules.dat file listed the folder name with the space. Other filters, that also 
had spaces in the folder names, worked. These filters had "%20" instead of a space.

I changed the folder name to have an underscore instead. Adjusted the filter to
match the new folder name, and no more problem.

Comment 55

[dado feigenblatt]

I noticed that if instead of immediately getting my messages I hit cancel on the
POP3 password box, open any message folder other than the Inbox, the rules then
work fine.
It just looks like that in spite of Mozilla showing the folders on the left
panel, it doesn't know of their existence until you kick it somehow.

Comment 56

[dado feigenblatt]

Oops. I forgot to add this.
Has anybody mentioned that the problem happens only to nested folders?
Or is that just me?

Comment 57

[David A. Cobb]

I had not particularly noticed that it's always nested folders - I don't have
any other kind.

This may not be relevant, or maybe it is.  I find if I create folder A and then
A/B using the context menu on the folder-list that I cannot successfully MOVE a
message from inbox to A/B.  IOW, as pointed out above, some parts of Moz at
least do not know about the existence of the new folder.  I imagine that firing
a filter to move the message to such a phantom folder might get an error that
disables the filter.

NAVIN asked: "So, did you verify that folder exists in the folder pane. The
folder-picker in the filter editor does not always show the right thing. You
mave have to reselect the target using that picker. "
I'M HAVING A SENIOR MOMENT, and don't understand at all.  I see only one way to
specify the target folder for a filter and that is the folder-picker on the
filter editor; if it doesn't always show the right thing how in H*** am I
supposed to pick it?  However, the answer is yes - the folder truly does exist.

So, did you verify that folder exists in the folder pane. The folder-picker
in the filter editor does not always show the right thing. You mave have 
to reselect the target using that picker. 

So, did you verify that folder exists in the folder pane. The
folder-picker
in the filter editor does not always show the right thing.
You mave have 
to reselect the target using that picker.

Comment 58

[Eric Krock]

Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:0.9.5) Gecko/20011011
I'm seeing this too using 0.9.5 on WinME with IMAP and rules that filter to 
folders on a network server. (Yes, the directories actually exist.) It doesn't 
happen every single time, but it occurs frequently.

1) The filters are disabled by Mozilla with the error alert (thanks for putting 
that in--otherwise I wouldn't have known that messages were getting mislaid) at 
the time that Mozilla Mail starts up, checks for new mail, and downloads it. 

2) I may be able to help in reliably reproducing this bug. Although I have 13 
filters, I'm only experiencing this bug on some of the (multiple) filters that 
are intended to move incoming messages into a single one of my folders, namely 
aa_Inbox. (I try to get all my valid non-spam email [that isn't intercepted by 
a few very specific preceding filters for specific folders] to go there so that 
spam will wind up being dropped into my regular Inbox. I will attach my 
rules.dat file.

3) The target folder, aa_Inbox, is indeed nested under the Inbox for the IMAP 
server. The hierarchy is:
<Kontiki IMAP server name>
--INBOX
----aa_a Inbox old     [note spaces, underscore in name]
----aa_email update    [note spaces, underscore in name]
----aa_Inbox           [note underscore in name]

The hyphens in the above diagram are not part of the folder names but are 
intended to show levels of nesting.

4) Like Henning, I think colliding/overlapping rules may be a factor here. The 
filters that are getting turned off are ones that are either or both of the 
following (I'm not sure):
a) they are being invoked by the incoming emails
b) they are filters that might be invoked by an incoming email, but might 
*also* be invoked by another filter earlier or later in the list. The folder 
that's experiencing the problem, aa_Inbox, has multiple rules that cause mail 
to drop into it. In order, they are:

A) "mass Inbox kontiki": any, if to or cc contains kontiki --> aa_Inbox
B) "ekrock": any, if to or cc contains ekrock --> aa_Inbox
C) "eric": any, if to or cc contains eric --> aa_Inbox
D) "mass Inbox 2 zodiacnetworks": any, if to or cc contains zodiacnetworks --> 
aa_Inbox

Note that for some of these emails, multiple rules would potentially apply 
(e.g. any email to [e-mail] would be caught first by A, but C would 
have caught it if A weren't invoked first). I suspect that a situation where an 
email would trigger more than one of the rules has something to do with causing 
the bug. I don't recall seeing the bug turn off all four rules at once; it's 
always turning off one or two.

(Originally, these were all one rule, an OR rule with four clauses, but that 
triggered a bug in N6.1, so I reduced the clauses to three. Then this bug 
started happening in 0.9.5, so I split apart the OR rule with three clauses 
into three separate rules to see if that helped, but it didn't.)

Nominating mozilla0.9.5 because:
1) this bug renders filters highly unreliable
2) it forces the user to manually sort the mail that arrived after the filter 
got turned off, every time this happens, wasting a lot of their time
3) filters are increasingly widely used as a defense against spam (I depend on 
them for all my email, as shown above)
4) the bug happens frequently (several times per day to me)

Comment 59

[Eric Krock]

Attachment: rules.dat for ericweb

Comment 60

[Navin Gupta]

You have just attached 4x rules.dat. 

Comment 61

[dado feigenblatt]

Maybe this is not related at all.
If I need to move a message, I usually drag it from the folder content pane (?)
to the destination folder on the folder tree.
If I try right clicking the message and try using "Move To", the folders will
expand one at a time. I have to release the button and right click again to see
only one level more down, and repeat to get down another level. Branches I
haven't explored won't show up. I have to repeat the procedure for each level in
each branch. I have many folders that are 4 levels down.

Mozilla 0.9.5+, Red Hat 7.2, 2.4.9-0.5smp

Comment 62

[Navin Gupta]

Attachment: patch to make folder-picker not show bogus folders

Comment 63

[Navin Gupta]

This happens when you have folders in rules.dat that do not really exist.
I have fixed it so that such bogus folders don't show up in folder-picker in 
filter-Editor, this will help the users to select correct/valid targets for 
their filters.

Comment 64

[Scott MacGregor]

Comment on attachment 55287 [details] [diff] [review]
patch to make folder-picker not show bogus folders

r=mscott

Comment 65

[Navin Gupta]

checked in the bogus folders patch. should make users more aware of invalid
targets. 

Comment 66

[Eric Krock]

Navin -- At least in my case, this is happening with a target that is a 100%
existing, valid folder. Messages are delivered into this target all day long. In
fact, other rules point at this same target, and they aren't all disabled when
this occurs; of all the rules that point at the target, only some of the rules
are disabled, seemingly randomly. 

Are you currently operating on the theory that the cause for disablement of a
filter rule is that the target folder doesn't exist? At least in my case, that's
not the case.
Sorry about attaching the wrong rules.dat file. How do I locate the correct
rules.dat on disk? I've got a bunch that are returned by Find File, and I'm not
sure which one is associated with my (only) Moz/N6 profile.

Comment 67

[Navin Gupta]

I am not saying that the previous patch will solve your problem, but it may 
help others. To look for correct rules.dat file look under Edit | Mail & News 
Account Settings | Server Settings ->  local directory  this will give you 
the location where your rules.dat is stored. 

Comment 68

[Navin Gupta]

I suspect special folder names (ie, underscore, space in folder name) to be the 
cause of the problem, but i will have to check.

Comment 69

[[not reading bugmail]]

I've seen problems where I cannot get those messages that are not totally
downloaded to my inbox will not filter/move to other folders till I get the
whole files.  And Del/Move more than 1 selection will stop if this is also the
case.. could be part of the reason some stop.. it cannot skip a message that is
not totally downloaded. (which seems to me to disable all operations if that
message is selected or part of a filter.

-dennis

Comment 70

[David A. Cobb]

Navin: Is the existence of "bogus" folders sensitive to whether the folder was
created on the folder-tree panel vs. using the "New Folder" button on the
filterEditor?  

Comment 71

[David A. Cobb]

Another Navin question:
NavinG writes: 2001-10-26 16:28
I suspect special folder names (ie, underscore, space in folder name) to be the 
cause of the problem, but i will have to check.

I know there is a quirk that Moz cannot tolerate Sharp(#) in a path, could you
dientify any other "problem" characters?  Nearly all of mine have "_" or "-",
some have "." -- I try to avoid spaces but there may be a few.

Comment 72

[Eric Krock]

Attachment: CORRECT Moz/N6 rules.dat for ericweb

Comment 73

[Eric Krock]

As an experiment, I've removed the underscore from my aa_Inbox folder that was
having this problem, changing it to aaaInbox. I'll report back as soon as I
either (a) see the problem again, even with the changed name, or (b) don't see
the problem long enough that I conclude this has fixed it. (I see this about 1-3
times/day so a few days should be enough.)

Comment 74

[David A. Cobb]

POSSIBLY RELATED: OTHER FOLDER FAILURES

If a machine crashes while Mozilla is open, a great many *.msf summary files are
damaged - as reported by the diskscan on reboot.  Even though my diskscan
[McAfee] thinks it's smart enough to fix the files, for *.msf it's batting zero.  

The most obvious symptom of the damage is that many filters, I believe those
targeted to the damaged folders, are silently disabled.  The most recent time I
wound up with about 300 messages in my inbox needing to be manually filtered --
what fun!  If I attempt to move a group of messages to their appropriate
destination, many simply cannot be moved.  My work-around has been to quit all
the way out of Mozilla and delete the associated *.msf files plus the inbox.msf.
 That fixes the move problem in about 80% of the cases.  

As an hypothesis, I think that anything that gets in the way of a filter
move-message operation may cause the filter to be flagged as disabled. 

Comment 75

[John DeCarlo]

I agree that there appear to be instances when the folders seem inaccessible to 
move operations, even by hand.  I find on my Windows 2000 system that when I try 
to move a message by hand from my IMAP inbox to one of my Local Folders, that it 
won't let me (it fails to open the third or fourth level folder) the first time 
I try it, but then lets me do it when I retry.

This does appear to be a related problem.

OTOH, I have never had a problem opening the folder in the left hand folders 
pane and dragging a message there.  Only when right clicking on a message and 
specifying Move or Copy to a Local Folder will it fail to open up the third or 
fourth level folder to show the leaf folders inside it.

Comment 76

[Eric Krock]

It's been 2.5 days now since I changed the name of my "aa Inbox" folder to
"aaaInbox" to remove the space, and I haven't seen the filter turned off once,
whereas before it was happening 1-5 times a day. I will update this bug
immediately if I see the problem again. Unless that happens, it appears that
having a space in the name of the folder may be a contributing cause to this
problem, and that deleting the space might be a workaround.

Comment 77

[David A. Cobb]

My somewhat educated intuition suspects this could be another symptom of a
faulure of the MOVE MESSAGE function that I reported in #104915.  The commentary
there may be useful [no-one but me is commenting ;-) ].  There's one comment
here that the failing filters are all MOVE actions, also a mention of messages
'not completely received.'  The following observation from #104915 may be a clue
to this.

<quoting-myself>
A CLUE !?

I took a look at the Message Source of a message I could not move.  The first
few lines of envelope were as follows:
 From - Fri Nov 02 20:11:12 2001
X-UIDL: <[e-mail]>
X-Mozilla-Status: From - Fri Nov 02 20:11:12 2001
X-UIDL: <[e-mail]>
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
Return-Path: <[e-mail]>

NOTE, The envelope is screwed up in such a way that the first X-Mozilla-Status
is not going to be anything Moz expects.  Actually, I'm seeing quite a bit of
this sort of crappyness with envelopes from other messages starting in the
middle of another message; I haven't filed a bug 'cause I half suspect my provider.
</quoting-myself>

Is it possible to verify that all the failing filters are MOVE's?  And does
anyone else who's seeing this error have failures manually MOVE'ing messages?

Comment 78

[Henning Schnoor]

Hi,

I can confirm failing filters are all MOVE-action types (but then, I don't have
any other). I don't have trouble moving the messages myself.

I think this may be related to the "space in folder name" situations, as others
have commented.

I'll check if messages who fail to be filtered have strange hearers.

Comment 79

[David A. Cobb]

[I thought I had mentioned this]
My failures do NOT involve spaces in folder names.  Most of my folders are
mailing lists and the folders are named by the list name.
OTOH, because I have seen bugs related to the tree depth of folders, ALL of mine
are named, e.g. "Local Folders/Public/_M_/mozilla-general" or "Local
Folders/Public/_O_/OOO.Discussion" (for the Discussion list at Open Office.Org.

Comment 80

[Håkan Waara]

Can you attach a rules.dat file with some filters that are failing? We need 
more specific testcases and details here...

Comment 81

[Eric Krock]

FYI: It's been 14 days since I took the spaces out of my folder names and I
haven't seen this problem once, whereas I was seeing it 3-4x/day before. So at
least in my case, seems safe to say that space(s) in the folder name were one
necessary condition for this to occur.

Comment 82

[David A. Cobb]

Attachment: Rules.dat from superbiskit

as requested

Comment 83

[Arnoud Berendsen]

Attachment: Another troublesome rules.dat, per request

I'm attaching my rules.dat as my filter for "W2KNews" does not work. While I
have quite a few rules, this is the only one that seems to be giving me
trouble.

I first though this might be because this rule moves something from Inbox to
Inbox/folder/subfolder. Most of my other rules moves something to Inbox/folder
(and  the few other ones that move stuff to folder/subfolder's are hardly ever
used so I wasn't sure if they worked or not). I tested it though, and this is
not what's causing it; other rules work correctly in this case.

While the rulename has a space in it ("Win2k news"), other rules with spaces in
the name do work correctly, so this is not it either.

I did find out one interesting point though: Case Matters!
The filter looks for the string "W2KNews:", and the messages from this list
have "W2Knews:" in them (note the lowercase n). These messages don't get
filtered and stay in my Inbox. However, if I send a message to myself with the
string "W2KNews:" (note the uppercase N), then the message DOES get filtered!
So changing the case matters for this filter. For other filters, case doesn't
seem to matter though. Could it be that the : in the filterrule is confusing
Mozilla into casesensitivity?

Comment 84

[David A. Cobb]

Perhaps what I'm seeing is no longer the same bug.  My filters do not get
flagged "disabled," but they DO fail massively.

Symptom: If I'm working on a local folder and the message transfer starts, I get
messages "The filter rule for This.Folder-Name cannot be executed.  Another
operation is in process for this folder."  I usually get that several times (say
10) all for the same folder.  Then when I look at my Inbox I find around 250
messages that were not filtered and should have been.  It appears the entire
filter process simply quit.

Additionally, I am finding "clumps" of two to five messages here and there in my
local folders that should have been filed in a different folder - thus more
filter breakage.

SIDE QUESTION: Does this seem to be the same bug or should I open a new one?

Comment 85

[laurel]

The local folder is also present in a couple other bugs... bug 109855, bug 115228.
Navin: Please assign a priority to this bug and if possible, let's clarify this
issue or consolidate the other mentioned bugs to this.  Let me know.

Comment 86

[Navin Gupta]

Lets consolidate the other mentioned bugs to this bug. I mean mark those
bugs as dependent on this bug. targetting for 0.9.9. 

Comment 87

[Jan Schnackenberg]

I keep getting disabled filters, too. I only have "move"-filters from IMAP to
"Local Folders". I have one filter, that always disables. It goes to "Local
Folders/FH/Arbeit/ZeosLib", so there is a space in "Local Folders".

Comment 88

[David A. Cobb]

Just a quick heads up.  This bug was not seen through most of the December
nightlies [ I don't recall the specific versions ].  As of 2002-01-03-03 it is
back WITH A VENGANCE.  I cannot seem to get one complete session with my mail
without running into some filter failure.

I'd suggest a hard look at code changed within the past, say, 5 days.

Comment 89

[Eric Krock]

Ditto David Cobb's observation. With 0.9.6 and previous releases, the problem
had disappeared after I renamed my rule-target folders to eliminate spaces.
However, after upgrading to 0.9.7 on WinME, I see that emails are again going
into my default Inbox where rules should intercept them and drop them into
aaaInbox instead. Regression. I wonder if this is related to the fact that in
0.9.7, Mozilla is now alphabetizing, coloring, ordering, and handling the
"Inbox" folder as if it were any other normal folder (no longer special-casing
it so it shows up at the top--this is a separate inconvenience, BTW, is it a bug
or is it the new 0.9.7 spec for some reason?).

Comment 90

[Håkan Waara]

This bug is quite interesting, because we still haven't figured out reproducible
steps, which makes it basically unfixable. :-)

So, speculation on what the culprit could be is not necessary -- what we
seriously need here is a testcase that is reproducible for anyone.

The best way to create such a testcase is to create a new profile, and using
that start from scratch; try to arrange it with as few steps as possible, so you
can reproduce the bug.

Here is a list of clues, so far:
* Happens only for "Move To Folder" action
* Could be related to filter name (e.g., it went away for ericweb when he took
away the spaces from the name)
* Happens only when moving from POP/IMAP -> Local Folders (?)

Now, let's find some reproducible steps (using a new profile, to make sure
anyone can reproduce it).

Comment 91

[David A. Cobb]

[e-mail] writes:
<quote>
This bug is quite interesting, because we still haven't figured out reproducible
steps, which makes it basically unfixable. :-)
</quote>

"Interesting" as in the old Chinese curse "May you live in interesting times!"

Would it be possible to build a version with some state logging, additional
assertions, or other debugging outputs enabled for Mail-News?  I see the "Debug"
menu for the browser but nothing similar in mail.  This and some similar bugs
are more than sufficiently annoying that I would put some time into debugging. 
However with a Win98 box, very limited disk, and all free tools, I can't build
Mozilla here.  And you'd be on version 2.0 by the time I figured out the logic
well enough to know where to put the traps.

Comment 92

[Navin Gupta]

we are going to do something about this bug in upcoming milestone.

Comment 93

[Henning Schnoor]

Just wanted to comment on something:

A while ago, someone mentioned the bug disappeared and then reappeared with a
certain nightly. I've been using the same build (I only use milestone builds)
for weeks and the bug disappeared for a few days, then suddenly was back. So I
doubt it has something to do with the code changed at that time.

I noticed the bug would often re-appear when I dual-boot my operation systems,
I'm sharing my mail folder between  Linux and a Windows profile. Usually, I only
use Linux, and when I boot Windows and download my mail with Mozilla using
Windows, the bug sometimes re-appears (then appearing with the Linux build as well).

But that isn't the only way it shows up, so this might be just coincidence.

Comment 94

[Axel Heider]

This bug reapeared today again, after I installed 20020107 on Win98.
It never occured with the nightly builds published in the last four
weeks.

Comment 95

[Henning Schnoor]

I followed one of the links to other bugs mentioned a few days ago - it seems
the bug (this and the one mentioned there) only occurs if the "Local Folders"
subtree in mailnews hasn't been expanded since Mozilla started. 

As soon as you expand the folder, the filters work.

I only gave this a short try, maybe others can verify this.

This behaviour would explain why it's been difficult to reproduce this bug.

Comment 96

[Michael Klose]

@Henning (Comment #95)

I can reproduce it every time.

I have 3 e-mail accounts. The second e-mail account is used for various mailing
lists.

In local folders I have a subfolder ("Linux&IPv6") under which I have all the
mailing lists (again as subfolders of this). My rules for the second mail
acoount configured are to move the mails from the mailinglists into these folders.

The OS Entry of this bug reads 'Windows 2000' but this happens under Win98 as well.


As soon as I collapse the localfolders and receive new mail, the folders cannot
be found. If I expand the local folders, they start working again immediately.

To test this, I suggest either subscribing to a lot of mailing lists, or
subsribe to a high volume one (Linux Kernel Mailing List(?))

Comment 97

[Navin Gupta]

I have tried lots of folder names with _, space, and other special chars for 
both local and imap and it works for me. 

The only case where filter gets disabled is when folders are collapsed. I am
going to mark this bug as dup of bug 115228 because that deals specifically with
this issue. 

Please log separate bug about any other issue or if you find a folder name that
causes filter to get disabled. 

*** This bug has been marked as a duplicate of 115228 ***

Comment 98

[laurel]

marking verified as a duplicate