551707 - filter click OK failed with "This filter cannot be saved because some search terms are invalid in the current context"

Status: RESOLVED FIXED

(MailNews Core :: Filters, defect)

Description

[Wayne Mery (:wsmwk)]

(transferred from  Bug 510603 -  Review error handling of custom search terms and filter actions)

Got "This filter cannot be saved because some search terms are invalid in the current context" when adding a subject contains rule to an already existing list. I never got this error before for this or any other filter.

I go the error twice when clicking OK. Before clicking OK on my next and last attempt I clicked in the text field of the rule, then OK, it it saved fine. I confirm whether that last step helped.

1. open filter
2. click + in middle of existing list
3. add subject contains rule "455650]"
4. click OK
fails to save

the list is about 10 of these subject contains.

Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.2pre) Gecko/20100302 Lanikai/3.1b1

Comment 1

[Wayne Mery (:wsmwk)]

I've probably added 3 or 4 search terms since comment 0 and not gotten errors, but today I just received this again, in the same filter of course.
All the search terms for this filter are subject contains.
Nothing in error console

steps:
1. added one search term, and added one bug# to the filter name. 
2. save, failed
3. second attempt: (no additional changes) save, failed
4. 3rd attempt: removed bug# from name, save, success
5. add bug# back to name, save, success
(now I'm confused)

at #2, filter name is at 142 characters.
clearly, I don't have solid reproducible steps.

bugs which mention "filter cannot be saved" - https://bugzilla.mozilla.org/buglist.cgi?type1-0-0=substring&field0-0-0=short_desc&type0-0-1=substring&field0-0-1=component&type1-0-1=substring&value1-0-1=filter%20cannot%20be%20saved&classification=Client%20Software&classification=Components&query_format=advanced&value1-0-0=filter%20cannot%20be%20saved&value0-0-1=filter&type0-0-0=substring&value0-0-0=filter&field1-0-0=short_desc&product=MailNews%20Core&product=Thunderbird&field1-0-1=longdesc

Comment 2

[Kent James (:rkent)]

I'm not sure to know what to make of this behavior. Whatever is happening, it is clear that Subject contains is a valid search term, so this has nothing to do with the error message "some search terms are invalid". I think that has turned out to mean "for some reason I don't understand your filter, perhaps search terms are invalid ..." Reviewing that issue is part of bug 510603, but frankly I can't see myself putting any effor tin that for the foreseeable future.

So there seems to be some timing-related issues with the filter editing code that is causing some intermittent failure. I would guess that the failure is real, and so the code that prevents you from saving until it is fixed is doing the right thing. But the message is misleading.

Comment 3

[Chris Evans]

I have been asked to report that I'm getting this too.  I'm using TB 3.0.4 on Windows XP sp3.  I have a lot of message filters in the "Local Folders" filter store and my folders are on drive D:  The message comes up with monotonous but not perfect reliability for three quite large filters that I have at the end of my filter list and sometimes it persists if I try three times to save the filter and over a minute.  However, if I persevere, TB will (as far as I can see) always store the filter and the filters appear to work fine. Very weird, definitely timing related in some way.  Happy to supply my filters if someone will remind me where to find them.

Comment 4

[Kent James (:rkent)]

See GS topic http://getsatisfaction.com/mozilla_messaging/topics/saving_message_filter_fails_then_tb3_crashes?utm_content=topic_link&utm_medium=email&utm_source=reply_notification#

I need to investigate this further. It is going to be difficult without a reproducible test case, but I get the sense it is worse with filters with lots of terms, so perhaps I can create a reproducible case if I make a filter with hundreds of terms.

Comment 5

[Wayne Mery (:wsmwk)]

hooking up gsfn topic, and also merged a topic into http://gsfn.us/t/p82w

not related afaict, but for reference purposes for other types of filter encounters ... http://getsatisfaction.com/mozilla_messaging/topics/message_filters_doesnt_show_filters_until_i_save_a_new_filter_then_disappears_again where add-ons possibly caused a problem

Comment 6

[Mugros]

I'm getting this since 3.1.
I have a pretty straightforward filter for my work group. Just a list of conditions like "From is [e-mail]". Today i added some more members and i got this error. Of course, the inserted email address is well-formed, i copied it via the "copy email address" function from an email.
The bug is not fatal, because (wild guess) it only occurs 33% of the time and simply by dismissing the dialog and clicking ok in the filter again works. One time i had to click around a bit more within the conditions and then it worked.

Comment 7

[Brian]

I just got this bug now.  Fortunately the workaround is simple.  The filter in question checks for any of about 10 "From" email addresses, and if matched, moves the email to a local folder, then stops filter execution.  I can make the OK button work by scrolling down to the bottom of the list of email adddresses to match and then clicking OK.  Scrolling part way down did not work.

The only add-on I have is Glodaquilla 0.3.2, which I installed to deal with the over-eager indexing problems I had when I first installed Thunderbird 3.

I am using Thunderbird 3.1.2 on Windows XP SP3.

Comment 8

[DCAHH]

I am experiencing this as well. Thunderbird 3.1.4, German. After I added a new filter search term I need to click several times into the text box of one of the exisiting filters before "ok" is accepted and my new seacht term is added to the filter.

Comment 9

[Wayne Mery (:wsmwk)]

Hi kent. were you able to determine anything from dl's file mentioned in http://getsatisfaction.com/mozilla_messaging/topics/saving_message_filter_fails_then_tb3_crashes#reply_2495781 ?

Comment 10

[Tomislav Svoboda]

I can reproduce the error:

I have been getting this error for many months and I finally sat down and worked through it and can reproduce elements of it very easily. I first started getting it within my very large filter but but have isolated it in a reduced version of it which can be copied to a new filter. The filter text file for msgFilterRules.dat that can reproduce this error is below. To reproduce the error:

1) save the file below as msgFilterRules.dat in your application data thunderbird profile folder (for me it is in this folder:
C:\Documents and Settings\Svobodat\Application Data\Thunderbird\Profiles\p4qr82qt.default\Mail\mailbox14.utcc.utoronto-2.ca)
2) open thunderbird (3.1.6), 
3) click on tools/message filters
4) in my case select the home pop account (select account that this filter goes to)
5) double click on 'friends and family' filter
6) immediately click on OK and get the following message:

"This filter cannot be saved because some search terms are invalid in the current context"

7) click on ESC or cancel, you can reproduce the error by repeating steps 2-6

Note: I am guessing that there is hidden code or text in the file that cannot be seen in the MS text editor that I am using. It gets saved in MS Word 2002 (10.6612.6626 SP3) and is also not lost if I email it to myself using TB via my utoronto email account. It may get lost in this posting. After I post this I will see if it gets preserved or not.


version="9"
logging="yes"
name="Friends and Family"
enabled="yes"
type="1"
action="Move to folder"
actionValue="mailbox://nobody@Local%20Folders/Inbox/f28aaa35/Friends%20and%20Family"
condition="OR (from,contains,xxxxxxx) OR (from,contains,aaaa) OR (from,contains,bbbbbbbbbb) OR (from,contains,cccccccc) OR (from,contains,ddddddd) OR (from,contains,eeeeee) OR (from,contains,ffffff) OR (from,contains,ggg) OR (from,contains,coleen macpherson)"

Comment 11

[Tomislav Svoboda]

so it works! I was able to reproduce it from the text above. I reproduced it by cutting and pasting the text above...with a small adjustment:

- to make the process above work, find your msgFilterRules.dat file, open it (after making backups etc.), cut and paste the above text into it, ie "version="9" ...", however delete the line breaks so that the 'condition....' statement is on only a single line.

My environment is:
TB: 3.1.6
Operating system: Window XP Professional version 5.1 (Build 2600.xpsp.080413-2111: SP3)

Comment 12

[Tomislav Svoboda]

so here is proof that it must be some sort of hidden text....cut and paste the text below (and delete the line breaks etc.) into your msgFilterRules.dat file and this version works despite appearing to be identical text. (To make it work I simply scrolled down after step 5, put cursor in the field with 'macpherson' in it and then clicked on ok. Somehow that step eliminates the hidden text.

version="9"
logging="yes"
name="Friends and Family"
enabled="yes"
type="1"
action="Move to folder"
actionValue="mailbox://nobody@Local%20Folders/Inbox/f28aaa35/Friends%20and%20Family"
condition="OR (from,contains,xxxxxxx) OR (from,contains,aaaa) OR (from,contains,bbbbbbbbbb) OR (from,contains,cccccccc) OR (from,contains,ddddddd) OR (from,contains,eeeeee) OR (from,contains,ffffff) OR (from,contains,ggg) OR (from,contains,coleen macpherson)"

Comment 13

[Tomislav Svoboda]

I've reproduced this many times. I hope someone can now fix this bug!!

Comment 14

[Wayne Mery (:wsmwk)]

Attachment: msgFilterRules.dat

when I attempt to use comment 12 testcase (attached), when I try to access to filter for that account (I assigned it to gmail) I get

Your filters do not work because the msgFilterRules.dat file, which contains your filters, could not be read. A new msgFilterRules.dat file will be created and a backup of the old file, called rulesbackup.dat, will be created in the same directory.

Comment 15

[Tomislav Svoboda]

Hi there, I don't know all the ins and outs of how to set this up on a different installation...you should have a folder called "Friends and Family" for starters, don't know if that will solve the access problem. Also, maybe search for the pre-existing msgFilterRules.dat file and cut and paste the text file I provided into that so that you know it's in the right folder etc.

I will try installing this on another computer and see if I can reproduce it on a fresh install, maybe tonight...t

Comment 16

[Wayne Mery (:wsmwk)]

more examples, which I am merging into http://gsfn.us/t/p82w
http://gsfn.us/t/124wh
http://gsfn.us/t/ttqy

Comment 18

[Wayne Mery (:wsmwk)]

bug 561762 won't fix this, but it may help clarify.

Comment 19

[Kent James (:rkent)]

From comment 6 and step "6) immediately click on OK and get the following message:" I have finally been able to reproduce this! Let me investigate.

Comment 20

[Kent James (:rkent)]

Attachment: Add initialization check and diagnostics

This adds the same checks as http://mxr.mozilla.org/comm-central/source/mailnews/base/search/content/searchTermOverlay.js#534 plus console errors if there is still an issue.

When I reproduced the bug this caught the problem.

Comment 21

[David :Bienvenu]

Comment on attachment 527067 [details] [diff] [review]
Add initialization check and diagnostics

taking review - if you had someone else in mind, feel free to request from someone else...

Comment 22

[David :Bienvenu]

Comment on attachment 527067 [details] [diff] [review]
Add initialization check and diagnostics

this does fix the issue.

I think the search term editing UI does things that make gecko extremely unhappy, and I'm not sure if that's not actually the underlying cause.

Comment 23

[Kent James (:rkent)]

Comment on attachment 527067 [details] [diff] [review]
Add initialization check and diagnostics

Checked in http://hg.mozilla.org/comm-central/rev/13461770d168

Comment 24

[Serge Gautherie (:sgautherie)]

Comment on attachment 527067 [details] [diff] [review]
Add initialization check and diagnostics

http://hg.mozilla.org/releases/comm-2.0/rev/6e2be81ce8ef

Comment 25

[Jarred Walton]

I get this probably once per Thunderbird session if I try to make a new filter. Steps:

1) Start TB.
2) Open email in inbox.
3) Copy "from" email.
4) Click Tools->Message Filters.
5) Open existing filter.
6) Hit "+" and add new email address to list.
7) Click "Okay" and you get the error.

I have been seeing this issue for as long as I can remember -- it is not related to TB 3 AFAICT. I think it might be related to the number of items in the filter, but I'm not positive. Also, as a workaround I discovered, when the error pops up, click OK, then in TB (but with the filter still open and unsaved), click on a different mail folder. Now you can save. Also, scrolling the filter list (not even to the bottom, just scroll a little bit) also appears to let you save the filter.

Anyway, what I really want to get across is that the current status of "Resolved Fixed" is inaccurate unless I need a newer version of TB than 3.1.10. I have received this error at least four times today (once initially, and then I managed to recreate it three more times in testing).

Comment 26

[Jarred Walton]

Hmmm... addendum: it appears this is fixed in an upcoming 3.3 release? If so, disregard the above comment. :-)

Comment 27

[:aceman]

Wayne, could you verify this bug is fixed?
Then, is bug 615172 a duplicate of this? It is also against an older TB version.

Comment 29

[Wayne Mery (:wsmwk)]

Comment on attachment 527067 [details] [diff] [review]
Add initialization check and diagnostics

yes, this is definitely gone in current versions. And not fixed in 3.1 releases.

This was a persistent issue. But I don't recall how pervasive, but seems like low risk and there have been no regressions from the page, so flagging for approval for the current "long term" support version.

Comment 30

[David :Bienvenu]

I think this is relatively safe to take for 3.1.17, though I haven't checked if it applies cleanly.

Comment 31

[Mark Banner (:standard8)]

Checked into 3.1.x: http://hg.mozilla.org/releases/comm-1.9.2/rev/ce1c3a8fa276

Comment 32

[Steve VanSlyck]

Still getting this message, "This filter cannot be saved because some search terms are invalid in the current context," with the SpamAssissin filter on TB 17.0.2. Workaround?

Comment 33

[:aceman]

Can you attach the msgFilterRules.dat file from your account (see the Local directory path in account settings) ?

Comment 34

[Steve VanSlyck]

Attachment: mail filters file

Attachment provided.

Comment 35

[:aceman]

That file has no filters stored in it. Can you write what you try to do in Thunderbird (how you create the filter) until it fails?

Comment 36

[Brian Weichel]

I was seeing this same problem if I disabled synchronization on a folder that had filters which searched the body.  Re-enabling synchronization fixed the problem.