534897 - SpamAssassinYes message filter magically appears for all accounts. Resolved by turning off the Trust SpamAssassin headers in account options.

Status: NEW

(MailNews Core :: Filters, defect)

Description

[Bo Gusman]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR 3.5.30729)
Build Identifier: 3.0

I recently upgraded from TB 2.x to 3.0 and noticed that my mail filters were not working properly. Investigating revealed that the first rule in the list was a rule SpamAssassinYes rule in which the first two of three criteria were blank

   Subject contains <empty>
   Subject contains <empty>
   Subject contains ***SPAM***

This rule appeared in all accounts managed by TB. 

I tried removing this filter rule but was surprised to see that it automatically reappeared. I removed ALL filter rules for all accounts and the SA rule reappeared. 

Reproducible: Always

Steps to Reproduce:
1. delete message filter rules for all accounts
2. observe empty filter lists
3. after a short period, open message filters and observe SpamAssassinYes rule back in place


Expected Results:  
I expect that removing a message filter will result in that message filter staying removed.

Comment 1

[Wayne Mery (:wsmwk)]

strange, not seen nor heard of this. 

other than simply having the extra entries, did this affect correct filter operation?

Comment 2

[Bo Gusman]

Hi Wayne.

If this rule is at the top of the list of rules, it apparently stops processing the others - there is no indication of a failure, but messages that *should* have been processed remain in the inbox.

I have since removed all of the rules and am slowly rebuilding them as I go, ensuring that the SpamAssassinYes rule is at the bottom. Things seem to be working just fine with this configuration, though the rule continues to return automatically after deleting.

This instance of TB is one of two that I have - the problem is occurring on my laptop at work. On my machine at home, I have also installed TB 3 and first thing last night I checked to see what *that* one was doing. The SpamAssassinYes rule is not present and the filters appear to be working just fine.

I can't say if that rule was in place in the 2.0 version.

Comment 3

[Kent James (:rkent)]

These filters appear as a result of the "Trust junk mail headers set by" ... SpamAssassin. I can get these filters to appear in the filter list editor by enabling that option, then restarting. I also had in my profile one SpamAssassinYes filter I did not expect, as well as one SpamPalYes, but I do tests all of the time so it is not inconceivble that I set it myself at some point. I'm not sure why the restart is required for me, and I am not sure if these appeared in TB2 under the same circumstances. I'm also not sure if these filters are supposed to be hidden or not. I do know that the "trust junk mail headers .. " functionality is implemented by a filter.

I would be surprised if these filters interrupt the functionality of normal filters though, unless they are firing to move messages.

So if anyone had time, it would be helpful to more carefully test the relationship between the "trust junk mail headers ..." setting and the appearance of these filters, both in TB2 and TB3 to see if there is any change in behavior. AFAIK there was little if any expected change to this functionality in TB3.

Comment 4

[Wayne Mery (:wsmwk)]

(brief, because I'm not going to retype all of what I just lost before clicking submit)

I see SpamAssassinYes rule exposed going back at least as far as 3.0b1, circa dec 2008. It is not exposed in v2 => regression. SpamAssassinYes rule is built at startup or enable time, and should not be exposed.

I don't see the failure of filters that Bo sees. However, I see two lines of Sender starts with Yes, which is troubling.

Perhaps someone can find the regression range. http://www.rumblingedge.com/2009/02/24/howto-find-regression-windows-through-manual-binary-search/


cc: some people from bug 381589, which also has some description of how SpamAssassin works in TB.

Comment 5

[Bo Gusman]

After turning off the "trust junk mail headers" option, the SpamAssassinYes rule no longer regenerates.

I can also confirm that if the SpamAssassinYes rule is the first rule of a set, the remaining rules are not processed. I have other rules that process based on subject and from headers that simply do not work if SpamAssassinYes is the first rule. It was those messages that weren't moved from the inbox that got me started looking into this. If I had to guess, it was the first to blank conditions that caused the problem.

I'm going to turn this on at home tonight and see what the SpamAssassinYes rule looks like there.

Comment 6

[Bo Gusman]

Attachment: screenshot of SpamAssassinRule

Comment 7

[Bo Gusman]

The attached screenshot shows the rule created on one of my accounts on my home computer. This is identical to what I see on my work machine. I can't believe that the first two blank criteria are correct. 

The problem is resolved by turning off the Trust SpamAssassin headers in account options.

Comment 8

[Wayne Mery (:wsmwk)]

Attachment: SpamAssassinYes.png

what's interesting is your two subject criteria seem greyed out. (mine are not) This may explain why your lower filters aren't triggering. 

Do you get a response if click on Subject?

Comment 9

[Bo Gusman]

I actually hadn't noticed that they are in fact greyed out but now that you mention it, I see that they are.

They are fully functional however. Clicking on either of them gets me a dropdown showing the various selections.

Another data point: my work machine was an upgrade over the top of TB 2. The install at home is a fresh install on a new O/S (Win7). I don't think I have a version of linux anywhere to compare against, but that won't be hard to install - got lots of those boxes to play with.

Comment 10

[Bob Henson]

I'm not sure if this is the same bug, but as well as an non-removable filter called "SpampalYes" (I have the TB option set to "trust" Spampal) I also have one called "mozilla-temporary-internal-MDN-receipt-filter" which appeared on installing TB 3.x. Any attempt to add data to them so that they actually do something results in the data disappearing when the filter window is re-opened. They don't seem to cause any problems with my other filters. I'm using TB 3.0.1 running under Windows 7.

Comment 11

[ucritoph]

The message filter titled spamassassinyes has started appearing and reappearing (after deletion) in my list of message filters even though I have never set such a filter up nor do I have Spam Assassin on my system. I am using Mac OS 10.8 and Thunderbird 17.0.8. Except for having been using another family member's Wifi connection, I added nothing recently to any of my programs or changed any settings in Thunderbird.

Comment 12

[Spampot]

I confirm the "SpamAssassinYes" filter automatically appearing if the option to trust SpamAssassin header is set for Thunderbird 31.5.0 on Windows 8.1 64-bit). As far as I know, the filter is supposed to be handled internally without visually appearing in the message filters (so this is probably a bug). Also, the bug seems to depend on a) whether it is a POP3 or IMAP acccount, and b) whether or not additional filters exist for that account.

Comment 13

[Ludovic Hirlimann [:Usul]]

Removing myslef on all the bugs I'm cced on. Please NI me if you need something on MailNews Core bugs from me.

Comment 14

[effemmeffe]

I noticed this rule because it moves a message from my presonal bugzilla into the spam folder.

Comment 15

[G J Piper]

This is happening for me on macOS Sierra too. Firefox 51.0.1 (64-bit).

Comment 16

[You can set nickname by putting [:nickname] in the field doesn't work? See Bug 1146127]

The 'SpamAssassinYes'-filter is still automatically being (re)created by Thunderbird itself in 2021. It also cannot be adjusted because of the error mentioned in bug 1070684: This filter cannot be saved because the search term "Junk Score Origin isn't" is invalid in the current context.

As far as I can find here this SpamAssassinYes-filter was first created/reported in bug 381589, a follow-up from bug 378340. Both closed 14 years ago.

Apparently the SpamAssassinYes-filter was so 'good' the programmers decided to add it to all new Thunderbird versions, thus creating new bug reports like:

• bug 1070684: This filter cannot be saved because the search term "Junk Score Origin isn't" is invalid, 7 years ago.
• And this one, bug 534897: 'SpamAssassinYes'-filter is automatically being (re)created by Thunderbird, 12 years ago;

Also, there is no information whatsoever on the 'SpamAssassinYes'-filter at the support pages:

• Thunderbird and Junk / Spam Messages;
• Organize Your Messages by Using Filters.

So this bug still hasn't been explained nor solved yet... Maybe someone of the development-crew should take some action here? Like explaining/solving things?

Note: SpamAssassin uses the text "*****SPAM*****" (five stars) not "***SPAM***" (three stars) by default in the subject nowadays, so the filter is not even working!

Comment 17

[Steve]

same here.

Comment 18

[David Rankin]

There appears to be a regression. New SpamAssassinYes filters were just create for all accounts in Tbird 91.10.0 along with mozilla-temporary-internal-MDN-receipt-filter that broke moving return receipts to the sent folder (that had otherwise worked correctly for 20 years). This scrambles filtering on incoming mail with most legitimate mail now ending up in the junk folder and messages being misdelivered to unrelated folders.

Comment 19

[Stuart]

(In reply to You can set nickname by putting [:nickname] in the field doesn't work? See Bug 1146127 from comment #16)

The 'SpamAssassinYes'-filter is still automatically being (re)created by Thunderbird itself in 2021. It also cannot be adjusted because of the error mentioned in bug: 1070684

This filter cannot be saved because the search term "Junk Score Origin isn't" is invalid in the current context.

[...]

Also, there is no information whatsoever on the 'SpamAssassinYes'-filter at the support pages:

• Thunderbird and Junk / Spam Messages;
• Organize Your Messages by Using Filters.

So this bug still hasn't been explained nor solved yet... Maybe someone of the development-crew should take some action here? Like explaining/solving things?

Note: SpamAssassin uses the text "*****SPAM*****" (five stars) not "***SPAM***" (three stars) by default in the subject nowadays, so the filter is not even working!

I would like to know what to do here also, after using Thunderbird since pretty much the beginning I still have this same issue. What do I do with this filter? Do I just delete it? Do I leave it alone? Does it do anything? Is the "junk score origin" filter doing anything even though it is invalid? I cannot find any official guides/docs or mentions on what to do here.

Comment 20

[Wayne Mery (:wsmwk)]

Is Trust junk mail headers enabled for SpamAssassin, in account options > Junk Settings ?

Comment 21

[You can set nickname by putting [:nickname] in the field doesn't work? See Bug 1146127]

As far as I know yes.
I mean (Dutch [SIC]): "Ongewenstberichtkopregels vertrouwen die zijn ingesteld door: <filternaam>" is on and SpamAssassin selected from the drop down list.
I guess "Ongewensteberichtkopregels vertrouwen die zijn ingesteld door: <filtername>" is the Dutch translation for "Trust junk mail headers for: <filtername>" or something alike.