229878 - When known junk message is filtered to Junk folder, it never gets marked as junk

Status: RESOLVED DUPLICATE of bug 181631

(MailNews Core :: Filters, defect)

Description

[Giles Puckett]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko/20031007
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko/20031007

A corporate mail gateway marks spam by prepending [SPAM] to the title.
When a filter is constructed to put all these in the Junk folder,
they never get marked as junk, no matter how well-trained the
Bayesian filtering is. If the filter action is removed, so they
stay in the Inbox, the messages get junked correctly.
(Sort of the opposite of 217682 - i *do* want the junk filter
to work after filtering, and if Mark As Junk was available as 
a filter action (181631) this bug would be more critical.)

Something to do with the fact that messages don't get junked
until you visit their folder? (204483, 200778)

Reproducible: Always

Steps to Reproduce:
1. Set up filter: Title begins with [SPAM] ==> junk folder.
2. Allow messages to arrive and get moved to junk folder.
3. Note that they are never marked as junk, no matter how
   much training you give it.
4. Remove the filter.
5. Allow more messages to arrive; note that the [SPAM] ones
   are now correctly marked as junk.

Actual Results:  
When messages get filtered to Junk, they never get marked as junk.

Expected Results:  
Should have marked as junk.

Comment 1

[Mike Cowperthwaite]

IMAP or POP mail?  If IMAP, see (possibly related) bug 206679.

I think you mean "if Mark As Junk was available as a filter action ... this bug 
would be LESS critical," no?

Are you perhaps encountering bug 219975?  This would be if you have the Junk 
folder selected for view at the point when the messages arrive and are filtered 
into the Junk folder.

Comment 2

[Giles Puckett]

In answer:

It's for POP mail.

"if Mark As Junk was available as a filter action..." - fixing this bug 
would then be more usefull, because I'd like to kick the junk filter off with
known junk bugs, and gradually have the filter take over - but that's 181631,
another story.

It's not quite the same as 219975, because the junk folder is not selected.
When I do select it, nothing gets marked as junk then, either.

Leaving the filter action out is an adequate workaround, but it would
be nice if the junk folder was treated consistently with others in that
junk mail is found and marked. (BTW, why does junk filtering only kick 
in when you visit that folder, after all mail has been downloaded?
Or is this a silly question betraying my lack of internal knowledge?)

Comment 3

[Mike Cowperthwaite]

I'll add Daniel to the CC list and see if he can answer that question.

Comment 4

[Daniel de Wildt]

Some folders like drafts, templates, sent and also the spam folder self are
never checked for spam. Sorry, but so is the implementation:
http://lxr.mozilla.org/mozilla/source/mailnews/base/util/nsMsgDBFolder.cpp#1757

Comment 5

[Daniel de Wildt]

> BTW, why does junk filtering only kick 
> in when you visit that folder, after all mail has been downloaded?
> Or is this a silly question betraying my lack of internal knowledge?

Sorry, but this is no forum.
(I don't why this is so implemented, but the junk filter is assigned to the
folders. So it could make sense to run the filter for the folder if it is
necessary.) Maybe someone in one of the irc channels could help you with this
question: irc://irc.mozilla.org/

To this bug:
It seems that it's a design issue that some folders shouldn't be checked for
spam. This is implemented by the patches for bug 197149 and bug 204807.
There might be two possible solutions with those filterrules:
1) Marking the mail as spam in the your message filter (bug 181631)
2) All mail moved to the junk folder should be automatically marked as junk.

This bug is identical to bug 218464, which is marked as dup of 181631.
Please reopen this bug with a corresponding summary if you want to see that 2)
should be implemented.

BTW, why moving the mails with a message filter?
The junk filter will detect such mails if you train it adequate.
The argument that the junk filter should be automatically trained for not
detected junk mail will not work, because ONLY manual marked junk mail is
trained (bug 197430).
You can move your not detected junk mails to a temporary folder (not the junk
folder) as a work around. Then train (once a day or week) the junk filter on
those catched mails and the junk filter should be trained well after a few days
(or weeks).

*** This bug has been marked as a duplicate of 181631 ***

Comment 6

[Mike Cowperthwaite]

I don't agree that this is "identical" to bug 218464 -- that bug requests that 
if a filter moves the message to the junk folder then it should be marked as 
junk automatically; this bug is about the junk mail controls not being run (at 
least not automatically) on the Junk folder.  However, I do agree that this 
bug's issue could be solved by implementing bug 181631.

A question: if you select   Tools|Run Junk Mail Controls on Folder
with the Junk folder open, does it *then* mark the messages as junk?

Comment 7

[Daniel de Wildt]

Mike: OK, "nearly the same" is the better description. The expected solution in
both bugs is the same. The mails "Should have marked as junk.". This is the
expected result of this bug and also a part of the expected result of bug
218464. That's the reason why I used "identical". 
I confirm that the automical training is not part of this bug. (Is bug 197430)

To your question:
Yes, the junk controls will work on the folder if I start it manual. I don't
know exactly why (I think it shouldn't), I will try it tomorrow again with a
self compiled build.

Comment 8

[Daniel de Wildt]

Found the reason why a manual run of junk control will work on the spam
directory. The important part of the junk filter is in
http://lxr.mozilla.org/mozilla/source/mailnews/extensions/bayesian-spam-filter/src/nsBayesianFilter.cpp
(these are the ClassifyMessage(s) methods). These methods are called from the
nsMsgDBFolder.cpp (see comment #4) but also from javascript. 

So javascript has it's own filter logic but uses the same native junk filter
routines. You will find the javascript routines in the "messenger.jar"
The "mail3PaneWindowCommands.js" calls "analyzeFolderForJunk()" if you run the
Junk Mail Controls manually.
The "mailCommands.js" contains the "analyzeFolderForJunk()" and whole logic
which mails should be checked. "analyze()" calls the native junk filter
"classifyMessage". The connection between the javascript junk filter and the
native junk filter is made in "getJunkmailComponent()":
http://lxr.mozilla.org/seamonkey/source/mailnews/base/resources/content/mailCommands.js

Note to step 3 in bug description:
Running the Junk Mail Controls manually on the spam folder should mark those
mails as junk (if the filter is trained well).

BTW, this inconsistency of the implemented logic in "nsMsgDBFolder.cpp" and in
"mailCommands.js" IS a bug IF they should be the same.