355197 - Junk filters do not activate when receiving new mail without training.dat

Status: NEW

(MailNews Core :: Filters, defect)

Description

[Andrew Schultz]

With linux seamonkey CVS trunk build (with bug 351545 fixed) if my profile has no training.dat file, new mail is not classified as junk/not-junk.  Selecting "Run Junk Mail Controls on Folder" does work.  The BayesianFilter NSPR logging generates only this line when receiving new mail:

junk probability threshold: 0.900000

If I manually mark something as junk, it starts working (because it generates the training.dat apparently)

Comment 1

[Andrew Schultz]

ah, bug 250084...

I guess part of what confused me was that invoking the junk filters explicitly via Tools classified everything as junk.  Perhaps if the code at

http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/mailnews/extensions/bayesian-spam-filter/src/nsBayesianFilter.cpp&rev=1.59&mark=1095,1100#1079

were switched to first return not-junk if there aren't bad tokens it would at least be consistent (and the code from bug 250084 could be removed unless I'm missing something).

The other thing that made me think this was a bug was that there was very little UI anywhere to get me to mark anything as junk (might be SM-only).  I opened the junk controls panel and set various prefs (and it was already enabled), but downloading mail did absolutely nothing.  When I invoked junk controls via the tools menu I did get the first-time junk mail dialog, which was what I was expecting from somewhere (in the absence of overly-aggressive junk mail marking).

Comment 2

[Wayne Mery (:wsmwk)]

dupe of bug 250470?

Comment 3

[dcam]

Junk mail does not transfer automatically -  It is marked as junk and just stays there and does not transfer to junk mail folder.  version 2.0.0.14 (20080421)

it can be moved manually

Comment 4

[Thomas D. (:thomas8)]

I'm quite sure to confirm this bug for
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091121 Thunderbird/3.0 (rc1build3)

Only when I manually marked a msg as junk for the first time ever, TB started the junk filter on other mails, too. Where other msgs might be only the newly arrived msgs. I noticed because it junked the wrong mails... :(

Comment 5

[Kent James (:rkent)]

I don't understand what this bug is supposed to be about, and what a "fix" would look like. If there is no training data, then of course the bayes filter is disabled - what other choice is there? Is the point to also disable manual Run Junk Controls?

Comment 6

[Andrew Schultz]

> If there is no training data, then of course the bayes filter is disabled

This was exactly the problem.  I explicitly went into the various settings to enable and/or verify that junk filtering /was/ enabled.  And yet those settings had no effect because there was no training.dat.

> Is the point to also disable manual Run Junk Controls?

No, the problem was just the inconsistent behavior which added to my confusion.  Junk Controls were effectively disabled in one context but were not disabled in another context.  Were the filters enabled and just thought all my mail was not-spam?  No, because when I run it manually, it thinks there is spam.

Comment 7

[Kent James (:rkent)]

(In reply to comment #6)
> > If there is no training data, then of course the bayes filter is disabled
> 
> This was exactly the problem.

If the "problem" this bug is describing is "If there is no training data, ... the bayes filter is disabled" then this bug is invalid, as it has to be that way.

>  Junk Controls were effectively disabled in one context but were not disabled
> in another context.  Were the filters enabled and just thought all my mail was
> not-spam?  No, because when I run it manually, it thinks there is spam.

If the bug is that you cannot see if and how the junk filters are working, then you are preaching to the choir with me. One of the main points of my JunQuilla extension is to show the working of the junk filters on a per-message basis.

> > Is the point to also disable manual Run Junk Controls?

> No, the problem was just the inconsistent behavior which added to my confusion.

If your issue is that behavior is inconsistent, then why do you deny that making the Run Junk Controls work consistently with the incoming junk filter is the purpose of this bug? You really need to define what it is that would satisfy your issue, beyond "I am confused, please make me unconfused." That requires something specific that we could change that would make things work more appropriately for you.

Comment 8

[Andrew Schultz]

> it has to be that way.

Technically it doesn't.  The code behavior in the absence of training.dat is not set in stone, as evidenced by the action when running junk controls manually.

> If your issue is that behavior is inconsistent, then why do you deny that
> making the Run Junk Controls work consistently with the incoming junk filter is
> the purpose of this bug?

The purpose of this bug is to make initial junk controls behavior consistent and clear.  Changing the manual behavior would be more consistent but not relieve much of my confusion.

> You really need to define what it is that would satisfy your issue, beyond "I
> am confused, please make me unconfused."

How you go about unconfusing me is up to you or whoever wants to take this bug.  I've explained why I was confused.  If the behavior isn't going to change (something like bug 179999), there should be UI somewhere that informs the user that they need to mark spam as spam to actually get the junk mail controls functional.

Comment 9

[David :Bienvenu]

(In reply to comment #8)
> there should be UI somewhere that informs the user
> that they need to mark spam as spam to actually get the junk mail controls
> functional.

There used to be such UI - is it not there anymore, or does it not get put up in all situations the junk filter is run?

Comment 10

[Wayne Mery (:wsmwk)]

I would expect the UI gets put up only the first time you enable junk in a given profile

Comment 11

[David :Bienvenu]

Maybe it needs to get put up if you try to manually run the junk controls?

Comment 12

[Andrew Schultz]

> There used to be such UI - is it not there anymore, or does it not get put up
> in all situations the junk filter is run?

I get the dialog when I try to run the controls manually.  It certainly should be given at that time.  It'd be nice to have it appear the first time mail appears (it doesn't), but it would rather intrusive, especially if someone does not care about the junk filters -- they'd ignore it and then later when they might care, there's no dialog.  I think in a perfect world, something like the notification bar would exist in this situation that could inform the user that some extra functionality exists

(In reply to comment #10)
> I would expect the UI gets put up only the first time you enable junk in a
> given profile

We have it enabled by default, so if we did this, the user would just see it when they made their initial account (again, not the most useful time).

Comment 13

[Wayne Mery (:wsmwk)]

xref "If training.dat is missing "Run Junk Mail Controls on Folder" marks all messages as junk", esp Bug 383873 comment 4