Open Bug 211826 Opened 21 years ago Updated 13 years ago

[junk] Junk detection runs on old folders when opening them

Categories

(MailNews Core :: Filters, defect)

x86
Linux
defect
Not set
critical

Tracking

(Not tracked)

People

(Reporter: BenB, Unassigned)

References

(Depends on 1 open bug, Blocks 1 open bug)

Details

(Keywords: dataloss)

Reproduction:
1. Set junk mail to be moved to junk folder
2. Open an IMAP old folder that you never opened (since junk mail was enabled?
with that profile?)
3. After a while, go to the junk folder

Actual results:
After step 2, you see activity in the status bar (the same as when you "mark" an
existing folder "Junk" or "Not Junk", i.e. every message is downloaded and each
causes the progress bar to run from 0% to 100% within an instant)
After step 3, you see that a number of messages were moved from the old folder
to the junk folder, some of them marked as junk, some (some of them marked as
junk, some not) are actually not junk.

Expected result:
Nothing.

Severity:
This is potential data loss, because I don't expect junk filters to run on old,
existing folder, only incoming mails, so I probably don't check the junk folder.
Add the facts that
- these folders are likely to be already filtered (possibly manually) for spam
- they contain a huge number of mails, and I have no time to check the junk mail
detection's choices for all these
- the junk mail detection did a poor job on that particular folder, with 5-10
false hits out of 2000 mails in the folder.
Duplicate of Bug 198961 ?
Not quite. This isn't new, filtered mail, but old, archived mail.
The issue is the essentially same, "non-junk" mail that has been previously and
deliberately moved (old or new, manually or by a filter) is being marked as junk
and potentially deleted without warning. Some of the other bugs that have been
marked duplicate of Bug 198961 are other variations on the same basic flaw. 

If it's not a duplicate of 198961 then it's a duplicate of Bug 200496 (which I
just marked as a duplicate of 198961 earlier today). 
I think this bug and bug 200496 are the same issue, and that issue is a problem: 
mail that is not new is still being subject to junk filtering.  This is a 
distinct case from mail that *is* new, which by design is subject to junk 
filtering regardless of which folder it resides in.  (The issue of whether that 
design is in fact wrong is bug 200788.)  (There are a few useful comments in 
200496.)

Neither is a dupe of bug 198961, which is quite rightly listed as an 
enhancement; and fixing that one doesn't necessarily address this problem, since 
it is focusing on filtered mail.

Bug 189970 is similar to 198961, but fixing it could address this bug, or at any 
rate the most common cases of it, at least if the defaults were set right.  It 
still wouldn't address the root symptom of this bug.
FYI, I don't remember if the "detected" mails were marked unread or not.
(unread is different from new)
Product: MailNews → Core
Mike in comment #5:
> I think this bug and bug 200496 are the same issue, and that issue is a
> problem: 
> ...
> Neither is a dupe of bug 198961, which is quite rightly listed as an 
> enhancement; and fixing that one doesn't necessarily address this problem,
> since it is focusing on filtered mail.

in retrospect is Joseph's dup done in comment 4 incorrect?
or, is this rightly a dup of bug 198961?

Ben?
Depends on: 189970
I wonder whether the fix for bug 329569 / bug 200788 has addressed this issue.

I don't agree that 189970 blocks this bug.
Assignee: sspitzer → nobody
QA Contact: laurel → filters
Product: Core → MailNews Core
(In reply to comment #5)
> I think this bug and bug 200496 are the same issue, and that issue is a
> problem: 
> mail that is not new is still being subject to junk filtering.  This is a 
> distinct case from mail that *is* new, which by design is subject to junk 
> filtering regardless of which folder it resides in.  (The issue of whether that 
> design is in fact wrong is bug 200788.)  (There are a few useful comments in 
> 200496.)

This sure sounded like 200496 and I don't quite understand why it was duped to bug 198961.  Regardless, bug 198961 comment 35 posits the solution is bug 189970 (which is probably why I set this bug as dependent)

bug 200788 got duped to fixed bug 329569 as pop related.

So I'm still left wondering, why did Ben's imap folder get reprocessed? 

Can anyone who has seen this problem clearly say it is gone for them?
(scary question given so few people cc to the bug)
I've still seen that (comment 0) rather recently.
(In reply to comment #10)
> I've still seen that (comment 0) rather recently.

can you clarify 
- the state of the messages in the folders affected?  i.e. are they read, unread, new?  
- where the folders are in the account hierarchy? 
- is clicking on junk folder a necessary step?
- is it random? is there a time delay before the message start moving?
> the state of the messages in the folders affected?  i.e. are they read,
> unread, new?

I don't remember.

> where the folders are in the account hierarchy?

Parse error.
where the folders in the account hierarchy? - Yes
where are the folders in the account hierarchy? - Anywhere, just a normal subfolder.
Cyrus server, if that matters. (IMAP of course.)

> is clicking on junk folder a necessary step?

No (as described by Actual Results).

> is it random? is there a time delay before the message start moving?

It's as random as the Junk filter is.
The Junk filtering starts reproducably (as far as you can find relevant folders which TB (your profile) didn't see yet).
(In reply to comment #12)
> > the state of the messages in the folders affected?  i.e. are they read,
> > unread, new?
> 
> I don't remember.

OK. Are the messages in these folders normally in a not new state?


> > is it random? is there a time delay before the message start moving?
> 
> It's as random as the Junk filter is.
> The Junk filtering starts reproducably 

I just want to be clear - it always happens, every time you enter a folder that you have never touched since junk filtering was turned on?
As I said, the Junk filter wrongly kicks in reproducibly, in the same situation. As I said, I don't remember whether the msgs were new or not.
Blocks: 517849
You need to log in before you can comment on or make changes to this bug.