239349 - Junk Filter incorrectly marks mails as junk, no junk icon in summary pane, junk icon in header pane (IMAP only?). "Delete mail marked as junk" deletes legitimate e-mail (Dataloss)

Status: RESOLVED FIXED

(Thunderbird :: Mail Window Front End, defect)

Description

[Peter Lairo]

User-Agent:       Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.7b) Gecko/20040329 Firefox/0.8.0+
Build Identifier: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.7b) Gecko/20040329 Firefox/0.8.0+

Junk Filter incorrectly marks mails as junk, no junk icon in summary pane, junk
message in header pane (IMAP only?)

Reproducible: Always
Steps to Reproduce:
1. Send an e-mail to yourself. Receive it.
2. Select the e-mail.

Actual Results:  
1. No Junk icon in sumary pane next to e-mail.
2. Header pane shows that message is considered junk.
3. Selecting "Delete Mail marked as junk" deletes this (valid) e-mail (DATALOSS).

Expected Results:  
1. e-mail should not be classified as junk (my own e-mails should be recognized
by now - extensive training).
2. e-mails Thunderbird (even when incorrectly) thinks are junk should be
correctly marked with the icon in the subject pane.

Comment 1

[Peter Lairo]

Attachment: Screenshot showing this bug.

Comment 2

[Peter Lairo]

Damn typo (s/markes/marked). Sorry for bugspam. :-[

Comment 3

[Thomas Winzig]

I can confirm that I have been seeing the same problem since at least the
3/26/2004 build, although I cannot reproduce it every time as the submitter can.

I have seen it happen only twice, and only on HTML emails.

Comment 4

[Peter Lairo]

I just received an HTML e-mail from my New York Times subscription (which I've
been receiving for years) that was incorrectly marked.

I noticed that e-mail with the "junk message" in the header are still rendered
as HTML (although I have it set to "sanitize" HTML). 

So in summary, the situation is:
1. Valid e-mail (HTML only?) is being treated as junk (internally? Where?).
2. Subject Pane: The junk icon does NOT appear.
3. Header Pane:  The junk "message" DOES appear.
4. HTML e-mail with the "junk message" in the header pane is not sanitized.
5. "Delete messages marked as junk" deletes these (valid!) e-mails (DATALOSS).

Comment 5

[Thomas Winzig]

I just had this happen to a plain-text message, so it is not limited to HTML,
although I see it most often with HTML messages...

Luckily I do not have the delete junkmail option turned on, so no data loss for
me yet. However, I do have tbird set to move junk mail into the junk folder in
another account, and sometimes it does not do it, it just marks it as junk. In
the context of this bug, for example, I have never seen it move one of these
"phantom" junk mails into that folder...

Comment 6

[Scott MacGregor]

if anyone figures out a pattern to reproducing this please let me know.

The next time it happens, try opening the message in a stand alone message
window. Does it havea  junk bar there when you reload it?

If you quit and restart, click on the same message, does it still show a junk bar?

Comment 7

[Thomas Winzig]

Ironically, the message I got from bugzilla with Scott's latest update to this
bug was phantom flagged as junk!

I opened it in a new window, still shows the junk bar. 

I switched to a different message in the summary pane, and then back, and the
junk bar was still there. I closed tbird and restarted it, highlighted the
message, and the junk bar is still there. 

I will leave the message in my inbox in case you want me to try something else.

Comment 8

[Thomas Winzig]

One pattern, for me at least, is that all messages that have received this
phantom flag contain a header that my mail server has added:

X-Whitelisted: 

Our server adds this header along with other info after it if it determines that
the message is from someone not likely to be sending me spam...

Comment 9

[Scott MacGregor]

are any of you using the new mail filter code for marking mail as junk / not junk?

I wonder if that's related somehow

Comment 10

[Thomas Winzig]

A-ha... I think you may have hit on something.

I have some filters in place, but none of them have that junk setting stuff
turned on. HOWEVER, I just checked, and my filters do have the drop-down
selection on that option set to "Junk" (I guess that is the default). Yet I have
not turned the checkbox on.

And it just so happens that the mail I have received that has this phantom junk
setting turning on ARE being processed by the filters.

So I am guessing that new filter option is partially applying a junkmail setting...

Comment 11

[Thomas Winzig]

That is definitely the problem. I can now reproduce it every time when I send a
message that will have one of my filters applied to it. 

The problem is the junk filter stuff is being semi-applied even without the
checkbox enabled. The only workaround is to enable it and mark it as notjunk. If
I change the drop-down to "NotJunk" but leave that action disabled, it reverts
back to "Junk."

Comment 12

[Scott MacGregor]

i don't know what made me think of that. That will help a lot in narrowing this
down.

Comment 13

[Scott MacGregor]

Attachment: the fix (add some missing break statements)

If you had any filter who had an action that set a label value, we also ended
up trying to set junk status on it because of a missing break statement after
processing the label action

Comment 14

[Scott MacGregor]

Attachment: the fix (add some missing break statements)

If you had any filter who had an action that set a label value, we also ended
up trying to set junk status on it because of a missing break statement after
processing the label action. This issue was compounded by the fact that we
would randomly mark it as junk or not junk because of an unitialized variable
in the filter action code for setting the junk score. So sometimes you saw it,
sometimes you didn't. :)

Comment 15

[Scott MacGregor]

Comment on attachment 145364 [details] [diff] [review]
the fix (add some missing break statements)

woops added the attachment twice

Comment 16

[Scott MacGregor]

Comment on attachment 145365 [details] [diff] [review]
the fix (add some missing break statements)

this bug is also present in seamonkey. I'll email for 1.7 final status. This is
a pretty severe bug that should get fixed for 1.7 as it causes data loss.

Comment 17

[chris hofmann]

a=chofmann for 1.7

Comment 18

[Scott MacGregor]

fixed