289011 - Messages marked as "not junk" should indicate this while training

Status: NEW

(Thunderbird :: Mail Window Front End, enhancement)

Description

[eric gorr]

User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.6) Gecko/20050317 Firefox/1.0.2
Build Identifier: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.6) Gecko/20050317 Firefox/1.0.2

While training junk mail detection, it would be useful if messages specifically
marked as "not junk" by the user would show a special icon in the Junk Status
column - perhaps a halo. 
 

Reproducible: Always

Comment 2

[eric gorr]

This is still a feature I would like to see implemented.

Comment 3

[jwq]

This is something I too would like to see. Could someone with the appropriate privs please confirm this RFE?

Comment 4

[Kent James (:rkent)]

This would be fairly simple to implement as an extension, even with the TB 2.0 code. With the improvements to junkstatusorigin in progress in bug 366491, the concept could be extended to include indication of setting by filter, whitelist, or imap flags as well. Whitelist in particular is a very common way that junk status is set, and is valuable to monitor if you are interested in junk processing.

I think it's a legitimate request, so I'll confirm this as an RFE. 

Comment 5

[Kent James (:rkent)]

Attachment: Use green circle (same as unread) for nonjunk

This patch essentially reverses the decision made in bug 182386 to make "unknown" and "not junk" appear the same. In the junk column, when an email is classified as non junk by any means, the marking goes from a small dot (as at present) to a slightly larger green circle. These are the same icons used in the read/unread column. Screenshot attachment to follow.

I justify this as follows (in addition to the reporter's comments):

1) When a user takes an action, in this case manually marking a message as not junk, there should be some visible change in the UI.

2) In the initial stages of getting junk mail filtering working, currently the user has no feedback at all that it is doing anything for messages that are marked as not junk. For the bayesian junk filter to work, the user must interact with it a little for training. He or she needs some encouragement that something is happening.

3) There is currently no way for the user to tell the difference between a state where the junk mail filter is turned off, or when it is marking all messages as good. This provides that feedback.

I had originally done this patch for bug 209898, but then I noticed that was a Seamonkey bug.

I intend to do an extension that would add additional icons to the junk column that would also show the reason the email was marked - by whitelist, by bayes, by user, etc.

Comment 6

[Kent James (:rkent)]

Attachment: Shows the junk status column with three states

Comment 7

[Dan Mosedale (:dmosedale, :dmose)]

Adding clarkbw for his thoughts here.  Comments 0-4 don't provide much insight into what problem we'd be specifically solving by doing this, so it feels like we're positing a solution to problem that isn't very well defined, which makes it difficult to reason about.  Comment 5 has some discussion here, so I'll respond to that:
 
> 1) When a user takes an action, in this case manually marking a message as not
> junk, there should be some visible change in the UI.

If a message is marked as junk, and the user changes that, there already are two visible changes in the UI: the spam icon goes away, and the green info bar does as well.  If the message is already marked as not junk, why would it ever occur to the user to mark it as not junk?

Is the idea here that we're attempting to add a state to the UI in order to create that motivation?

> 2) In the initial stages of getting junk mail filtering working, currently the
> user has no feedback at all that it is doing anything for messages that are
> marked as not junk. For the bayesian junk filter to work, the user must
> interact with it a little for training. He or she needs some encouragement 
> that something is happening.
>
> 3) There is currently no way for the user to tell the difference between a
> state where the junk mail filter is turned off, or when it is marking all
> messages as good. This provides that feedback.

It sounds to me like these are two ways of saying that high-level goal here is to try and convince the users to do more training work than they're likely to do given the current UI's lack of feedback, because that's likely to make the adaptive spam filters perform better.  Is that a reasonable summary?

Comment 8

[Kent James (:rkent)]

(In reply to comment #7)
> 
> It sounds to me like these are two ways of saying that high-level goal here is
> to try and convince the users to do more training work than they're likely to
> do given the current UI's lack of feedback, because that's likely to make the
> adaptive spam filters perform better.  Is that a reasonable summary?
>

This is not the change that is needed to "convince the users to do more training". That would be adding an Uncertain folder which displays messages that fall within a certain range of junk percent, for example 10-90 as I run. From the discussions that I have had so far with mailnews leads, there is little interest in that feature however. Since it can be easily implemented now in an extension, I'll eventually do an extension that allows that, as IMHO it is essential for proper usage of the Bayes filter.

The current change is simply trying to give the user a little more feedback that the Bayesian filter is actually doing something. The motivations are for diagnostic purposes, and to show some responsiveness in the program. The change is supposed to be very subtle.

Comment 9

[Dan Mosedale (:dmosedale, :dmose)]

(In reply to comment #8)
> 
> This is not the change that is needed to "convince the users to do more
> training". That would be adding an Uncertain folder which displays messages
> that fall within a certain range of junk percent, for example 10-90 as I run.
> From the discussions that I have had so far with mailnews leads, there is
> little interest in that feature however. Since it can be easily implemented now
> in an extension, I'll eventually do an extension that allows that, as IMHO it
> is essential for proper usage of the Bayes filter.

I actually think there's a pretty high value in that, and I look forward to using your extension and perhaps arguing for it's inclusion in the core after I've used it a bit.

> The current change is simply trying to give the user a little more feedback
> that the Bayesian filter is actually doing something. The motivations are for
> diagnostic purposes, and to show some responsiveness in the program. The change
> is supposed to be very subtle.

OK, thanks for the clarification.

Comment 10

[Dan Mosedale (:dmosedale, :dmose)]

Comment on attachment 337707 [details] [diff] [review]
Use green circle (same as unread) for nonjunk

This needs Bryan to make a ui-review call on it before I can do a code review; removing it from my review queue for now.  Please re-nominate if/when it gets ui-review+; I'll also ping Bryan via email to make sure it's on his radar.

Comment 11

[Dan Mosedale (:dmosedale, :dmose)]

It just now occurred to me that perhaps "marked as non-spam" is what the blue sun in JunQuilaX is intended to convey?  I've been running with JunQuilaX for a few months now, and I've never quite figured out what that meant.  

Assuming that's the case, it's certainly possible that I'm more dense than normal w.r.t. to this UI, but I rather suspect that many people would have a similar issue inferring what a green dot meant here as well.

Comment 12

[Kent James (:rkent)]

(In reply to comment #11)
> It just now occurred to me that perhaps "marked as non-spam" is what the blue
> sun in JunQuilaX is intended to convey?
>

I'm not sure what you mean by the "blue sun" - I usually use the same icon as "Unread" to indicate "Marked as not junk in any way", but there is no blue sun in my Windows versions - I've only seen green dots, as in my proposal here. Perhaps you are using a different theme? In any case, icons are not great at communicating status, as it's not easy to pick something that is subtle yet meaningful.

I don't really have strong feelings about whether this bug is accepted or not. I personally disagree with the decision done in bug 182386, and this bug is an opportunity to undo that. But that is easily done in an extension as well, which is what I have done in JunQuilla (http://mesquilla.com/extensions/JunQuilla or https://addons.mozilla.org/en-US/thunderbird/addon/9886) among other things.

The Bayesian filter requires some care and feeding by the user to keep working well. JunQuilla provides enough information for the user to provide appropriate training, while the current TB interface does not. (The missing piece is primarily the Uncertain folder though, not the icons which are the subject of the current bug). Your statement in comment 7 "If the message is already marked as not junk, why would it ever occur to the user to mark it as not junk?" is the heart of the issue. But you can make a good argument that server-based junk filters are really what most people are currently using, and the research that I have seen show that a server-based solution is likely to outperform a client-based solution anyway. So perhaps the complex interface required to really support the manual training needed for the Bayes filter to work belongs in an extension anyway.

(For the record, "JunquillaX" was a modified version of a prototype of JunQuilla, where additional columns were added to support debugging of the gloda database. JunquillaX was distributed privately to a few individuals. The gloda functionality was later updated, separated, and made a separate extension, named GlodaQuilla, and available at https://addons.mozilla.org/en-US/thunderbird/addon/9873  It was not intended to be a demonstration of spam management using JunQuilla).

Comment 13

[Dan Mosedale (:dmosedale, :dmose)]

(In reply to comment #12)
> I'm not sure what you mean by the "blue sun" - I usually use the same icon as
> "Unread" to indicate "Marked as not junk in any way", but there is no blue sun
> in my Windows versions - I've only seen green dots, as in my proposal here.
> Perhaps you are using a different theme? In any case, icons are not great at
> communicating status, as it's not easy to pick something that is subtle yet
> meaningful.

Ah, that's a "mac theme" vs. "windows theme" issue indeed.  Yeah, picking icons is absolutely a Hard Problem.

> But you can make a good argument that server-based junk
> filters are really what most people are currently using, and the research that
> I have seen show that a server-based solution is likely to outperform a
> client-based solution anyway. So perhaps the complex interface required to
> really support the manual training needed for the Bayes filter to work belongs
> in an extension anyway.

From a purely architectural standpoint, there's a lot to be said for having all filtering done on the server.  Part of the win of Thunderbird, though, is that lots of ISPs don't seem to do a good enough job of spam-filtering, and this helps with that problem.  You're exactly right that we need to figure out the right balance of user-experience simplicity vs. spam-filtering power here.  I suspect Bryan is likely to have opinions on this.

Comment 14

[Bryan Clark (DevTools PM) [:clarkbw]]

I'm really hesitant about this change, I'll try my best to explain why.  Currently, without the green state, users mark the messages they feel are junk and then remove those messages from the view.  All other messages are, possibly wrongly, assumed to be not junk.  However with the addition of the green state we're providing the user with the knowledge that not all mail is "not junk" as much as mailnews has no opinion on some of the messages.  Either people will understand what the green state means and want to change all their messages into that state, or they won't understand what the green state means and will likely be confused by the difference shown between the "not junk" and "unknown" messages.

In the screenshot provided you can see all three changes yet without the text I don't believe it's clear what exactly each state means.  Also since we're sharing with the icon with unread it's likely to be more unclear.

This definitely makes sense as an extension, but as a standalone checkin the consequences are a bit larger than I'm comfortable with.  Perhaps excellent  documentation would be the thing that makes this better.

Comment 15

[Kent James (:rkent)]

Since I can and do provide this functionality in an extension (JunQuilla), and it seems to generate so little enthusiasm among the official product drivers, I think I will just abandon this effort for now.

Comment 16

[Wayne Mery (:wsmwk)]

Attachment: screen shot of "J" and X'd "J"

(In reply to Bryan Clark [:clarkbw] from comment #14)
> I'm really hesitant about this change, I'll try my best to explain why. 
> Currently, without the green state, users mark the messages they feel are
> junk and then remove those messages from the view.  All other messages are,
> possibly wrongly, assumed to be not junk.  

yes, wrongly

> However with the addition of the
> green state we're providing the user with the knowledge that not all mail is
> "not junk" as much as mailnews has no opinion on some of the messages. 
Correct, indicating all states eliminates confusion or opportunity for incorrect interpretation.  Much like the "Read" column never has a blank row, read messages explicitly have a small green ball. 

> Either people will understand what the green state means and want to change
> all their messages into that state,
What's bad about that?

>  or they won't understand what the green
> state means and will likely be confused by the difference shown between the
> "not junk" and "unknown" messages.
Can it be worse than current situation?

> In the screenshot provided you can see all three changes yet without the
> text I don't believe it's clear what exactly each state means.  Also since
> we're sharing with the icon with unread it's likely to be more unclear.
We can certainly overcome that. 

> This definitely makes sense as an extension, but as a standalone checkin the
> consequences are a bit larger than I'm comfortable with.  Perhaps excellent 
> documentation would be the thing that makes this better.

Attached is what I currently use, adapted from someone else's work, using the J character rather than icons.

Comment 17

[Magnus Melin [:mkmelin]]

(In reply to Wayne Mery (:wsmwk) from comment #16)
> > Either people will understand what the green state means and want to change
> > all their messages into that state,
> What's bad about that?

It creates a task for the user, when software should do the opposite whenever possible.

Comment 18

[Wayne Mery (:wsmwk)]

(In reply to Magnus Melin from comment #17)
> (In reply to Wayne Mery (:wsmwk) from comment #16)
> > > Either people will understand what the green state means and want to change
> > > all their messages into that state,
> > What's bad about that?
> 
> It creates a task for the user, when software should do the opposite
> whenever possible.

and yet, this is the way junk works now - it works best when you continue to train it. And the UI is not explicit on what messages could use action - the user is left high and dry to guess which messages that are not flagged in the UI are not junk versus which ones are questionable.

Comment 19

[Wayne Mery (:wsmwk)]

https://wiki.mozilla.org/index.php?title=Thunderbird:Folder_Pane&diff=0&oldid=101868#Possible_Junk.3F suggests adding a "possible junk" folder, much as junquilla " adds an Uncertain folder for messages needing user decision."