482648 - Concept proposal for better Junk Mail controls/UI/Filters

Status: RESOLVED INCOMPLETE

(Thunderbird :: Preferences, enhancement)

Description

[klonos]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2a1pre) Gecko/20090305 Minefield/3.2a1pre (.NET CLR 3.5.30729)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b2pre) Gecko/20081113 Shredder/3.0b1pre

First of all let me say that I am new to bug reporting, so if this belongs somewhere else, or I incorrectly use the '[Meta]' tag, feel free to take this over and edit as you like or break it to smaller bugs/feature requests.

I've been tracking a lot of bugs/feature requests concerning the way Thunderbird handles Junk Mail and/or the way controls/options for this are presented to the user. I see that a lot of people have very nice ideas or suggestions that are backed up by legit reasons. These bugs should be marked as 'Depends on' of this bug (again if I understood correctly the use of '[Meta]'). Also let me say that some people agree with ideas others present, while others seem not to. Everyone has a reason, so what I suggest as the best solution to satisfy all is *Options for everything*. This way, if one doesn't like an option while someone else requires it... first one disables it, the other leaves it enabled (if enabled is the default, otherwise it's the way around).

So, this is sort of a summary of this research of mine + some features I have seen in other mail clients + some suggestions of my own. Here it goes...

1. The 'Privacy' section in the UI options should perhaps be renamed to 'Security', since all tabs under it are security-related. Or even better change to 'Privacy & Security'.

2. The 'Junk' tab should be renamed to 'Global Junk Mail settings'. To clearly distinct it from the 'Junk Mail settings' under each account (per account specific settings).

3. There should be a global bayesian storage as well as one for each account.

3a. Insert a checkbox under the 'Global Junk Mail settings' that would enable or disable tree more options (radio-box):

[x] 'Enable Global spam learning engine'
    (o) 'Use separate rules for each account'
    ( ) 'Use global rules for all accounts'
    ( ) 'Allow the use of both a global and a per account set of rules (Advanced)'

The first option (checkbox) would do exactly what it says, or the user could simply prevent tb from checking mail for spam by not enabling this option at all. I suggest this should be enabled by default and if a user disables it (some people rely on their mail server's antispam solution), there should be a message warning the user of the risks taken by this action.

For the second set of options (radio-box), I suggest per account rules as default. You see, most of us have more than one mail accounts. Some are for professional use, while others are for personal. We all receive joke-mails from friends containing words like 'sex' or 'viagra' or some sort of profanity in our personal mail. We don't want them marked as spam and we don't want the bayesian filter excluding such words from mail received in our professional mail account. The best solution here is per account rules. Please let me know your thoughts on this.

3b. A listbox of all mail accounts should follow, each one having three checkboxes next to them (before or after). This would be a preview of which accounts have their spam learning engine on and which off (per account disable of filtering) and also allow the user to change this setting per account (first checkbox). The second checkbox would allow spam found in the account's mail to be 'passed' to the Global spam engine or not, thus allowing the global engine to learn from this account's spam or not. The third checkbox would allow the account's mail to also be checked against the global engine's bayesian filter rules (or not).

4. All account settings should change to a tab style instead of a tree-like structure... but this suggestion is a bit off-topic. Anyways, under each account's 'Junk Mail settings', there should be a 'Enable spam learning engine for this account' checkbox. This would be on by default if the 'Enable Global spam learning engine' mentioned in suggestion #3a above is also enabled. This would also reflect the users' choice in the first checkbox of the listbox mentioned in suggestion #3b and vise-versa. More checkboxes and radio-boxes should provide controls for the rest of the options in 3a and 3b and also reflect users' choices made in tb options panel.

5. There should be a 'Learn spam/not spam...' wizard (a button in tb 'Global Junk Mail settings' and in the 'Junk Mail settings' tab/tree-node of each account:

- Step 1: Point to a folder with .eml files or to a single .eml file.
- Step 2: 'This is spam'/'This is ham' radio-box.
- Step 3: Listbox with accounts and checkboxes next to them (also select all button) with title: 'Select account(s) to be trained'...
- Step 4: Learn button!

6. Support of free and paid services with spam lists should be implemented (spamhaus.org for example).

7. There should be the option to create our own Junk mail folders. I collect spam to be fed to my mail servers and Junk folders do tent to get large. So, I have my own Junk mail folder (I call it 'Spambox') and I have subfolders in it. I move a year's spam in its own folder at the end of the year (subfolders '2007', '2008' and so on). Since all-year-long spam-folders also get too big, I also have quarterly subfolders in each year's spam-folder ('Q1', 'Q2', 'Q3' & 'Q4'). This means a lot of manual work every 3 months and at the end of each year. What I would like to see implemented is the option to select some sort of 'This folder contains Junk Mail' checkbox in a folder's properties (right-click). Every time a subfolder is created under these custom Junk Mail folders, there should be a message asking if the user wants this newly created folder to inherit its parents' Junk Mail folder flag/status.

8. There should be triggers and filter actions for spam that is either automatically detected as spam by Thunderbird's bayesian filter or manually marked by the user. So far, we only have the options to move/delete/mark as read for Junk Mail. I use a Global inbox for all my incoming mail, so when I mark mail as Junk I want it to be moved to a specific Junk Mail folder and marked as read. Some people want it deleted and others might want it to be automatically & silently forwarded to a spam list project like the Spamhaus Project mentioned above. This suggestion actually depends on bug 59365 and some depends of bug 66425. Why not have also filter actions triggered when mail is marked as read/unread or tagged with a certain tag.

9. In fact this is a combination of suggestions 7 & 8 above and also has to do with bug 19442. I would like to have Junk Mail (or mail in general) moved to a folder not necessarily already existing in my folders. I want it moved to a subfolder of my 'Spambox' depending on the date of receipt (only the year to be exact). If that year's folder is not present, it should be created and also flagged as Junk Mail folder. Then marked/detected Junk Mail should be moved there. In my case I want the filter that is triggered to also move the mail to a subfolder depending on the quarter of the year it has been received. So, I am also talking about not only regular expressions as bug 19442 suggest, but 'if/then/else' or 'case of' support in filter rules as well:

actions 'Create folder and move mail there' & 'Create folder and copy mail there' -> Type 'foldername'

'Foldername' should also accept tokens from system/date and also mail subject/headers.

Taking it a step further, I also want to have special tokens/conditions available for the date filter such as %today%, %yesterday%, %previous/this/next week/month/year% and so on for both filters and searches. I hope this makes sense to you people and also hope I am not off-topic (which I think I am)...

Reproducible: Always

Comment 1

[Kent James (:rkent)]

Hi klonos,

The use of Meta bugs is sometimes effectively used when there is a list of related bugs being actively worked on. Even there it is somewhat controversial (and I never use them myself). But it is usually not effective to just use a meta bug as a place to give a long list of issues, like what you have done here. That would be more appropriate as a wiki page that you could point people to as a summary of your evaluation.

Yet you have clearly given a lot of thought to the questions of junk management. I would encourage you to try to learn the culture of how bugs are managed here, and try to put some of that energy into participating in the processes in a more conventional way. Now is actually a good time, because Bryan has filed some bugs asking for reconfiguration of bug management UI. (bug 482617, to which you haved cc'd yourself but not commented) which are likely to actually get worked on in the near term.

Your comment "take this over and .. break it to smaller bugs/feature requests" is exactly what needs to be done with your bug. Could you please do that? For each issue that you mention, you need to research within Bugzilla to find out if it has already been filed and discussed, or if it is a new issue. If not filed, then you can file a new bug.

Having said that though, I think that spam management in general is not a very hot topic these days around Mozilla mailnews - and I say that as the main developer still working in that area. So much of the world is using server-based anti-spam filters now (which have some clear technical advantages) that interest in client filters is reduced. Yet there is some need for cleanup of the interface, and I'm sure that Bryan would appreciate any help you could give in that area.

I've taken most of my recent work and put it into an extension JunQuilla rather than into the main program. It would also be helpful if you could suggest specific things that should be done in such an extension.

For the main TB work, the culturally appropriate way to work right now is to take a bug (either an existing one or a new one) that clearly explains a single issue that needs addressing in junk management. Make sure that the issue is clearly defined, perhaps add you own comment, then request blocking-tb3 on the bug. Or even better, offer to do some of the work yourself!

Comment 2

[Thomas D. (:thomas8)]

Hi clonos,

thank you for your valuable contribution. You are absolutely right there's a lot of junk-related bugs out there, and we need a more general approach to this (especially for sorting out the UI between global and per-account). I'm sure your ideas will be helpful for solving these problems. Before that, we must first get an overview of all the existing bugs around this topic, and again you are right, we need a meta/tracker bug for this. However, I propose that we should not use your bug as a meta bug. I think it's better to have a separate bug for that where we will only maintain a list of all the bugs, without discussing the problems. Therefore, I removed the [Meta] tag from your summary and opened bug 531787 as a meta/tracker bug. So your bug provides one comprehensive possible concept how we might solve these problems. Someone else might provide a different comprehensive proposal. Both proposals can be tracked in the meta bug, which in itself will be neutral.

If there's any questions, please ask. We definitely appreciate the effort you put into your thoughtful proposal, and I think we really need a comprehensive approach for this. Given a lot of other problems, there'll be no guarantees that the general problem will be tackled by Thunderbird main developers any time soon (we also have similar "problem clouds" for attachments, tags, faceted search, etc. etc.). On the other hand, Kent is right that small and specific changes will have more chances to get implemented. And since there's hundreds of bugs, keeping it as short as possible is also helpful (just use bullets instead of full sentences wherever possible, and don't take this comment as an example... ;)). (The best chances for implementation will be if you provide mockup UI screenshots and actual patches.)

The next step where you could help is collecting all related bugs in the tracker bug 531787. I think we should only add bugs which have Thunderbird or Core as product (no SeaMonkey bugs).

Comment 3

[Wayne Mery (:wsmwk)]

per comment 1, needs to be broken into smaller bugs

Comment 4

[Phoenix]

Resolved per whiteboard