Closed Bug 17483 (killfile) Opened 20 years ago Closed 17 years ago

[FEATURE] implementation of news filters

Categories

(MailNews Core :: Backend, enhancement, P3)

enhancement

Tracking

(Not tracked)

VERIFIED FIXED
mozilla1.3alpha

People

(Reporter: laurel, Assigned: sspitzer)

References

(Blocks 5 open bugs, )

Details

(Keywords: helpwanted)

Attachments

(2 files, 10 obsolete files)

43.98 KB, image/gif
Details
71.80 KB, patch
Bienvenu
: superreview+
Details | Diff | Splinter Review
This bug is to track the backend implementation of news filters.

Once this is implemented in basic terms, please provide info on any intended
exceptions to the 4.5 news fitlering implementation.
QA Contact: lchiang → laurel
No you didn't.  (sspitzer)
Target Milestone: M15
moving to m16.
Target Milestone: M15 → M16
Sol, make M16 if beta2.
Target Milestone: M16 → M17
[FEATURE] bugs past M16 are OUT for this release.  Marking M20.  If you disagree 
with this action, please help me explain it to the PDT.
Target Milestone: M17 → M20
Moving target milestone to "future" to be reviewed at a later time
Target Milestone: M20 → Future
Keywords: mozilla1.0
not sure what changes are required to the front end or back end to get this
working, but it sure would be nice.

4.x windows (maybe mac, too) supported it.

note to self, check the migration code.
Keywords: 4xp
this bug covers the FE and BE work necessary.

the first thing to do is to get newsgroups to show up in the filter picker.
Summary: [FEATURE] Backend implementation of news filters → [FEATURE] implementation of news filters
See bug 84036. News accounts were visible in the filter picker at one point,
even though that was never implemented.
Blocks: 97112
It would be great if the same UI could be used for mail and news so that I can
e.g. apply filters to both (flag message if my nick name appears, etc...).
But then some filters make no sense and had to be disabled. It might be easier
to create a separate UI in the end, wouldn't it?
Adding my vote of "please please please do this" to this.  Also, OE has this.
Keywords: nsCatFood
I'd also like to add a please-please-pretty-please-with-sugar-on-top vote. 
There are a couple of posters on some of the newsgroups that I read who are
really getting on my nerves, and I'd like to be able to block them.
*** Bug 108709 has been marked as a duplicate of this bug. ***
Keywords: nsbeta1
Keywords: nsbeta1nsbeta1-
Almighty Mozilla team, I besiege thee! We needeth news filters to block all
those evil cratures on usenet! So have mercy!
cybermage: I beseech thou to use the correct words if you insist on making light
of such a heavy matter. For the request for which you have pleaded, we require a
chronology potion, such that time be stopped whilst we affect ourselves upon
this feature.
about correct words: I'd need a spellchecker with a proper AI for that...; in
any case, it should be animated, in the form of a doggy, or a cute cat. Hmmm,
sounds awfully familiar somehow.

:-)
Blocks: 134740
*** Bug 138760 has been marked as a duplicate of this bug. ***
Blocks: 131164
*** Bug 153627 has been marked as a duplicate of this bug. ***
No longer blocks: 134740
It is unbelieveable to me that this bug has not been fixed.  Newsgroup filtering
is essential to any useful newsgroup tool, plus version 4.x had this capability.
 Get it done already!  I went out of my way to create a bugzilla login just to
submit this.  This question comes up so many times on the netscape newsgroups
it's not even funny.
Blocks: 10097
No evangelism comments, please. If you want this fixed, write the code, or vote
for it.
*** Bug 162106 has been marked as a duplicate of this bug. ***
Just added my vote.  Would love to help with this, but my C programming skills
are as rusty as the gun turret they just pulled up from the USS Monitor.
Blocks: majorbugs
*** Bug 168527 has been marked as a duplicate of this bug. ***
With 96 votes and no detectable activity, how about marking this helpwanted?
(I'm not sure who is supposed to do that. The owner, I'd think.) As the bug is
also assigned, chances are that a contributor looking for a bug to fix won't
catch this one in her net :-/
helpwanted.

Especially in observance of bug 163993 and bug 92997.
Blocks: advocacybugs
Keywords: helpwanted
Alias: killfile
Blocks: 19442
Blocks: 19403
Blocks: 16903
Blocks: 73075
Blocks: 163935
Bug #19442 is a general filtering feature, and should not be considered blocked
by this bug.
No longer blocks: 19442
As a 'non-coder', is there anything else I can do to help implement this
feature, besides voting for it?
solon: you can certainly help in writing test cases / test outline for this
feature if it ever gets implemented.  

Take a look at http://www.mozilla.org/quality/mailnews/tests/index.html for
examples of the test outlines already present for existing features.
Blocks: 16913
This lack of functionality renders the news reader nigh on unuseable.
Particularly when groups that I use are being flooded by script kiddies. Even if
it were implemented using the 'custom header' filter as used in the mail
filtering, it would be a god send. Why is there not a common message filter
class that can parse and filter messages on their headers?

IMHO - this is the single biggest functional deficiency in the news component.
Does it appear anywhere on the road map?
Severity: normal → enhancement
Keywords: mozilla1.0mozilla1.3
I take issue with the severity change from "normal" to "enhancement". This is
not an "enhancement", this is basic newsreader functionality.
Garth, the plain fact is that the newsreader does work. The lack of filtering is
a huge problem, but at least you can read Usenet. The fact that filtering is a
very, very badly needed feature doesn't mean it isn't a feature, and thus for
Bugzilla purposes considered an enhancement.

What is needed is a higher prioritization of this bug. Currently, it is set only
at P3, which is somewhere in the middle. Now that Mozilla 1.2 is nearly out the
door, it's time this badly needed feature got more attention. I'm not aware of
anything in the newsreader that is needed more. I recommend P1 priority.

The keyword "helpwanted" is outstanding on this bug. If somebody could
contribute a patch, I'm sure the assignee would be appreciative.
I've said it before and I'll say it again - contribute something or stop
complaining. Nobody owes you this feature.
implementing this will mean we can turn on the spam feature (when it lands) for 
news.  (see #169638)

for spam, we need to hook into the bayesian filters after the normal filters 
are run.

here's the steps I'll take to implement this:
step 1

1) show newsgroups (not servers) in the filter list dialog
2) make it so the right criteria and actions show up for online news filters
(no offline yet.)
subject / sender (contains/is/begins with/ends with)
then mark thread, watch thread, ignore thread.  (4.x had delete?)

3) fix the news code to get the filter service, and do the right thing to the 
incoming news message

step 2:

4) look into 4.x delete action
5) filter logging
6) offline news filters (different table for UI)
Attached image screen shot
working on this js error:

JavaScript error:
 line 0: uncaught exception: [Exception... "Component returned failure code:
0x8
0004001 (NS_ERROR_NOT_IMPLEMENTED) [nsIMsgFolder.getFilterList]"  nsresult:
"0x8
0004001 (NS_ERROR_NOT_IMPLEMENTED)"  location: "JS frame ::
chrome://messenger/c
ontent/FilterListDialog.js :: setServer :: line 162"  data: no]
I think I see what "delete" did in 4.x	on delete, we wouldn't even both to add
it to the db.  (different than ignore)
Attachment #105001 - Attachment is obsolete: true
Attachment #105003 - Attachment is obsolete: true
more items:

1)  instead of n.m.p.xul/rules.dat, we should be doing n.p.m.xul (summary file 
is n.p.m.xul.msf).  this is how 4.x did it, so if we do it the same way, 
migration will work.

2)  show ignore thread / watch thread in action menu list (as well as delete 
and mark as read, which are already there

3)  hide other actions for news

4)  allow filter create to set the filter type (InboxRule / NewsRule)
I think a pretty separator between the mail accounts and the news
accounts/groups in the drop-down would make the distinction between them clearer.
Attached patch updated patch (obsolete) — Splinter Review
Attachment #105005 - Attachment is obsolete: true
"I think a pretty separator between the mail accounts and the news
accounts/groups in the drop-down would make the distinction between them 
clearer."

do you find the screen shot (see http://bugzilla.mozilla.org/attachment.cgi?
id=105002&action=view) unclear?

We don't have a separator between accounts in other pickers.
> do you find the screen shot (see http://bugzilla.mozilla.org/attachment.cgi?
id=105002&action=view) unclear?

Not really but I'm not an average user.

> We don't have a separator between accounts in other pickers.

Here it's a little different though. Mail accounts are selected directly while
newsgroups are selected by first selecting the news server (well, expanding its
menu) and then the particular group.

That said, I don't think separators between mail servers and news servers in
other pickers would be a bad thing.
cool, I was able to create filters to watch and ignore threads.

still need to:

1) hide other actions for news
2) test that delete and mark as read filters
3) fix "manually run selected filter(s) on folder" picker in filter list dialog 
for news
4) test out filter logging

there's some optimizations that could be made, too.

but I'd land what I've got now for 1.3 alpha, as it's usable. 
mark as read worked.  (so cool, when I post to newsgroups, I mark my own posts 
as read.)

some more todo items:

1) pre-flight account picker (if newsgroup selected in folder pane)
2) spin off bugs about offline news filters
3) test logging
Target Milestone: Future → mozilla1.3alpha
Attached patch new patch (obsolete) — Splinter Review
completed items:

1) create filter from messages UI should be enabled for news
2) run filters folder picker and button hidden for news
3) pre-flight account picker (if newsgroup selected)

todo list:

1) pre-flight account picker (if host selected) and
   account central filter UI (pick the first newsgroup?)
   
   select first newsgroup, but what if no newsgroups?

2) test logging
3) test delete
4) optimize, code cleanup, self review
5) log bug, about offline news filters
6) log bug, about filter after the fact for news
7) log bug, plug in spam stuff
8) log bug, disable "Run filters on Selected folders" UI when server selected
Attachment #105038 - Attachment is obsolete: true
delete works as does logging.

updated todo list:

1) pre-flight account picker (if host selected) and
   account central filter UI (pick the first newsgroup?)
   
   select first newsgroup, but what if no newsgroups?

2) optimize, code cleanup, self review
3) log bug, about offline news filters
4) log bug, about filter after the fact for news
5) log bug, plug in spam stuff
6) log bug, disable "Run filters on Selected folders" UI when servers are 
selected
I think we should be able to do "isn't" rules with with news filters.

We can't do it with online news searches / filters, but we can with filters. 
(we can do anything with offline news searches, since we have the bodies 
locally, but offline news filters aren't implemented yet).

So instead of:

[Subject / Sender] [contains / is / begins with / ends with] [...]
[delete | watch | ignore | mark as read] 

Looking at how the code works, we could also do:

message-id, date, size, references, lines (though not sure how useful some of 
these would be to most users)

We should be able to do "isn't" and "doesn't contain".

When mscott's view code fully lands, we'll be able to do 
[Sender] is in [Personal Addressbook]

I think we can also add more actions, like labels, priority, and flagging.
a few ? nits, other than that, looks fine. 

+      if (getScopeFromFilterList(gFilterList) ==
Components.interfaces.nsMsgSearchScope.news)
+        defaultAction = nsMsgFilterAction.Delete
+      else
+        defaultAction = nsMsgFilterAction.MoveToFolder;
+      
can use the ? operator here and below.

+    var hide = false;
+ 
+    if (element.getAttribute("hideforscope") == aScope)
+      hide = true;
+    else if (element.getAttribute("showforscope") &&
(element.getAttribute("showforscope") != aScope))
+      hide = true;

this can just be:
var hide = (element.getAttribute("hideforscope") == aScope) 
               || (element.getAttribute("showforscope") &&
(element.getAttribute("showforscope") != aScope));

firstItem = (server.type == "nntp") ? msgFolder.URI : rootFolder.URI
+                    if (server.type == "nntp")
+                      firstItem = msgFolder.URI;
+                    else
                     firstItem = rootFolder.URI;


and this comment can either be removed, or should be reworded - it was basically
a note to myself in the code this was taken from:

+        // for ignore and watch, we will need the db
+        // to check for the flags in msgHdr's that
+        // get added, because only then will we know
+        // the thread they're getting added to.
Comment on attachment 105053 [details] [diff] [review]
updated patch, skip the filter stuff when downloading headers if the filter count is zero

sr=bienvenu, modulo the nits.
Attachment #105053 - Flags: superreview+
Attached patch updated patch (obsolete) — Splinter Review
1) add more functionality.

Beyond, 4.x, 1.3 alpha will be able to do:

[Subject | Sender] [contains| doesn't contain | is | isn't | begins with | ends
with] [...]

and also:

[Date] [is | isn't | < | > | == ] [...]

and the availabe actions are
[delete | watch | ignore | mark as read | label | set priority | flag]

and once it's supported (from mscott's view code), we'll be able to do:

[Sender] [is in] [addressbook]

2)

in 4.x, news filter name was netscape.test.dat, not netscape.test, so I fixed
that.
Attachment #105145 - Attachment is obsolete: true
see two spin off bugs:

bug 178397 (about using HEAD command for more headers)
bug 178398 (about other XOVER fields)

I'll seek a re-review from bienvenu, and land in early 1.3 alpha
Blocks: 178397, 178398
Is there a bug for mscott's view code? (because I REALLY want to be able to do
"sender is not in address book" - obviously not part of this bug, but I'd like
to find the right one
Comment on attachment 105173 [details] [diff] [review]
themes part checked in (bug #178061).  but patch includes fix for #123767

sr=bienvenu (but you can change all the PR_FREEIF's to PR_Free, if the code
returns immediately after the Free (PR_Free checks for null - PR_FREEIF checks
for null, AND nulls out the pointer)
Attachment #105173 - Flags: superreview+
> Is there a bug for mscott's view code?

Bug 176850.
fixed.  I'll spin up other issues that I know about in new bugs, and list them
back here in this bug.
Status: ASSIGNED → RESOLVED
Closed: 17 years ago
Resolution: --- → FIXED
Now that this bug is being dealt with, I've run out of excuses to give ben@netscape.com as to why I haven't reinstalled Mozilla yet..
spin off bug:

implement filter after the fact for news.  [bug 178870]
Marking this bug verified, feature implemented, generally working. Any specific
issues found with news filters will be logged separately.
OK using nov19 commercial trunk.
Status: RESOLVED → VERIFIED
*** Bug 182987 has been marked as a duplicate of this bug. ***
I'm afraid, I cannot see where the specific bug I reported is duplicated here.  Please 
clarify.
Stewart, you've reported Bug 182987 as newsgroup filters (usenet killfiles) not
opening a dialogue box in 1.2.  That's because newsgroup filters have not been
implemented at all until the 1.3 stream, and that's what this bug (the one
you've been duped to) refers to.

Newsgroup filters have been implemented, and are available if you want to
download one of the nightly builds that will be the basis for the 1.3 release.

Hope this helps
 

*** Bug 184648 has been marked as a duplicate of this bug. ***
Might be we have a backend for News Filters, but we only had it for a very short
time, actually it does not work, pls. see the many bugs concerning this feature.

Actual build confirming this problem:
Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.5b) Gecko/20030910

I am afraid we have a regression here

Rainer
News filters don't work in 1.6final or 1.7alpha either
Someone should reopen it
News filters don't work in 1.6final or 1.7alpha either
Someone should reopen it
this bug was fixed. if things don't work today, find or file a new bug.
See unconfirmed bug 225454, http://bugzilla.mozilla.org/show_bug.cgi?id=225454

It seems to have started not working again with 1.6.0
Product: MailNews → Core
No longer blocks: 16913
No longer blocks: majorbugs
News filtering works for me running Thunderbird version 1.5.0.9 (20061207) but only for postings I receive AFTER I set the filter.

Consequently, the menu item "Tools -> Run Filters on Selected Folder" is NOT available for news folders. Please see enhancement request bug 178870.
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.