Closed Bug 63735 Opened 24 years ago Closed 18 years ago

news.mozilla.org should have antispam guard

Categories

(mozilla.org Graveyard :: Server Operations: Projects, task)

Product:
mozilla.org Graveyard
Component:
Server Operations: Projects
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: bugzilla, Assigned: justdave)

References

(
URL
)

Details

Now a days about every 3 posting to news.mozilla.org is a spam. Something should 
be done about this.

There's also a lot of empty postings that just have the "subscribe" or 
"unsubscribe" subject in it...
News issues -> marki@netscape.com
Assignee: rko → marki
Moving to nobody until we have better picture of where we'll be going with 
mozilla.org support by IC.
Assignee: marki → nobody
Recently the number of spam postings has become huge...
I centain groups almost 75% of all postings are SPAM...
Depends on: 62228
spam postings on news.mozilla.org has reached a level that's unacceptable
Severity: normal → critical
Help,

My favorite news group is nothing but SPAM.

Realy though, if you look at them, they may be coming from different sources, 
they are all have the same look.  This must be from one specific source.

Jeff.
one good antispam solution is http://www.brightmail.com
looking at the spam we get, it's not really alot of messages, the annoying
point is that the few messages go to alot of groups.
This is especially bad in low traffic groups, as you need to read the group,
which you'd otherwise just skip over.

Sounds like deleting spam by a set of topics should do it. I saw endico deleted
a few by hand in bug 62228, could that be scripted?

Axel
Spam is good. Lowering priority since I didn't want to see red.
Severity: critical → normal
Making this as a blocker because our newsgrops are so littered with
crap that many people refuse to use them. This severely weakens our 
community and wastes the time of the brave souls who manage to use
them anyway.


To answer Axel's question. No, I can't script what I'm doing.
Severity: normal → blocker
One idea I had is to have a newsgroup called "mozilla-spam", and everything that
gets crossposted to it is deleted on all groups.
Perhaps we can call it "mozilla-weed", and call the process "smoking" :-)

(Heck, Microsoft calls the downloading of a newsgroup "slurping"...)
We have a new server now so moving to Markus for comments.
Assignee: nobody → mbaur
*** Bug 112192 has been marked as a duplicate of this bug. ***
This is getting out of hand. Since apparently we (finally!) have a new server,
can we please do something about this. There are more spam posts than non spam
ones.

Possible solutions:

a) Don't propogate the newgroups via usenet - require the use of n.m.o. Will cut
down on usenet spam, not mailing list spam
b) Require people who mail a list directly (ie not via usenet) to be subscribed
to a mozilla list (any one, not necessarily the one they're sending to), or a
dummy mozilla-readers@m.o list.

Yes, this would be a pain, but the spam levels are getting unmanagable now. Does
anyone know what % is mail vs usenet?

BTW, what happened to the newsgroup reorg?
To be blunt, after-the-fact deletion is not really a solution for many people;
those of us in the 21st century prefer push technology like mailing lists :) I'd
strongly suggest shutting down the usenet interface. (and like bbaetz I'm very
curious about the percentage of of non-spam posts that come in from the news
interface as opposed to the mail interface.)
I, for one, enjoy living in the 20th century :)  For me, it's much easier/more
convient to read/post via the newsgroup because the e-mail address I uses most
"forces" me to use Outlook as a client, and while Outlook does allow you to
filter things into another folder when they arrive, it sucks for threading and
it insists on always sending things in HTML (or even worse, Rich Text).  Also,
the company standard is that my name on outgoing mail is "JACOB STEENHAGEN" and
I dislike sending to mailing lists that way (I much prefer the simple and proper
case "Jake").

Having said that, there was a mailing list the I was on once that had the option
once you subscribed that you could recieve no mail.  You could then read/post
using a newsgroup, but it would only accept postings from you if you were
subscribed to the mailing list.  I personally would not object to this type of
solution [although there are probably others that would :( ]
Jake: that last solution sounds like a reasonable one if it's implementable. 
Thats what I meant to say with my option (b).

endico: Is doing something like this possible?
The spam level on netscape.public.mozilla.webtools is such that we have had some
serious discussions about moving the Bugzilla discussion to a public mailing
list that's not gatewayed to a news service.  We're in the process of obtaining
the bugzilla.org domain at the moment, which I'll have control over within the
next week.  The server that will be handling it has Majordomo2 on it, which
issues passwords to each subscriber when they sign up that they can use for
managing their subscription via the web interface.  It would be trivial to write
a web interface (which required the above password) to post messages, which
would satisfy those like Jake that hate the way their work email clients send
email. :)

Although the mailing list archives aren't currently browsable via the web, it's
on the feature list, and majordomo2 is still alpha, so it probably won't be long
(the API is there, nobody's written a UI for it yet).

But if something could be done about the spam on the mozilla mailing lists, I'd
just as soon not take such drastic measures. :)
call me brave or shupid, I read all spam I could get on news.mozilla.org for
one day. ALL came thru 
Path:   secnews.netscape.com!gateway
and not a single one came by nntp.
Seems like a brutal mail spam solution on gila should get rid of most issues.
doh, he's right.  I just took a look through everything that came in today, and
none of it has "Newsgroups:" headers when viewed from the mailing list side,
which means it originated on the mailing list side.  Kill it at the mail gateway
and we'll have most of this solved.
So, can this be done ASAP?

If you don't want to restrict incomming mails, then all we need is a filter the
incomming mail. I recommend razor - http://razor.sourceforge.net/. The only
problem might be that we get the spam mail before anyone reports it as spam, so
it would pass. This is unlikely to be effective on news postings, because they
tend to have random text inserted to avoid these hash-based systems.

If you're worried about false positives, or something, then just hook it up to
add a header so that we can filter on it.

Can something please be sone about this, though, at some point before the
mythical new news server is used? Some groups have more spam than mail, and its
totally useless.
Moving tickets to Ray.
Assignee: mbaur → daruszka
even better than vipul's razor, which has some false positives, try 
SpamAssassin -- with the proper threshold, I have gotten *no* false 
positives, and only maybe 5% of spam makes it through on my (very 
forgiving) settings.
Why don't we close the mail gateway? Why do we possibly need both. The whole
point of nntp is you don't need a mailing list. Alternatively, make the mailing
list require subscription, AND make it use ordb.org.

But I'd much rather see it eliminated. It currently breaks threads and other
stuff, and mozilla and netscape default to sending html by mail, which is very
taboo for nntp messages.

Until then, I've been able to block 90% of spam with the following kill file.
It's for slrn, but should be easily adaptable. I don't think the comments
actually can go in the file, I just added them here for...commenting.

# People that might post something interesting are exempted from the rest
# of the rules (and highlighted, to boot)
Score: =1000
From: @netscape\.com
Score: =1000
From: @mozilla\.org
Score: =1000
From: mpt@mailandnews\.com
Score: =1000
From: bbaetz@cs\.mcgill\.ca
Score: =1000
From: dbaron@fas\.harvard\.edu

# these two post a lot of the crap, auto-kill them
Score: =-9999
NNTP-Posting-Host: 66.68.103.208
Score: =-9999
NNTP-Posting-Host: 212.64.25.42

# auto mark these deleted, but still show them. I'm considering autokill here.
Score: -9990
Content-Type: text/html
Score: -9990
Content-Type: multipart/alternative

# lots of japanese spam. Since I can't read it anyway, even if it was legit,
# autokill it
Score: =-9999
Content-Type:.*charset="Shift_JIS"
Score: =-9999
Content-Type:.*charset=iso-2022-jpg

# Hate to ban a whole country, but they had it coming. Tiwanese ISPs need to
# crack down on spam. They're banned from the mail server at work too.
Score: =-9999
From: \.tw$

# Cross posting is against the rules anyway. But only autodelete, still show,
# just in case (unless it was HTML too, -9998 is the threshold)
Score: -8000
Newsgroups: ,.*,

# All caps = autokill
Score: =-9999
~Subject: \c[a-z]


If any of that's really screwed up, got vote for the cant-paste-at-end-of-input
bug, which made that take a lot longer then it should have.
/me has changed his email address

No, the mail gateways are useful (although I wish we could convince people not
to post html....)

LAst weeks status update said that m.o was considering using the dcc - hopefully
that should help.
News -> Markus
Assignee: daruszka → mbaur
Have you ever thought of a cancelbot? At least for the most obvious spam and for
the HTML messages.
FWIW, gila.mozilla.org is now blacklisted on bl.spamcop.net.

http://spamcop.net/w3m?action=checkblock&ip=207.200.81.215

It's also blacklisted on outputs.orbz.org, but that's mothra's fault, and that's
covered on bug 130841.

spamcop is listing gila because of it's news gateway which is probably throwing 
a lot of spams out. That is not mothra's fault AFAIK.

I still don't understand why gila or ywing/xwing would be on any of open relays 
lists. mothra sends emails out directly nowadays - it's not using gila nor 
xwing/ywing as relays.
Right; this bug isn't about mothra.

Gila on the spamcop list because all the preceeding Received: headers are
stripped off, so it looks like gila is the originator of every message to the
newsgroups, including the tons of spam. If the original headers were preserved,
then gila wouldn't be the first server in the list, and so wouldn't get blamed.

This should be done even after the spam filter is set up, since any filter won't
be perfect.
Mass changing IC's ticket to reflect current situation.

mozilla.org, AOL employees:

If you want IC to look at issues reported in bugzilla, please open a Helpdesk
ticket and ask it to be routed to AOL R1 Server Operations. We currently have no
way to handle comprehensive problem resolution through bugzilla. This is not a
change in the way we are supporting mozilla.org - we are still supporting you on
the level as before. IC's support is based on Helpdesk ticket system - not
bugzilla which only few hard-core people are looking at. 

Also, projects are handled elsewhere - not in bugzilla. If you have projects you
need us to deliver please feel free to contact me directly.

Summa summarum: tickets -> Helpdesk
Project initiations -> RKotalampi@aol.com
Assignee: mbaur → nobody
Could someone at AOL/Mozilla/Netscape please open a Helpdesk iicket as Risto says.
Tonite news.mozilla.org has just hit by a subscribing nitemare! Some guy just
subscribed all the mail2news gateways to all kind of subscribtions lists (100+).
We like to avoid this in the future!

*** Bug 127623 has been marked as a duplicate of this bug. ***
adding self to cc list
Isnt this issue largly fixed by the new code on the mail->news gateways that
only allows those that are on the mailing list to post to the mailing list?
this issue hasn't been fixed. Closing the mail -> news gateway is another bug.
This bug is about hooking up the news.mozilla.org to some kind of anti spam
program. So that the "usual" spam automaticlly gets deleted. A lot of spam
companies sends out spam directly to the newsgroups.
Blocks: 62228
No longer depends on: 62228
*** Bug 224944 has been marked as a duplicate of this bug. ***
QA Contact: endico → justdave
The lists are being targeted again.  Roughly 75% of postings in the past 4 days
on mozilla lists I'm on have been spam.  Are they from the same place?  Any
chance we can get a block on the sending site?
Who can change the configuration of the mailing lists, so that only mails from
subscribed senders are accepted?
In my opinion this would be an easy and effective solution of the problem.
See comment #20.

In the current state, the lists are not reasonably usable...

(In reply to comment #40)
> Who can change the configuration of the mailing lists, so that only mails from
> subscribed senders are accepted?

The lists are already configured that way.  All of the current spam is coming in
through the usenet gateway, which is exempt from that rule because it would
block all the legitimate posts as well.

> See comment #20.

Comment 20 was written 3 years ago, when we were on a different mailing list
system that didn't have the "block nonsubscribers" option available.
Is comment 10 plausible? Most of the spam is cross-posted to most of the groups,
so it would be quite effective.
Severity: blocker → normal
Assignee: nobody → justdave
Component: Server Operations → Server Operations Projects
QA Contact: justdave → justin
No longer blocks: 62228
Depends on: giganews
(In reply to comment #14)
> a) Don't propogate the newgroups via usenet - require the use of n.m.o. Will cut
> down on usenet spam, not mailing list spam

This is the option we've taken, and it seems to be working.  There's been a couple slip through, but there's a hell of a lot more spam out there on the net in general now than there used to be, and the spam level on our groups in miniscule in comparison to how it used to be, so I'll consider this solved at the moment.  There's still other things we can do probably, but we'll attack those as we come to them.  Please file new bugs for specific issues if new problems arise.
Status: NEW → RESOLVED
Closed: 18 years ago
Resolution: --- → FIXED
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.