Closed Bug 638209 Opened 13 years ago Closed 13 years ago

2011 March - 1st batch of newsgroups to convert to moderated

Categories

(mozilla.org :: Discussion Forums, task)

Product:
mozilla.org
Component:
Discussion Forums
Platform:
x86
macOS
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: justdave, Assigned: justdave)

References

Details

In this next pass of newsgroup conversions, we'd like to convert the following newsgroups to moderated:

mozilla.community.uk
mozilla.governance
mozilla.jobs
mozilla.legal
mozilla.support.other

list-specific notes:

mozilla.jobs:
 - should be moderated by Ben Bucksch per bug 598060 comment 89 and 90
 - should allow non-member posts (depending only on SpamAssassin to flag things
   for moderator approval)
mozilla.support.other:
 - should allow non-member posts (depending only on SpamAssassin to flag things
   for moderator approval)
waiting on actually putting this through until we get a couple other things straightened out with Google yet, but that's in motion and I expect to hear stuff really soon now.
I thought none of the mozilla.support.* newsgroups were to be moderated.
(In reply to comment #2)
> I thought none of the mozilla.support.* newsgroups were to be moderated.

Originally, yes.  See bug 635078.
Gerv: care to do announcements that these are about to change?  Looks like Google got the automation working again finally. :)

Whatever time frame you think is appropriate as long as I've got 48 hours notice to get Giganews on board.
About to change, or has now changed?

Gerv
About to.
Giganews has been notified.
Whiteboard: [3/11/2011 - 11:50pm PST]
Giganews has confirmed, we're on for Friday night. :)
These have been switched over in Mailman.

Giganews has committed to making the switch at 11:59pm.  Google polls the config at 12:02am usually.  How long it'll take them to process it after they poll I couldn't tell but we'll watch and find out. :)
Whiteboard: [3/11/2011 - 11:50pm PST] → [waiting for Giganews and Google]
Giganews says:

---8<---
The groups you requested were moderated just before midnight PST. This may take some time to completely filter through our system. Please let us know if you have any questions. 
---8<---

Google polled at 12/Mar/2011:00:02:05 -0800
I can't find any way to tell whether a group is actually moderated or not on Google short of posting to it and seeing what happens.
Also of concern is that there was a new group creation included in the batch picked up by Google that included these moderation changes, and that new group creation hasn't happened yet. (mozilla.dev.identity)
Testing mozilla.support.other, it looks like the GG end is now sent through the mailing list, but the Giganews end is not yet.
yep, I see the mail from google for that post in the logs.  Cool :)

I reopened the giganews ticket last night because my test post failed to moderate.
The new group mentioned in comment 13 did show up this morning on Google.
I kicked back on Giganews about the moderation, he gave it another poke, and it seems to be successful this time.  I have moderation events in the mailman logs from items posted via giganews now.
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Whiteboard: [waiting for Giganews and Google]
Dave, Gerv, thanks for the trust.
I just went through the moderation queue, seems like it wasn't checked for 3 years, found 2 false positives from 2008 that I approved, rest was spam.

If not already done, can you make it so that
a) all newsgroup posts have to go through SpamAssassin, and possibly mailman moderation?
b) all posts with a SpamAssassin spam value over 5 are automatically Rejected, and all above 8 are automatically Discarded, or something along those lines (you know what's best on your system)?
(In reply to comment #18)
> If not already done, can you make it so that
> a) all newsgroup posts have to go through SpamAssassin, and possibly mailman
> moderation?

Everything goes through SpamAssassin already.

> b) all posts with a SpamAssassin spam value over 5 are automatically Rejected,
> and all above 8 are automatically Discarded, or something along those lines
> (you know what's best on your system)?

Everything over 6.31 is discarded currently.
perfect, thanks!
> Everything over 6.31 is discarded currently.

Doesn't seem to be working:
The jobs@lists mailing list has 1 request(s) waiting for
your consideration at:
...
Pending posts:
From: mailer-daemon@... on Wed Mar 16 00:28:26 2011
Subject: Mail delivery failed: returning message to sender
Cause: SpamAssassin identified this message as possible spam (score 8)
Maybe I was misreading the headers.  That was based on the "required=6.31" that SpamAssassin was putting in its assessment headers.  Maybe I'll need to cron that or something.
Also, there is spam which is going right through to the list, SpamAssasin doesn't seems to even suspect it, so I never get to moderate it.
I did get to moderate 2 spams (one quoted above), with a score that should be rejected outright.
Seems like there's something not right in the setup yet.
(Ideally, of course, I shouldn't have to moderate spam every day, see comment 18.)
I've changed the jobs list to full moderated.  This means everything that comes in will need to be moderated for the time being.
Please update "Forums" at <http://www.mozilla.org/about/forums/> to reflect moderation per bug #620085.
Done.

Gerv
There is now more spam appearing in mozilla.support.other than before moderation began.  See comments numbers 8 and 9 in bug #635078.
ditto for .jobs. REOPENing.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(In reply to comment #29)
> ditto for .jobs. REOPENing.

Jobs is full-moderated, and you're the moderator.  Is there stuff getting through that you're not approving?  (The way it's configured currently, NOTHING should show up on that list without an approval from you).
How was the list setup last year? Was subscription required? If so, did posters (from all channels) get notified that they need to subscribe? How was spam dealt with? We had quite some spam, but less than now.

> Jobs is full-moderated

But was just an emergency response and not how it should be. It needs to be an open list, because the whole purpose is for outsiders to contact the Mozilla community.

I can moderate the "maybes", things which look like spam but may be not, but that should be only exceptional. I can't be there every day all the year and the bottle neck of the list, posts regularly only going through after I approve. Even a second moderator doesn't change the fact that there's a bottle neck and delay.

Is it possible to have a setup like in comment 18, i.e.
- all posts, from all origins, need to go through SpamAssasin
and:
1) posts which are likely ham should go through directly
   (without moderator intervention)
   (we'll have to tweak that until almost no spam gets through,
   but most ham goes through.)
2) posts which are uncertain (e.g. score 0.1-5) should be held for
   moderation
   (and the author notified, without spamming innocent spam From: victims)
3) posts with a high spam probability (e.g. score 5 or 8 and higher)
   should be Discarded immediately
   (without moderator intervention)
   (Optionally, we might Reject it, but again without spamming
   innocent spam From: victims)

Before you changed it to full moderated, we had more spam than ever before, in fact, in 4 days, we had more spam going through than in the 4 months before. So, the SpamAssasin setup is not really working, and the situation is worse than before.

Dave, is such a setup possible?
Dave? The daily moderator mails are annoying, I can't take this for long.

(An alternative that would work for me is send me the mails individually, and if my server SMTP-rejects them or I do not answer, they are rejected or discarded. If I explicitly approve them, they are passed on. This is a more risky setup, but would be more bearable for me. What annoys me is the digest and having to manually discard the spam, even if I can discard many at once.
The much preferred setup would be the one described in the last comment, though.)
The mails are already supposed to be sent individually unless you disabled that in your settings.  I just checked, and it is enabled.  That's "Should the list moderators get immediate notice of new requests, as well as daily notices about collected ones? (Details for admin_immed_notify)" on the main page.

This batch of groups is done, let's either continue the spam discussion on bug 598060 or get a specific bug for it. (It likely affects all of the groups and not just this one)
Status: REOPENED → RESOLVED
Closed: 13 years ago → 13 years ago
Resolution: --- → FIXED
(In reply to comment #33)
> The mails are already supposed to be sent individually unless you disabled that
> in your settings.  I just checked, and it is enabled.  That's "Should the list
> moderators get immediate notice of new requests, as well as daily notices about
> collected ones? (Details for admin_immed_notify)" on the main page.

I can't find an option to disable the daily summaries :(
> The mails are already supposed to be sent individually unless you
> disabled that in your settings.  I just checked, and it is enabled.

FYI, I enabled manually that after your tip where the option is.
You need to log in before you can comment on or make changes to this bug.