Closed Bug 120030 Opened 23 years ago Closed 21 years ago

Munge email addresses to make life difficult for the spammers' spiders

Categories

(Bugzilla :: User Accounts, defect, P1)

defect

Tracking

()

RESOLVED FIXED
Bugzilla 2.18

People

(Reporter: colin, Assigned: gerv)

References

()

Details

(Whiteboard: See comments 133 through 142 for what this did and bugs that expand on it)

Attachments

(5 files, 7 obsolete files)

985 bytes, text/plain
Details
355 bytes, text/plain
Details
4.27 KB, patch
Details | Diff | Splinter Review
432 bytes, patch
Details | Diff | Splinter Review
2.12 KB, patch
justdave
: review+
Details | Diff | Splinter Review
I've just became a Moz Bugzilla user to report a bug, and my e-mail is thus
listed on the page of a bug I've just reported. I think that user preferences
should definitely have an option to "obscure e-mail" or enter modified e-mail
addresses (with, possibly, the real addresses available only to logged-in
bugzilla users) so as not to give spam harvesters a nice database of addresses.
If not, maybe 1) an option to remove the user account and remove the e-mail
address from the bug report (yes, I know it would be quite an annoyance) or 2)
including a (bold?) comment on the bug submission form about the user's e-mail
being displayed (so they may choose to set up a throwaway account).

Thanks.
see also bug 54159, which isn't exactly the same issue, but the discussion there
offers some suggestions that'll go with this one.

bug 115125 seems to be the same issue but only suggests one solution.  This bug
is more complete, so this one is going to survive.
*** Bug 115125 has been marked as a duplicate of this bug. ***
Recently changed bugzilla email address, and within a few days had unsolicited 
advertising to the new address.

While the new address is not a "bugzilla-only" address, bugzilla.mozilla.org is 
the most probable candidate for having leaked it to harvesters.

A few thoughts:

Legitimate use of direct email to bugzilla users is relatively rare, so it 
seems ok to me to make this a little more awkward (although at least possible 
to admins).

Most other publicly visible systems have a unique ID that is the key for that 
system, and an email address as a separate field. There is no reason not to 
provide an easy mechanism to get one from the other, but it should be possible 
for users to opt out of, and ideally should trigger an administrator's 
attention if over-used (e.g. by harvesters).

I guess one question is whether a new field is needed, or if it could just use 
the existing "real name" as the handle that everyone else uses...

Also, might there be different levels of access - make available to "core" 
bugzilla team only / to any logged-in user / to the entire world.
I think this is definitely going to be an important issue as the spam gets worse
and worse out there...

Setting out a couple versions solely for the reason that this is going to
require major sweeping changes to the way people log into and use bugzilla.

Displaying of names for comments and name fields won't be a big issue, but...

Points that need to be resolved in order for this to occur:

Bug ownership, QA contacts, and CC lists are all handled by entering email
addresses.  Right now the email address is the only thing about an account
that's guaranteed to be unique.  It's perfectly legal right now to have more
than one account with the same real name on it.  It's also perfectly legal to
leave your real name blank (which currently displays your email address instead
of your real name if you do this).

Last time I heard a count, bugzilla.mozilla.org has approximately 16,000 user
accounts.  Probably 4000 of those are still recently active.  Whatever we do is
going to have to be backward compatible with the old system forever, because
even if we force people to pick user IDs the next time they log in, we can't
logically expect those 16,000 users to log in and create a new user ID, or force
those who have duplicate real names to make them unique.

Everyone does currently have a unique user ID number in addition to their email
address, but expecting people to remember numbers is unrealistic.
Severity: normal → major
OS: Windows 2000 → All
Priority: -- → P1
Hardware: PC → All
Target Milestone: --- → Bugzilla 2.20
I don't see why this would `require major sweeping changes to the way people log
into and use bugzilla'. All that's required is to replace @s, and post-@ .s,
from e-mail addresses whenever printing them in bugs or bug lists, like MailMan
does. myk mozilla org, Roland.Mainz informatik med uni-giessen de, and so on.
Summary: RFE: Bugzilla bugs list is Spamma's Paradise → Bugzilla bug lists are a spammer's paradise
I just started getting LOADS of spam on an email account that was 
created just for bugzilla.  please fix this mozilla team, this is a 
serious issue!!!!!!!!
See bug 54159, which should hopefully be fixed for 2.16.
mpt: what about the mailto: links on all the comments?  Should they just go away?
*** Bug 111497 has been marked as a duplicate of this bug. ***
ok, mpt's suggestion I think is best, but there's still a few problems...

The idea of Bugzilla is that the discussion is supposed to go in the bug, and
not in email.  I'd be happy if the mailto links all go away (except maybe in the
admin interfaces and probably anywhere "%maintainer%" is mentioned).  On-screen
representation can be transformed using the space method mpt suggests.

$email =~ s/^(.*)\@(.*)$/;
($username, $domain) = ($1, $2);
$domain =~ s/\./ /go;
$email = "$username $domain";

however, what we still need to determine is how things like reassigning a bug
and putting people in the CC list....

perhaps the CC list item values can have the userID instead of the email
address, and the visible items in the list can be the transformed versions.
If we show @.-replaced email addresses everywhere, people should IMO be able to
use those munged addresses everywhere - when adding to cc lists, reassigning,
querying, login and so on. So there should be complete backwards replacement
logic in all the places where email addresses can be entered.
*** Bug 145499 has been marked as a duplicate of this bug. ***
just voted for this one. My bugzilla address is starting to get an appreciable
amount of spam (30% of today's traffic was spam).

Also, including a bit of perl code I use for munging addresses, in case folks
might find it handy. It has the advantage of not using "just" decimal encoding,
which is what most tools do (and is therefor probably the first things spammers
will learn to beat). It can be beaten by spammers, and one day will be, but
currently keeps addresses clean (in my experience).

srand;
my @encode = (
    sub { '&#' .                 ord(shift)   . ';' },
    sub { '&#x' . sprintf( "%X", ord(shift) ) . ';' },
    sub {                            shift          },
);
$_ = "mailto:$_";
s{(.)}{
    my $char = $1;
    if ( $char eq '@' ) {
        # this *must* be encoded. I insist.
        $char = $encode[int rand 1]->($char);
    } elsif ( $char ne ':' ) {
        # leave ':' alone (to spot mailto: later)
        my $r = rand;
        # roughly 10% raw, 45% hex, 45% dec
        $char = (
            $r > .9   ?  $encode[2]->($char)  :
            $r < .45  ?  $encode[1]->($char)  :
                         $encode[0]->($char)
        );
    }
    $char;
}gex;
$_ = qq{<a href="$_">$_</a>};
s{">.+?:}{">}; # strip the mailto: from the visible part
#  $_ is now your email link with a bit of spam sheilding

Given the amount of places that addresses are used in the user interface, where 
munging and un-munging might be needed, can't we just fix this properly rather 
than messing around with munging...?

e.g. Add a column to the profiles table called "email_address" with the initial 
value the same as login_name, change only the email sending code to use the NEW 
column, and then provide a mechanism for the login_name to be changed to 
something other than an email address when confirmed by the owner of the email 
address.

Probably would have login_name being either the same as email_address or a non-
valid address (no @ sign) to prevent impersonation.

After all, spammers are in principle at least just as capable of unmunging an 
address as anyone else, and some people might not want their address directly 
revealed AT ALL. There is no reason to have a condition of reporting a bug to 
be having ones email address displayed to the world.
I agree fully with comment #14.
I can imagine one way how to solve this - we can separate users in 2 parts:

1. Not logged in and not privileged users - they should become realnames with no
emails (maybe there should be userid in brackets for easier identifying)

2. Privileged users (a new checkbox in user configuration - "can see email
addresses") - they become old style

I think such patch to create could not be very hard, but here needs some more
deeper discussion on that.

Also - take into account Joel Peshkin's last email to
mozilla-webtools@mozilla.org - 
http://groups.google.com/groups?group=netscape.public.mozilla.webtools&selm=3CF25355.90104%40peshkin.net
As a service to people who have the same problem with this as I do, I have
attached a chunk of really badly written perl (written before coffee) that will
go through your POP mailbox and delete anything that was delivered to your ISP
by a blacklisted spamsite.

Change the userid and password as you see fit.	You may also have to change the
m/received:\s+from\s+[^\[]*\[([^\[\]]+)\]/i statement to match whatever your
ISP does with the last hop's IP address.

When testing this, comment out the Delete() call and log the output.  Don't say
I didn't warn you.

You can change the list of blacklists in @suff.  I found it interesting to run
this against my accumulated messages and evaluate which blacklists work the
best. (This list takes out about 80-90% of mine) If this is your main account,
I suggest spamcop and visi.  ORDB takes out sites that have open relays but
have not been caught sending spam.
Attachment #85353 - Attachment mime type: application/x-perl → text/plain
Just something to remember when a patch for this gets written: beware of bugs
like bug 147486 and bug 146447 when you're displaying realnames; when I did the
2.14.1 branch audit, the places where realname was used was actually pretty
small/limited, but it sounds like the proposed solution(s) to this problem would
increase the use of realnames... just be sure to properly escape those.
http://www.mozillazine.org/talkback/read.php?f=1&i=5080&t=5080

Anyone know what "spam crawler-proof JavaScript links" are?
Wouldn't it be easier to just add some measures like the following.....
1) Display only realnames unless the user is logged in
2) Display only realnames in bulk bug listings
 
You can play tricks with document.write, and js mailto links which do string
concatentation. We're not going there for bugzilla.

What happened to mod_throttle? I know the limit got bumped up when netscape went
to NAT - did it get set too high?
the real reason bz shows email addresses are because they're used as userids.
Would it be worth considering splitting that up, so that user id != email
address? The transition plan would set everyone's ids as equal to their email
addr, and then let people change it. New accounts would have to specify both. Or
is that just really complicated and not worth it?
Why aren't we going there?

Yes, mod_throttle has been rendered ineffective by Netscape's NAT, although I
don't think it really stopped spammers much anyhow.

Splitting usernames from email addresses has been discussed before (although I
can't find a bug on it) and is a good idea, but it should still be possible to
email other users, perhaps by funneling those emails through Bugzilla, although
I've always hated that approach because it means I can't use my own familiar and
feature-rich email interface.
How 'bout only displaying email addresses when you're logged in? I doubt a
spambot would create an account...
its not too hard to split it up - all the lookups are done via an accessor
function anyway.

You'd have to change all the places we display the email address to not do so,
though, and then there is the issue with how you mail people. OTOH, we try to
discourage direct mailing, so...
Jonas, spammers have created a program to load bug pages to get the emails. Why
do you think they won't create an account? It would be a one time and easy job.
They could even add themselves to the CC lists and wait to receive notification
when someone add a comment or change the CC list to get new email addresses for
free.


One a side note, yesterday, I changed my email address to a bugzilla specific
address. Today I received my first SPAM on this address. So, except if I had bad
luck, bmo is harvested every day. Bad Thing (tm).
(It's good that I can easily filter them using the From field)
I this sort of thing should be in the e-Mail links
Nahor, how can we get a copy of that harvesting program?
*** Bug 168443 has been marked as a duplicate of this bug. ***
Myk, I said to you that I didn't receive any more spam since my first post here.
Today I just received 2 of them ("Make money without doing ANYTHING!!! A must
look..." and an asian one). So, yes, I'm definitely positive that such an
harvesting program exists :) (They wouldn't get the get the addresses by hand,
would they? :p).
I get those exact same emails almost every day and they are always identical 
except different emails. So annoying...

I had a really good idea of how we could achieve this. You know those sites 
that when you sign up for an account, they give you a bunch of letters and 
numbers in an image with different fonts and stuff? We could probably have a 
CGI that makes an image like that with the email address that would require a 
very sophisticated text-recognition system to retrieve. When the user clicks 
on the name, it brings them to that page.

A simpler alternative is simply /. (Slashdot.org) style email mangling.

My idea is that we could do the simpler method first. We could also make a 
fake Bugzilla email and set it up so that the only way you could learn its 
email is through bugzilla. The hard part is that we would have to keep the 
email a secret until - at least - we make some fake comments with it. Then, 
with the mangling scheme intact, we could see if it gets any spam and if it 
does, we would have to do something a little more tricky (like my first 
suggestion). Still, we wouldn't know for sure if the email has leaked out 
somewhere.

Remember, email spiders can probably piece together common email mangeling. 
There is actually a similiar conversation going on in the mozdev.org mailing 
lists. I think it would be cool if someone made an email alias site to do the 
image idea I came up with for everyone.

I would never know if mangling worked with my own email because its all over 
the place, and I'm sure a lot of yours are too.

bugs.kde.org have bz set up so that only logged in users see the email - others
just see user<space>example<space>com in bug listings, and email addresses in
bugs aren't linkified.

I think that thats a reasonable solution, and I've asked them to contribute
their code for that to us - see http://bugs.kde.org/show_bug.cgi?id=48148

It obviously won't stop everything (eg crawlers folloeing buglist.cgi links and
recombining the email addresses, or stuff like xml.cgi which has to remain
unmunged), but I think that its a good balance.
it's not perfect, but works quite well
additionally to this patch I changed our templates to not use   
<a mailto: when user.login is 0, so that crawlers don't even  
try to readd the @ there. 
  
Ah, I thought you did something nice with template filters. Hmm.
The Bugzilla email account I set up has been the 3rd most spammed of all.
Something needs to be done on THIS side of the problem, not on the servers of
the spamee (as the submitter of the Perl code suggests this should be done).
*Please* someone do something about this SOON! heh
 Hello,

I have visited bugzilla.mozilla.org/show_bug.cgi?id=107714 and noticed that 
your website is not listed on some search engines. I am sure that through our 
service the number of people who visit your website will definitely increase. 
SeekerCenter is a unique technology that instantly submits your website to over 
500,000 search engines and directories -- a really low-cost and effective way 
to advertise your site. For more details please go to SeekerCenter.net.

Give your website maximum exposure today!
Looking forward to hearing from you.

Best Regards,
Vanessa Lintner
Sales & Marketing 
www.SeekerCenter.net     

 
This is getting ridiculous. The email even comes with a picture of the 
bugzilla's website.
 
 
*** Bug 185127 has been marked as a duplicate of this bug. ***
Seems the patch didn't make it into 2.17 :( What happened (or didn't happen)?  

I think the throttling will help too; hope the NAT stuff is fixed to make this
work again.  (Of course, I pity anyone who's still not using filtering.  I
bounce everything spammy, so I don't have to worry much about false positives.)

Yahoo Groups show From:  Jarvis Foobar <jarvis_Foobar@yah...> 
Clicking on <jarvis_Foobar@yah...> opens up a mail form that sends to the user
w/o revealing his email.  Could open up a heavily throttled and logged by userid
& IP (and login-requiring) page providing a real mailto: link.
*** Bug 187836 has been marked as a duplicate of this bug. ***
Boy it would be nice to have this fixed before the next stable release.  It was
my (and my users') first question about bugzilla when we set it up
(bugzilla.targetware.net).

Stephan Kulow's patch fixes the display issue in query search results, but
leaves addresses unmunged in the actual bug listings (e.g. "Assigned To:" and
"Reporter") fields; it's a pretty safe bet that if harvester bots can get to a
query report page, they can then follow the links to the actual bug descriptions.
Incindentally, the requirelogin parameter will probably help a number of sites
with this.
<slightly_offtopic/> For the absolutely paranoid, you can do something like I
did http://www.geocities.com/netdemonz/ ;-)</slightly_offtopic> 
Dave Miller (comment #4): 
Stephen Lee (comment  #14):

I agree that its better to do it properly than just hacking in munging.

For users that don't have a login_name different than their unique ID, we could
force them to create one on their next login.

Stephen suggests we use the emails for those who haven't created a login_name. I
agree with this, but believe this will cause users that don't know about this
change to continue to get SPAM. Perhaps, showing their unique numeric ID would
be better than showing their email for CCs. The same for comments unless their
name is entered into prefs, where that would be shown instead (only for comments).

Users could have their own generated publicly available information page. When
you click on the login_name in the comments (or the numeric ID if they haven't
created one), it would bring you to the personal page you could grab their email
from.

The two methods for bringing up the personal page would be:
show_user.cgi?userid=UNIQUE_ID_HERE
show_user.cgi?login_name=LOGIN_NAME_HERE

This personal information page could have the email obfuscated by something akin
to what I do on page http://www.geocities.com/netdemonz/ (generate the email
text in javascript using a key). This could be done server-side by requiring the
user to paste the key given into a field and hit submit. A crawler wouldn't
understand the english and wouldn't realize they have to paste the PGP key into
the field and submit it.

Also, I would like it if two more fields were added: user_contactpage and
user_description that would be shown on show_user.cgi. The former would be an
URL to a contact page (like http://www.geocities.com/netdemonz/) in case they
need a user to register before sending email. The second would be a description
that allows a few HTML tags (i.e. <a>) where people could talk about themselves
and provide links to bugpages, personal homepages, blogs, etc.

See also bug 184456 about http://lxr.mozilla.org/ being a spammer's paradise
(emails within code).
Reading these comments, I realized why my daily pile of spam has increased since
January 6, when I joined this forum to vote for another bug. I agree that this
one is important and I'm going to vote for it now.

I agree with Brian Bober's approach (#44) except for the "obfuscating" part,
which seems a little overkill. What about a limit of e.g. 3 extracted adresses
per day per user?
There's no reason to make it complicated.  The system should never list email
addresses, period.  

If someone needs to send someone a message, they should use the system, and
negotiate for email if they want.
I agree with #47.  If someone wants to contact another user, they should have to
reenter their username and PW, even if they are already logged into Bugzilla. 
I'm lookiong up and down this list of comments on this bug and see email address
after email address.  This is not good.


1. Have names associated with a unique numerical ID.

2. If someone wants to email another user directly, they can click on the name.

3. Reenter their username and PW to access the email address for that particular
user.

4. Accees to another address would require re-entering the username and PW.

5. If there is a need to email groups, have the end user do a little more work.


Whatever is done, don't have a simple click of the ole mouse button be the means
to directly connect with another individual.  Make it require at least a little
effort.
I'm don't think a name and password each time would stop a spammer.  There is
nothing stopping one from getting his own temp account and harvesting addresses.
 It is already happening.  Another step won't stop a perl spider.

Maybe if there was one of those "fuzzy characters in an image" recognition
thingy like you sometimes see on yahoo for each address requested.

But really, I recommend that email addresses should *never* be shown, and all
communication should be thru bugzilla, through the associated user id's.  

If people are working together they can exchange addresses in private messages,
that no one else can read.   My $.02  =)
Myk beat me to it.  KEY POINT: 
None of the addresses that were obscured, whether in "human-readable" or
"HTML-obscured" form, received a single piece of spam, leading us to conclude
that e-mail address "harvesters" are not presently capable of collecting such
addresses. While this may change as time passes and technology develops, for
the time being it appears that obscuring an e-mail address is an effective means
of avoiding spam.
So just replacing @ with &#64; seems to do the trick, for now!
Hmmm... that &#64; trick sounds like a good idea, do all browsers recognize that
as the same as @?

My other suggestion would be to be a bit tricker... if a user is logged in,
provide a normal e-mail link (to satisfy the developers who like that) if not,
provide a link to either log in and reload the bug w/the e-mail addresses or
login and display that e-mail address (the later would be more obvious behavior,
but I think the former would be more useful).
The longer-term way to go still seems to me to be a full login_name vs. email
address split...

but how about this for a "quick and simple" patch for a short-term measure?

I trust that someone can manage to work through the subtle complexities of this
patch to apply it to the tip.
Attachment #117993 - Flags: review?
In my haste to create a patch doing the blindingly obvious, I missed the fact
that value_quote is entirely independent of html_quote, so that the one-line
change needs to be done twice, once in each function.

Looking at lxr.mozilla.org, it seems that this patch will need to be applied
against Util.pm instead of CGI.pl, but should apply cleanly to that file on the
tip. Might the change also be needed in the new xml_quote function?
Attachment #117993 - Attachment is obsolete: true
Comment on attachment 117997 [details] [diff] [review]
V2: Simpler patch (vs. 2.16.2) to obscure @ in user visible strings


In order to be considered for inclusion, this would have to be made applicable
to the tip rather than 2.16.2

Replacing @ with &#64; may be a good way, but catching it in value_quote and
html_quote does not appear to be sufficient in 2.17
If there is output that doesn't go through one of the html_quote, value_quote 
or xml_quote functions, isn't that a bug in the output code for not filtering 
it's output?

Did I miss something important?
  <passes baton to someone who has a tip installation to test>

As far as I can see, this does have the required effect in 2.16.X, as the page 
looks no different to the user but "View Source" shows &#64; in all the 
relevant places that a spammer might pick up addresses from. 1.0@bugzilla.org 
must be getting a lot of spam though ;-)


As an aside, it'd be nice if the code structure wasn't completely re-arranged 
after every stable release, or stable releases were made more often, so that 
there's a reasonable chance of patches made against the last stable release 
actually applying to the tip.
  <goes into hibernation waiting for next stable release>
One of the major points of the 2.18 cycle was modularizing the code so it's more
flexible.  It's an unfortunate side effect.

Most of the 2.17 templates now use the built-in filtering in Template Toolkit
rather than using the filters provided by Bugzilla.  On the other hand, I thing
we can override and extend TT filters...  bbaetz, gerv, or myk would know better
than I...
And yes, 1.0@bugzilla.org is getting a lot of spam (as is 0@bugzilla.org, seeing
as how some spambots apparently can't deal with a . in the username part of the
address :)

I have both of those addresses blocked at the server with a "Go away, spammer!"
response. :)
WRT comment 57, the templates currently say "[ foo FILTER html]"

Why not just make it "[foo FILTER htmlemail]" when we want to obscure email
addresses?   That would make it possible to use more elaborate mechanisms if
they become necessary. (like <img src="images/atsign.jpg" alt="@">) or whatever.


Why can't we just have a form for sending the email to another user? Click on
their name, and get this:

+------------------------------------------+
| Send email to: [ Michael Hamilton      ] |
|                                          |
| Message:                                 |
| +------------------------------------+   |
| |                                    |   |
| |                                    |   |
| +------------------------------------+   |
| [Send] [Preview]                         |
+------------------------------------------+

That way, emails NEVER have to be displayed publicly no matter what. When user A
sends the email, the respond-to: address is their email and the to: address is
user B, the one they wrote on the form. Either user name or UID would work.
Maybe because spammers could you that for sending their spam? Like they do with
the FormMail script.
Maybe because that would add to the server load? As if it doesn't send enough
mails already :p
No, because it would only work if you were logged in.
And what prevents the spammer to create an account? To have a script to auto-login?
As long as human intervention isn't required (with those images with text in it
and the like), everything can be automated. So for now, the simpler the better.
If just scrambling the @ works then we should use that before making something
like a send-email page.
Later, other scramble algorithm will be the easiest. Next will be to require the
user to be logged in before seeing the addresses.
Last, maybe, a send-email page. But this is too much subject to abuse to be
viable I think (DoS attack)
But logging in leaves an audit trail.  Their account can be disabled and their
IP (or their whole network if necessary) blocked.  That's no different from them
sending direct email, and they still don't have your email address to sell.
Possible solutions to this problem:
1) Obfuscate email addresses on the site - comment #11
2) Allow only logged in users to see email addresses - comment #1
3) Create a form that would allow bugzilla users to contact each other, don't
show email addresses at all. - comment #60

Ideally it would be possible to implement each of these options and have the
administrator choose which ones they wanted to use.

Obfuscate email addresses on the site
Pros:
  Seems to prevent spam
  Does not change the way bugzilla works
  Still allows users to contact each other directly using their existing software.
Cons:
  May not work for very long

Allow only logged in users to see email addresses
Pros:
  Would catch most web crawlers that harvest email addresses.
Cons:
  Would not stop somebody specifically targetting bugzilla for harvesting.
  Would change the way bugzilla works for users who are no logged in.

Create a form that would allow bugzilla users to contact each other
Pros:
  Email address would never be able to leak out.
Cons:
  Spammers might try to use the form instead.
  Bugilla users would have to use bugzilla to contact each other.


Examples:
Slashdot - Slashdot uses an obfuscation method to protect email addresses of
users.  They used to let users choose their own obfuscation, but now obfuscate
everybody's email addresses the same way and change the obfuscation
occasionally.  Their form is a more severe form of obfuscation that is not
automatically removed by your emailer.

Ebay - Ebay used to show address only to signed in users.  They found this did
not work well at protecting addresses and moved to a form for ebayers to contact
each other.
  
You're right. Didn't think about selling the addresses.
As for the audit trail, it's still subject to distributed attacks/spamming. I
think I have read somewhere that some spammers uses plenty of different servers
to send their spam so they can't be blacklisted.
And actually worse, if a spammer can use bugzilla directly to send their spam,
even if it's for a very short time, mozilla.org will get on some online
blacklists and prevent quite a few users from getting their notifications (as
well as direct emailing from fellow bugzilla user for that matter).

So yes, while it would be nice to *never* show email addresses so that they
can't be collected and sold, I think the risk is too high to get mozilla.org (as
well as other bugzilla installations) to get blacklisted.

An intermediate solution is to have to explicitly ask bugzilla to get someone's
email address. That way, you still have your audit trail. Addresses could still
be harvested but bugzilla could automatically trigger an alarm if one user ask
for more than x addresses a day and/or more than y addresses total. With x and y
small enough, even any significant distributed harvesting would be blocked.

Or even better, bugzilla could offer a "private message" system. No mail is sent
(except notification) and the recipient must read the message online (comment
#49). That way, email are never shown and bugzilla won't be blacklisted because
of some spamtrap.
I don’t know about you lot but I really think this bug should have a very VERY
high priority 


<offtopic> I don’t understand Spaming, I don’t understand why its commercially
viable, who want to do business with people who annoy the hell out of you? You
buy something form them all your doing is funding it, same goes for popups!
Lets just Boycott them
Nahor: Regardless of how many servers they use, we can block them all and 
cancel their accounts on Bugzilla. The form-sent emails would have the user's 
name on it, so then we could just contact the admins about it. The SPAM would 
be ignored by the Bugzilla user and definately not replied to. In fact, we 
could put on the top of the generated mail to not respond directly and not 
respond to apparent SPAM, but forward it to spambuster@mozilla.org or 
something.

Stephen:
> Spammers might try to use the form instead.
Would be too much work. There is a lot easier prey. Their accounts would be 
cancelled probably within 2 days of the first SPAM sent.
> Bugilla users would have to use bugzilla to contact each other.
The first time at least. It wouldn't be that big of a deal though.

Now, for a limited time only, you too can enjoy the benefits of my new product...

OK, I don't really have a new product, I'm just trying to prove a point:
Bugzilla already sends email directly to anybody with a Bugzilla account for any
registered user. If we implimented a contact form, it wouldn't really be that
different from what you see here; we would send the message in plain text and
only to Bugzilla users, not any arbitrary address.  There also wouldn't be some
mysterious "send to every e-mail address in the system" checkbox (possible
exception for the admin, but not likely to be used by mozilla.org).
Good point.  3 points:

Is anyone on the bugzilla mailing list subscribed (so to speak) to this bug? 
Are the folks there aware of this discussion?  I used to be on that list;
curious what the focus of the next release of bugzilla is. (i.e. if it's
performance, then we're out of luck, but if it's security, we're in luck).  I
wonder how much duplication of effort is going on/if they have a plan. 

Possible solutions to this problem: (from comment #65, revised)
1) Obfuscate email addresses on the site - comment #11, extant patches.
2) Allow only logged in users to see email addresses - comment #1
3) Create a form that would allow bugzilla users to contact each other, 
   don't show email addresses at all. - comment #60
4)Use existing functionality to allow bugzilla users to contact each other,
   don't show email addresses at all. - comment #47 and comment #69.

IMO, the easiest to code would be a bugzilla setup option implementing #4.
Other options could follow if the demand was there.  I think
bugzilla.mozilla.org, for example, could work fine with just option 4.

Also I imagine the submitted patches would be more readily accepted if they
worked via a setup option, so the old spam-friendly functionality is still
available.  E.g. some bugzilla installations (I set up one, for example) are
private and don't need the protection!
Number 4 is ok except sometimes its necessary to talk privately to someone. 
Also, the signal-to-noise ratio would most likely become too high.
I have not seen this direction addressed fully yet, so here I go:

Like comment #14, I say duplicate the login_name field to emailaddress

add a user preference to show email=[never,login,always]

add an option to update the login_name and email addresses fields. Still 
require unique logins, and emails.

when not showing email address use "UserName [USERID]"

USERID is the pk for the login table.

when doing searches, and adds of cc's etc. do a replace of ##### to the 
foo@bar.com.    that is if it does not have an @ then assume it is a login or 
login id.

never allow @ in new login ids. This means either login=email or login has no @

I could not care less about how paranoid people will directly communicate, this 
is not the purpose. Don’t build a email form, it will just give people an 
excuse to make my use of Bugzilla more annoying.

IN SUMMARY:

user option - login name:
  no @, no whitespace.

user option - email preference: 
  never show email / show email to users / always show email

when not showing email: 
  UserLoginName [USERID]

in boxes where email has traditionally been entered:
  if there is no @ sign, do a look up and translation to the email address/pk
I'd plump for option 2), if any. It's very useful being able to Copy Email
Address someone to CC them, and stuff like that. Option 2) keeps out spammers
(let's face it, no spammer is going to make the effort to get a Bugzilla
account) while still keeping Bugzilla functional for normal users. We could use
the "foo bar com" form for not-logged-in people.

Gerv
> let's face it, no spammer is going to make the effort to get a Bugzilla
> account

Why not? Sure for now they probably won't. If they don't even unscramble
addresses yet, they probably won't open accounts. But is it worth the risk? Is
it better to fix leaking hole in a boat or to replace all rotten wood in it and
prevent futur holes? If the cost is acceptable, I think the second is better.

I personnally like the idea of private messages or sending mail through the
server better because it's a harder to have a leak in that system, and the leaks
will be smaller (limited to bugzilla users, possible to blacklist IPs/user
accounts, ....)
So now, it's far easier to implement something that just scramble emails. But is
implementing one of the two options above that still hard?
So, if we accept the last few points, we should probably do the following....

1) If you are logged out, no email addresses are shown but real names are.
2) If you are logged in, depending on some sort of preference, the email shown
is either the real email or else a value that can be used in any field that
could take a login_name (cc, assignee, etc..).   

Other implications...

Fields such as CC need to look like "myk1 (Myk Melez)" when displayed.  It
should be possible to enter a name using either the login_name or the munged_name.

The system should provide a messaging mechanism that lets a user send messages
to a user knowing only the munged_name.

The system should be able to manufacture an aliases list mapping munged_names to
login_names so that a clever mailer (probably one with an extremely tight spam
filter) can deliver mails addressed to myk1@bugmail.bugzilla.org to the real myk.

> a clever mailer (probably one with an extremely tight spam
>filter) can deliver mails addressed to myk1@bugmail.bugzilla.org to the real myk.

Not a good idea. (Can you say dictionary attack?  My domain (elvey.com) which
has perhaps a dozen valid addresses was recently dictionary attacked (I know
because half were spambait- virgin addresses never used anywhere, ever, and have
been harvested and spammed.)
I think we should assume spammers will sign up for accounts to get email
addresses if they can.  Gerv, why not accept the latest patch (V2) - it's like
2, but much simpler to implement (it's a 2-line patch!) , and if we need more,
we do more.
Re comment 74:
> Is it better to fix leaking hole in a boat or to replace all rotten wood
> in it and prevent futur holes?

Of course it's better in the long term to sort out all the "rotten wood"... 
when the boat's on dry land. But our "boat" is out on the high seas and leaking 
water fast! Can't we at least stop the leak before we start replacing all the 
timbers?

I already did a trivial patch that (on 2.16) should hold for long enough to 
sort out the proper repair, but doesn't apply to the tip, and we'd rather only 
use a "stable" base installation here. It can't be *THAT* hard to do something 
equivalent on the tip? can it? anyone? please?
I read the other day about "Joe Jobs" whereas spammers make life difficult for
anti-spammers by sending fake spams that seem to originate from the anti-spammer
in order to give him trouble. This leads me to believe that a spammer with a
particular vandetta against Mozilla might go out of his or her way and create an
account to spam Bugzilla users.
Comment on attachment 117997 [details] [diff] [review]
V2: Simpler patch (vs. 2.16.2) to obscure @ in user visible strings

Re: comment 77

I don't see why that is a problem. Simple band-aiding can sometimes be an
issue, because the underlying problem is just made more complicated with each
band-aid. I don't see how that would be the case, here. I think checking in
your patch until a better solution is decided upon would be a great idea.
Attachment #117997 - Flags: review?(bugzilla)
Comment on attachment 117997 [details] [diff] [review]
V2: Simpler patch (vs. 2.16.2) to obscure @ in user visible strings

I am changing this into a review request without anyone specifically asked.
Sorry.
Attachment #117997 - Flags: review?(bugzilla) → review?
Comment on attachment 117997 [details] [diff] [review]
V2: Simpler patch (vs. 2.16.2) to obscure @ in user visible strings

this can't get reviewed for checkin because it's not against the tip.

I'm clearing the review rather than denying it because I don't want to say that
it's bad to use it on 2.16.x...

A fix for the tip would be slightly more involved because of the templates
(almost nothing is printed by the user-facing CGIs anymore, not even error
messages), so a custom filter for Template Toolkit would need to be provided
(and then find everyplace in the HTML templates where emails are used and run
them through that filter)
Attachment #117997 - Flags: review?
Hmm. I like the principle behind the current patch, because it allows email
addresses to be present for everyone, and to the user there appears to be no
change. But Dave is right: you would need to write a custom filter (which is OK;
we have several already, in Template.pm I think) and then make sure every email
address got run through it as well as the html filter.

Gerv
Anyone considered removing/obfuscating email addresses in emailed comments?

A spammer could just put one of his email addresses in CC of severall (or simply
all) bugs and wait.... then harvest email addressess in that mailbox.

Does it still make sense to publish user's email addresses? By now many of them
are bugzilla-specific (like mine) and users could setup filters on their
mailboxes to only allow mails sent by bugzilla to reach his mailbox trough this
address, to eliminate any spam or legitimate users (co-developer) from reaching
them directly, bypassing bugzilla.

I got my bugzilla-specific address spammed already (i am voting for this bug)
and if it will become worse i will be forced to set up such filter.
Re: comment 83

If a spammer did do that, then they'd have to enter a valid address. Otherwise,
bugzilla would start getting bounces from them and (I assume) cut them off the
list. I can't imagine any spammer actually inserting a valid address.

By the way, not to be mean or anything, but I personally would not like to see
any more comments on this bug besides somebody saying "I have updated the 2.16
path to use the new template filters. Review?" I would do it myself, if I had
the time to learn the bugzilla code.

-M
Attached patch Simpler Patch against 2.17.3 / template filter (obsolete) — — Splinter Review
I have updated the 2.16 pat[c]h to use the new template filters. Review?


Note that this is just the filter and I haven't yet searched for the relevant
places where it should be activated.
Attachment #120797 - Flags: review?
To illustrate how bad it is (for those that don't know):
http://www.cdt.org/speech/spam/030319spamreport.shtml
*Every* single spam I have received in the last 6 weeks is addressed to my
Bugzilla email address.
Comment on attachment 120797 [details] [diff] [review]
Simpler Patch against 2.17.3 / template filter

Presumably the equivalent "quick and dirty" method would be to replace the
normal HTML filter with a bugzilla_html filter (or whatever) which does the
normal HTML filter and also the obscure_email filter?

i.e. every @ sign for any purpose in html output is written &#64;
Attachment #117993 - Flags: review?
I just changed my Bugzilla email address, and within 2 days I'm getting spammed
at this brand new address.  Bugzilla is the ONLY place it's listed.

I was NOT getting any spam identifiable as from Bugzilla at my old (earthlink)
address, though I've been posting for over 6 months. 

Is it possible NEW EMAIL addresses are somehow being accessed?

SUGGESTION: Set up a mail server for bugzilla members, with extremely strict
spam filtering.  Each user could have an account with USERID@bugzillamail.org
that would be their public address, and would filter spam and then forward to a
hidden email address.  I know this would cost $$, but I'd paypal $10 a year for it. 
Marcus (or someone else) needs to address the issues SLee raised in the code
review of Marcus' code.

I did the same (bugzilla3sd was my third bugzilla contact address) and got spam.
Now any email to that address not from bugzilla itself (or an otherwise
whitelisted address) bounces to the (alleged) sender, with threats and
instructions on how to get around the filtering.  Unfortunately, I expect one
day to get spam at this addresss that gets around this by pretending to be rom
bugzilla itself.  My mail host offers free and paid accounts that provide what
you're looking for.  But expecting every bugzilla user to get a fastmail account
doesn't seem to be as good a solution as the ones already proposed for
addressing this bug. 

I propose marking this bug as dogfood. 
Can someone just put in the patch to obscure @ until we have time come up with
something better? Thanks. It should be reasonably adequate for now.

By the way, how long does it take for you to disappear from the spam lists? Do
they ever remove your email addy? For instance, if we fix this, is Jim Booth now
forever screwed?
Spam lists are forever.  I still get spam sent to an account I haven't posted
news from since 1995 or so.  You can never get off them.

Fixing this bug is important for the whole Mozilla project.  Mozilla depends on
user bug reports, and if people think they will get spammed for submitting them,
then the bugs will go unreported.
Comment on attachment 120797 [details] [diff] [review]
Simpler Patch against 2.17.3 / template filter

According to that report, spam lists get stale relatively quickly, so email
addresses that disappear from the net stop getting most spam after a relatively
short time.
Attachment #120797 - Flags: review? → review?(myk)
Email addresses do NOT EVER expire from spammers' lists.  

See for reference: http://www.spamresource.com/nadine/
That's a story about someone who signed up for a sweepstakes, which is different
from someone harvesting addresses from Bugzilla.  Read the report referenced in
comment 86, which presents evidence that addresses removed from the public web
(like will happen with b.m.o when this patch goes onto that server) get
significantly less spam within a week or two.
I can confirm; after I changed my email in this b.m.o to the current one my old 
account's spam was reduced drastically within weeks (was & is not published 
anywhere else)... and the new email started to recieve it, approximately the 
same amount as on previous address.

After this bug is fixed spammers will probably just hold on to old spam lists 
longer because it won't be so easy to harvest.... but then we'll just change 
our bugzilla-specific email redirectors >:-) 


This looks fine, although we should call the filter just "email" or something
like that - we may do other things to email addresses in the future. 

And how does this affect where we use identity, or whatever that access method
on the Bug object is that produces strings like "Gervase Markham
<gerv@mozilla.org>"?

Gerv
Thats part of User.pm. I've rewitten that for bug 180635, and there is now a
separate email vs identity accessor (which give the same result, but...)
*** Bug 209577 has been marked as a duplicate of this bug. ***
>:-/

From: "Sarah Williams" <return@tm-hosting.com> | This is spam | Add to Address Book
To: "netdemonz@yahoo.com" <netdemonz@yahoo.com>
Subject: BUGZILLA.MOZILLA.ORG
Date: Sun, 6 Jul 2003 19:57:09 +0800

Hi

I visited BUGZILLA.MOZILLA.ORG, and noticed that you're not listed on some
search engines! I think we can offer you a service which can help you increase
traffic and the number of visitors to your website.

I would like to introduce you to Trafficmagnet.com. We offer a unique technology
that will submit your website to over 300,000 search engines and directories
every month.

You'll be surprised by the low cost, and by how effective this website promotion
method can be.

To find out more about TrafficMagnet and the cost for submitting your website to
over 300,000 search engines and directories, visit www.trafficmagnet.com.

I would love to hear from you.

Best Regards,

Sarah Williams
Sales and Marketing
E-mail: sarah_williams@tm-hosting.com
http://www.trafficmagnet.com

This email was sent to netdemonz@yahoo.com. We apologize if this email has
reached you in error.
We honor all removal requests. Please click here to be removed from our mailing
list.
WTF?  Hasn't anything been done yet?  Doesn't anyone else find this type of mail
offensive?  

--------------------------------------------------

Return-path: <williamsdowo22@hotmail.com>
Envelope-to: mozilla@mgmiller.net
Delivery-date: Wed, 16 Jul 2003 06:50:59 -0700
Received: from [80.179.101.202] (helo=afzhg2324.com)
	by mx.mailix.net with smtp (Exim 3.33 #1)
	id 19cmgK-000324-00
	for mozilla@mgmiller.net; Wed, 16 Jul 2003 06:50:57 -0700
From: "Mr.Williams Dowo" <williamsdowo22@hotmail.com>
Reply-To: williamsdowo22@hotmail.com
To: mozilla@mgmiller.net
Date: Wed, 16 Jul 2003 14:49:57 -0700
Subject: HELP ME!
X-Mailer: Microsoft Outlook Express 5.00.2919.6900 DM
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Message-Id: <E19cmgK-000324-00@mx.mailix.net>


Dear Friend=2E

As you read this=2C I don't want you to feel sorry for
me=2C because=2C I believe everyone will die someday=2E

My name is Williams Dowo=2C a merchant in Nigeria=2C in
south africa i have been diagnosed with Esophageal
cancer
which was discovered very late=2C due to my laxity in
caring for my health=2E It has defiled all forms of
medicine=2C and right now I have only about a few
months to live=2C according to medical experts=2E

I have not particularly lived my life so well=2C as I
never really cared for anyone not even myself but my
business=2E Though I am very rich=2C I was never
generous=2C I was always hostile to people and only
focus on my business as that was the only thing I
cared for=2E But now I regret all this as I now know
that there is more to life than just wanting to have
or make all the money in the world=2E
I believe when God gives me a second chance to come
to this world I would live my life a different way
from how I have lived it=2E Now that God ! has called
me=2C I have willed and given most of my property
and assets to my immediate and extended family
members and as well as a few close friends=2E
I want God to be merciful to me and accept my soul
and so=2C I have decided to give alms to charity
organizations=2C as I want this to be one of the last
good deeds I do on earth=2E So far=2C I have distributed
money to some charity organizations in the U=2EA=2EE=2C
Algeria and Malaysia=2E Now that my health has
deteriorated so badly=2C I cannot do this my self any
more=2E I once asked members of my family to close one
of my accounts and distribute the money which I have
there to charity organization in Bulgaria and
Pakistan=2C they refused and kept the money to
themselves=2E Hence=2C I do not trust them anymore=2C as
they seem not to be contended with what I have left
for them=2E
The last of my money which no one knows of is the
huge cash deposit of twenty four million dollars
that I have with a security and finance limited in
Amsterdam=2E
I want you to help me collect this deposit and
dispatch it to charity organizations=2Eemail me at
this address
williamsdowo22=40hotmail=2Ecom
along with your telephone and fax Numbers=2Cso that i
can contact you=2E
I have set aside 30% for you for your time and
patience=2E

God be with you=2E
Williams Dowo
Severity: major → critical
Target Milestone: Bugzilla 2.20 → Bugzilla 2.18
Attached patch patch v2 unrotted (obsolete) — — Splinter Review
Here's attachment 120797 [details] [diff] [review] updated to work with the tip.	Nothing changed
otherwise.
Comment on attachment 128992 [details] [diff] [review]
patch v2 unrotted

Looks good, "works".  r=myk.  I'll check it in since it looks like slee doesn't
have check-in privileges.  Someone still needs to create a patch which uses
this filter everywhere email gets displayed.

re: comment 88, the problem with overloading the HTML filter is that Bugzilla
performance is highly dependent on that being fast and lean.  In fact, we got
the Template Toolkit to back out a change last year that made that filter more
comprehensive but with a high performance penalty.

re: comment 97, perhaps, but the more specific "obscure_email" name is ok for
now while that's really the only thing the filter does.
Attachment #128992 - Flags: review+
a=myk

Checked in.  Leaving bug open in anticipation of second patch.  Anyone have time
to go through templates looking for addresses to filter?

Checking in Bugzilla/Template.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Template.pm,v  <--  Template.pm
new revision: 1.8; previous revision: 1.7
done
Checking in checksetup.pl;
/cvsroot/mozilla/webtools/bugzilla/checksetup.pl,v  <--  checksetup.pl
new revision: 1.241; previous revision: 1.240
done
Checking in t/004template.t;
/cvsroot/mozilla/webtools/bugzilla/t/004template.t,v  <--  004template.t
new revision: 1.25; previous revision: 1.24
done
Attachment #120797 - Flags: review?(myk)
Re: myk Comment 103/104

I don't suppose I'd be checking *that* patch in anyway if I did have such 
privileges... mine was the 2 line V2.16.2 patch intended merely to demonstrate 
that this ought to be a trivially simple change. It was supposed to be a 5-
minute fix!!!


Hunting a little harder, I see that bugzilla 2.16.2 contains the line:
        html => \&html_quote , 
in the section of globals.pl equivalent to the current Template.pm, which 
caused bugzilla's html filter to override the template toolkit's.

Can't we just go back to that model, and override the built-in html filter?


suggested meta-patch:

diff -u -r1.8 Template.pm
@@ -247,8 +247,9 @@
 
-            # Simple filter to obscure the '@' in user visible strings
+            # Override html filter to obscure the '@' in user visible strings
             # See bug 120030 for details
-            obscure_email => sub {
+            html => sub {
                 my ($var) = @_;
+                $var = call_to_the_builtin_template_toolkit_html_filter($var);
                 $var =~ s/\@/\&#64;/g;
                 return $var;
             },



(I'm not familiar with the current code structure as it bears almost no 
resemblance to the last stable release, so someone would need to fill in the 
call_to_the_builtin_template_toolkit_html_filter with something else to make a 
proper patch)
I've just installed this patch and i can't cc to addresses that haven't an valid
account. Is that the meaning of this patch? 
If it's true than the patch is working. (2.17.4 + all patches)
Re: Guenter's comment 106:
CCing required a valid account even before this patch. The patch in 
attachment 12899 [details] just replaces every "@" with "&#64;" in the served pages.
>Can't we just go back to that model, and override the built-in html filter?

We can if there's no performance overhead to it, but that's unlikely.  The HTML
filter gets run thousands of times on many pages, and even a slight increase in
the time it takes to run it can result in a noticable increase in Bugzilla
performance.  Show me numbers that contradict what I'm saying, however, and I'll
change my mind.
I've now made what should be a "proper" patch rather than a meta-patch. As I
don't have CVS, the patch was hand-edited, but I believe it should apply. The
previous already-checked-in attachment 128992 [details] [diff] [review] needs to be applied before this
one.

It removes the "obscure_email" filter, and instead overrides the standard
"html" filter, augmenting it with the same functionality.
Comment on attachment 129021 [details] [diff] [review]
Further patch to apply obscure_email filter globally

Since I'm still only using 2.16.2 I have no way to directly test this. As per
myk's last comment #108, it should be tested in order to evaluate the impact on
performance.

Even if the performance hit is too great, I would still think it better to
combine the filters in this manner rather than having the rather long-winded [%
xxx FILTER html FILTER obscure_email %] in the templates.
Attachment #129021 - Flags: review?
This patch provides contact.cgi.  It allows any bugzilla user to contact any
other bugzilla user.  To use the contact.cgi, you must know the bugzilla id
number of the user and realname of the desired user.

This patch is the first step in providing a fix for this bug.  Once contact.cgi
is in place, links to contact.cgi can replace mailto: links.  Links to
contact.cgi take the form:
contact.cgi?toID=1&toName=Bugzilla+User

Contact.cgi provides a method of emailing users without ever seeing their email
address.  This will prevent spammers from harvesting from bugzilla.
Flags: approval?
So if this is a formmail, couldn't this be used to spam users anyway?

Other technology such as CAPTCHAS could also be introduced (with associated
overhead) to thwart spammers from automatedly using the new formmail.
contact.cgi is different than formmail in several important respects:

1) It cannot be used to send mail to anybody.  It can only be used to send mail
to registered bugzilla users from another logged in, registered bugzilla user.

Formmail typically allows email to be sent to any email address.

2) Contact.cgi never reveals email addresses.  You do not need to know the email
address of the recipient, nor will you learn that address during the send processes.

Formmail typically has email addresses in hidden fields where they can still be
seen by viewing source.


A spammer could use contact.cgi to send email to bugzilla users, but that could
be detected and stopped.  A spammer already has the ability send everybody email
by commenting on bugs.  Fortunately, we don't see much in the way of "Here is a
patch: Buy my herbal viagra."  I also tooks some specific steps to make using
the contact.cgi hard for spammers.

1)  You have to know the recipient's real name.  This prevents a spammer from
using contact.cgi to send spam to bugzilla user's 1 to n.

2) To use contact.cgi, you must be logged in.  This means that you have to have
been through the process of creating and validating an account.  Spammers just
don't go through that trouble.

Note that my contact.cgi patch is a patch against the CVS tip (2.17.4).  It
provides three files:
1) contact.cgi
2) template/en/default/contact/form.html.tmpl
3) template/en/default/contact/sent.html.tmpl

It also modifies
globals.pl:DBID_to_real_or_loginname() - This function was currently unused, it
had some bugs.
template/en/default/global/user-error.html.tmpl - Added two error messages:
invalid_recipient and invalid_referrer
Comment on attachment 129331 [details] [diff] [review]
Patch to provide contact.cgi - Allows bugzilla users to contact each other without knowing email addresses

I'm not sure I see the advantage of this patch...

As stated by the author, mailto: links can be changed to links to this form,
but the main problem is the currently login_name == email_address, and that
this is the primary key by which users specify Assignee, QA, CC, Reporter, etc.
etc.

The patch provides a form to allow contact users, but does nothing to prevent
the email addresses being littered all over the UI (of which mailto: links are
just a small fraction) 

I thought that in the medium term the aim was to separate login_name and
email_address, so that login_name could still be publicly displayed, but that
in the short term we just wanted to avoid email addresses being automatically
harvested by *current* general-purpose spambots.

According to earlier comments, simply replacing the @ with &#64; was sufficient
to confuse general-purpose email harvesters while the "separate login_name and
email_address" was worked on.

Any news from anyone who's in a position to do performance (or other) testing
on my global obscure_email patch in attachment 129021 [details] [diff] [review]? That was intended as a
low-impact "quick fix" that could be easily removed when a better solution was
checked in...
Comment on attachment 129331 [details] [diff] [review]
Patch to provide contact.cgi - Allows bugzilla users to contact each other without knowing email addresses

This will fail miserably.

Here's a short list of problems which you may encounter if you try to contact
users.

first, if you used this feature to contact me based on the contact info for the
comment which i'm writing now you'll trigger mail to somewhere that doesn't
exist.

This is actually typical for me. And I don't intend to change it just because
you think you've 'solved' 'the problem'. The reason my accounts are setup as
they are has very little to do with spam.

second, try contacting regexp@js.bugs or general@browser.bugs
There's absolutely no person who owns these accounts, and there's certainly no
maildrop for them.

third, the correct way to contact me is by triggering mail to my shadow(s),
this is how you should contact the .bugs accounts too.

Yes I know that requests are broken and don't do this. That's a bug and if I
controlled bugzilla I would have blocked their landing until they included
correct mail generation. 

fourth, you don't honor any pref so that people can refuse to get mail from
this thing (sending mail to any of the .bugs accounts which i setup is a
violation of their configuration which explicitly says that they don't want
mail). everything except requests does and again see caveat about requests
being evil and broken and ...

fifth, 
the email has no bounce information and doesn't appear to implicate
bugzilla-daemon. 

sixth,
how does someone who generates an email find out if the email bounced?

--
technical nit about the patch:
we aren't living in 1998 anymore:
+  # The Initial Developer of the Original Code is Netscape Communications
+  # Corporation. Portions created by Netscape are
+  # Copyright (C) 1998 Netscape Communications Corporation. All
+  # Rights Reserved.

In fact, NSCP doesn't exist either. please use a correct MPL template and fill
in a valid date.

http://www.mozilla.org/MPL/boilerplate-1.1/mpl-sh
--

What should you do?
see one of the features described in my votes proposal since the preceding are
addressed by it.

Also. i'd suggest any work for this be split off to another bug. You should
probably fork it off my votes bug since it has a specification.
Attachment #129331 - Flags: review-
I don't see anything to approve here yet.  Request approval again after you have
a reviewed patch ready to check in.

Let's get the obscuring code in here and then open another bug for any more
complex anti-spam stuff.  I'd love to have the obscuring stuff in before 2.17.5 :)
Flags: approval?
Comment on attachment 129021 [details] [diff] [review]
Further patch to apply obscure_email filter globally

we're getting there, but this patch is malformed (line counts were off in the
header so it wouldn't apply it).

I fixed it up and did the rest of the missing stuff, will upload a new patch
shortly.
Attachment #129021 - Flags: review? → review-
Attached patch global obscure patch v4 (obsolete) — — Splinter Review
OK, this takes Stephen Lee's patch, fixed so it'll apply, and adds filtering
for the remaining places I could find unfiltered emails.  What I did is I went
through filterexceptions.pl and removed the exception lines for anything that
looked like an email.  I then ran the filter test and went and added FILTER
html to anything that it flagged.
Attachment #120797 - Attachment is obsolete: true
Attachment #128992 - Attachment is obsolete: true
Attachment #129021 - Attachment is obsolete: true
Attachment #129364 - Flags: review?(myk)
Hmmm... interesting, so email addresses aren't currently filtered?

I assume that "><script>alert('hello');</script><"@example.com is a 
syntactically correct email address, 'cos I just received an email that I sent 
to an address (basically) the same as that. I note that bugzilla currently 
disallows the use of this address though.

My webmail dutifully gave me a "hello" alert box :-/  (goes off to file bug 
report with them)
See spinoff bug 215439.
Attachment #129331 - Attachment is obsolete: true
> Hmmm... interesting, so email addresses aren't currently filtered?

That is correct.  That was left that way when the XSS holes were fixed a while
back under the assumption that the emailregexp enforced making email addresses
be HTML safe by default, however they were flagged in the filterexceptions file
as things that really should be filtered.  Just nobody'd gotten around to it
yet.  Guess this patch takes care of that. :)
i strongly vote for some sort of hiding of emailaddresses.
one option before you change to much of the application is to remove addresses
that has not been used for a while (6 months).

to the database that stores email addresses one would add a field for some kind
of autoID, 
userID   mailAddress
1        mymail@here.now
2        other@else.where

then when an address has not logged in or posted anything within say 6 months,
you simply exchange the mailAdress with the userID, also one could add a note.
userID   mailAddress
1        1 (removed)
2        2 (removed)

that would still make the emailAdress field unique.

this is of course not a full solution but at least poeple that show little
interest in reading/writing bugs will have a little more integrity and hopefully
 get less spam.
for instance this cleanup could be run once in a month.
Whiteboard: DO NOT COMMENT unless you're helping implement this
Ba$tards got my email alias.
Had to add a procmail filter just to weed out the ****.
Not creating a new alias if it'll get spammed too.
Why are we using the "html" filter for this? In the future, we might want to do
more specific processing for email addresses; surely it would be more
future-proof to have a specific "email" filter?

Gerv
There are many cases where we don't know we're filtering email addresses, but we
are.  Such as in the activity log, where all we know we have is an old value and
a new value.  By adding this into the html filter it catches all of those cases
as well.
Fair enough. As long as all browsers understand that entity reference.

Gerv
Attached patch Patch v.5 (obsolete) — — Splinter Review
Now that we've done that global filtering cleanup, this simple patch is all
that's required to spamproof email addresses throughout Bugzilla.

Gerv
Attachment #129364 - Attachment is obsolete: true
Comment on attachment 131041 [details] [diff] [review]
Patch v.5

Dave - rubber-stamp again? :-)

Gerv
Attachment #131041 - Flags: review?(justdave)
Comment on attachment 131041 [details] [diff] [review]
Patch v.5

obscure_email also needs to be removed from checksetup.pl and 004filter.t
Attachment #131041 - Flags: review?(justdave) → review-
Attached patch Patch v.6 — — Splinter Review
With review markups.

Gerv
Attachment #131041 - Attachment is obsolete: true
Attachment #131042 - Flags: review?(justdave)
Comment on attachment 131042 [details] [diff] [review]
Patch v.6

r=justdave
Attachment #131042 - Flags: review?(justdave) → review+
a= justdave on this specific patch, but not to close the bug.  This is a
*really* good begining, but it's only a temporary solution, and the spammers
will eventually catch on and upgrade the email-collecting bots.  So we need to
leave this open for future more-comprehensive anti-spam measures.
I don't know if this should belong in bug 215439, but I don't think the contact
page should contain a user entry field. Instead it should just send out a stock
message "MyRealName" <MyReal@Address> would like to contact you regarding your
<Report|Assignment|CC|Comment on|Change to> Bug <bug#> <bug description>. Then
the requestee can deal with those messages as they see fit.

Alternatively provided this doesn't slow Bugzilla down too much it could have a
contacts table similar to the attachments table which lists all users who have
requested to contact you (again this request automatically gives you the right
to contact them) which when granted would display those address as currently.
neil@parkwaycc.co.uk: Put thos comments on bug 215439.
In Comment 133 we had an "a= justdave" for the current patch, but nothing seems 
to have happened re: checking this in and actually getting it live on b.m.o.  
Marking the approval+ for him to see if this speeds things up ;-)

This bug is getting quite long, and with all the discussion here, it's a long 
read to find out what any the actual proposed solutions are... Might it be 
worth getting things like making login_name != email_address etc. as separate 
RFEs, with this as a meta-bug for the overall problem?
Flags: approval+
Or even better, to keep it clean, we should start a new bug for a more advanced
solution and go ahead and let Gerv mark this one fixed when he checks in his
patch. :)

Since this bug now contains a veritible information overload, we should probably
take a summary of all the ideas presented here and start a new bug with it, so
you can get to the point with less reading.
Assignee: myk → gerv
Blocks: 218917
As per last comment from justdave, I've "start[ed] a new bug for a more advanced
solution". Bug 218917 is for the login_name!=email_address bit, which seems (in 
my biased opinion) the logical next step.

IMHO, any further address munging techniques, user profiles, contact forms, 
etc. should be separate enhancements that build on that rather than being 
rolled into one bug discussing 10 different things like we had here...
Fixed using Patch v.6.

Checking in Bugzilla/Template.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Template.pm,v  <--  Template.pm
new revision: 1.11; previous revision: 1.10
done
Checking in checksetup.pl;
/cvsroot/mozilla/webtools/bugzilla/checksetup.pl,v  <--  checksetup.pl
new revision: 1.250; previous revision: 1.249
done
Checking in t/004template.t;
/cvsroot/mozilla/webtools/bugzilla/t/004template.t,v  <--  004template.t
new revision: 1.28; previous revision: 1.27
done

Gerv
Really fixed.

Gerv
Status: NEW → RESOLVED
Closed: 21 years ago
Resolution: --- → FIXED
fixing summary to reflect what actually got fixed.  The spin-off bug(s) will
handle the rest of the items discussed here.
Summary: Bugzilla bug lists are a spammer's paradise → Munge email addresses to make life difficult for the spammers' spiders
Blocks: 219021
I've done a rather comprehensive write-up of the "Log in to see emails" idea as
bug 219021. I didn't see that anybody else had filed it yet, so there we go.

-M
No longer blocks: 219021
For the folks that aren't aware: b.m.o (bugzilla.mozilla.org) is still running
the *ancient* Bugzilla Version 2.17.1; I expect it'll be a long while before
this gets into a new bugzilla release and before that release then gets
installed on b.m.o!  Someone please post to this bug when that happens.  (And
don't go changing your b.m.o address to your main address 'till that happens!)
Actually, 2.17.5 is not far away, and b.m.o. is going to upgrade to the tip
pretty soon.

Gerv
Attachment #129364 - Flags: review?(myk)
Well, 2.17.5 was just released (as was 2.16.4, both on 11/2) so this bug'll be
fixed in the <b.m.o. is a spammer's paradise> sense soon. 
Munging @ to " at " isn't necessary.  Mung it to &#064; instead.  This will
render as "@" on a browser screen but throw off spiders at least as effectively.

There was a study of how spiderbots work about a months back that recommended
this method.
Eric: that's the solution that this bug implemented.

Gerv
*** Bug 251581 has been marked as a duplicate of this bug. ***
Whiteboard: DO NOT COMMENT unless you're helping implement this → See comments 133 through 142 for what this did and bugs that expand on it
(In reply to comment #144)
> Actually, 2.17.5 is not far away, and b.m.o. is going to upgrade to the tip
> pretty soon.
> 
> Gerv

Gerv, your comment is from Sept 2003, these days b.m.o. is running 2.17.6, but
it seems to me that the e-mail addresses are still spider-readable on b.m.o.?

Test: listed a new address exclusively on b.m.o. last week, got first spam today.
The Bugzilla on b.m.o. is running with this fix, yes. Please let us know what
sort of spam level you start getting as time goes on.

I'd say that if we wanted to do further work, the right thing to do is only give
email addresses in mailto: form for logged-in users. That may help a bit more. 

Gerv
Two questions...

* Will harverster writers create an account just to grab emails? (They can spam
from a different IP address and email than they sign up from)

* Will the harvested emails eventually get forgotten by the harvesters?
Brian:

a) probably not. We're not that big in the grand scheme of things.
b) no idea - ask the spammers!

Gerv
Further discussion of having login be required for seeing emails can go in bug
219021. :-)
I wouldn't mind a link to a website making them jump through hoops to get my
email address be in place of my email. It'd be nice if we could have personal
URLs added and choose between them or emails showing.
*** Bug 256920 has been marked as a duplicate of this bug. ***
I just got the first spam (paypal phishing, actually) to my (bugzilla-exclusive
Bug 120030 -protected) address, and we can fairly safely assume others have/will
soon.  Looks like its now time to go ahead and implement a solution along the
lines of Comment #146 or Bug 219021 or even Bug 215439 or Bug 218917.  Comment
#137 suggests reopening this bug is INappropriate, (overriding Comment #133);
perhaps Comment #146 needs its own bug?
As the status whiteboard appropriately says: See comments 133 through 142 for
what this did and bugs that expand on it.
Incidentally, Patch v.6 (Attachment 131042 [details] [diff]) did not obfuscate email addresses that appear in bug comments.  For example, look at the source in Bugzilla for this address:  foo@bar.com

This issue will likely be fixed by a patch for Bug 316826 (see Attachment 222002 [details] [diff] for a fix).
QA Contact: matty_is_a_geek → default-qa
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: