1259725 - [spam] Make user profile not indexable

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: Profiles, enhancement)

Description

[Jeremie Patonnier :Jeremie]

What problem would this feature solve?
======================================
Some spammer just create profile to promote their "service". We should discourage that behavior by making user profile not indexable by search engines.

Who has this problem?
=====================
Registered users of MDN

How do you know that the users identified above have this problem?
==================================================================
Spam account that provide unwanted services (if not illegal) usually do not edit on MDN and just try to get their profile reference by search engines.

How are the users identified above solving this problem now?
============================================================
MDN profile are indexed by search engine, making them valuable for spammer.

Do you have any suggestions for solving the problem? Please explain in detail.
==============================================================================
Let's make user profile not indexable.

Is there anything else we should know?
======================================

Comment 1

[John Whitlock [:jwhitlock]]

@groovecoder - You've claimed that spam on profile pages isn't as valuable. Can you explain why?

Comment 2

[Luke Crouch [:groovecoder]]

Simply because the profile pages are not as much traffic. But I supposed spammers don't really care if the marginal cost of spamming another type of page is effectively 0.

Comment 3

[John Whitlock [:jwhitlock]]

The current situation on MDN profile pages:

* The "About Me" field does not allow HTML, including links [1]. URLs are not turned into links [2].
* There are website fields ("Website", "Twitter", etc.) that are turned into links, with rel="me external" (_not_ rel="nofollow").
* Website fields are validated that they are valid URLs.
* Profile pages are linked from document's "contributor bar", from dashboards, etc., without rel="nofollow"
* robots.txt allows scraping the profile pages
* The meta tag on profile pages is <meta name="robots" content="index, follow">, and property tags for Twitter and Facebook sharing are included.

[1] https://developer.mozilla.org/en-US/profiles/jwhitlock
[2] https://developer.mozilla.org/en-US/profiles/sheppy
[3] https://github.com/mozilla/kuma/blob/master/kuma/users/models.py#L96-L131

Comment 4

[John Whitlock [:jwhitlock]]

The tag is now <meta name="robots" content="index, follow">.  This work was done in bug 1353079.

Comment 5

[John Whitlock [:jwhitlock]]

Oops, I mean it is now:

<meta name="robots" content="noindex, nofollow">