903085 - Make spam unattractive with nofollow

Status: RESOLVED FIXED

(Webmaker Graveyard :: Thimble, defect)

Description

[Brett Gaylor [:brett]]

We're starting to get spam. rel="nofollow" would be a deterrent. Possible to do this in a way that the user can't edit?

Comment 1

[Jon Buckley [:jbuck]]

bleach is capable of doing this to a document: http://bleach.readthedocs.org/en/latest/linkify.html

Comment 2

[Pomax]

note that we're ditching bleach incredibly soon.

Comment 3

[Pomax]

based on the informaiton from https://support.google.com/webmasters/answer/96569?hl=en, we can add a general nofollow rule in the finalizedHTML that is used for the embed shell - remixing it won't show that tag (just like our <base> tag) and it should prevent google etc. from following spam page that people create through thimble

Comment 4

[Pomax]

Attachment: https://github.com/mozilla/thimble.webmaker.org/pull/195

Comment 5

[David Humphrey (:humph)]

Comment on attachment 787729 [details] [review]
https://github.com/mozilla/thimble.webmaker.org/pull/195

This isn't quite as good as link-level nofollow, but it probably does what we want for a lot less complexity.

Comment 6

[Pomax]

curious why you say that? It does exactly the same as link-level nofollow in our case, since we would be linkifying all links anyway? Crawlers that actually respect nofollow (like google etc) should behave the same way whether they see this nofollow or nofollows on every individual link, unless I missed something in the article

Comment 7

[David Humphrey (:humph)]

I just mean it's a big hammer that also covers any links we inject into the doc vs. ones the user does.  It's not critical, since we isolate our stuff in the details iframe.

Comment 8

[Pomax]

ahh. yeah, given that all our links go into our shell rather than the embedded data I went with the full-doc nofollow. In it goes.

Comment 9

[[github robot]]

Commit pushed to master at https://github.com/mozilla/thimble.webmaker.org

https://github.com/mozilla/thimble.webmaker.org/commit/d061910ce076282192b13976af1e661cf9f50090
Merge pull request #195 from Pomax/bug903085

added a nofollow rule for finalized HTML, to prevent spam indexing