Closed Bug 665231 Opened 9 years ago Closed 9 years ago

Add robots.txt

Categories

(Websites :: markup.mozilla.org, defect)

defect
Not set

Tracking

(Not tracked)

VERIFIED FIXED

People

(Reporter: wenzel, Assigned: wenzel)

Details

(Whiteboard: [qa-is-happy])

Googlebot is sucking in most of Markup, and it shouldn't.

robots.txt should restrict Google's access to the parts that are actual content, like the about page.
https://github.com/mozilla/markup/commit/6d10746

Added! Matt, to test this, check out if robots.txt is accessible on dev, and if it looks reasonable :)
Assignee: nobody → fwenzel
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Target Milestone: --- → 1.1
QA verified ... looks reasonable to me
Status: RESOLVED → VERIFIED
Whiteboard: [qa-]
How is it qa- if you verified it ;)
haha ... true dat :)
Whiteboard: [qa-] → [qa-is-happy]
You need to log in before you can comment on or make changes to this bug.