Closed Bug 866827 Opened 11 years ago Closed 11 years ago

make input not crawlable

Categories

(Input :: General, defect, P1)

Product:
Input
Component:
General
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: willkg, Assigned: willkg)

Details

(Whiteboard: u=user c=dashboard p=1 s=input.2013q2) 

input data shouldn't be crawlable. Need to look at (and fix) the robots.txt file so that google and other friends aren't crawling the site.
This should be pretty straight-forward. It entails figuring out if there is a robots.txt file and if so, fixing it so that the site says "don't index me!" and if there isn't one, adding one that says "don't index me!"

Marking this as mentored.

If you're interested in working on this bug, add a comment here or ping willkg on #input in irc.mozilla.org.
Whiteboard: u=user c=dashboard p= s=input.2013q2 → u=user c=dashboard p= s=input.2013q2 [mentor=willkg]
Grabbing this and removing the mentor tag.

Pull request: https://github.com/mozilla/fjord/pull/100

Landed in master in: https://github.com/mozilla/fjord/commit/87dcb66
Assignee: nobody → willkg
Whiteboard: u=user c=dashboard p= s=input.2013q2 [mentor=willkg] → u=user c=dashboard p= s=input.2013q2
Pushed to production just now.
Whiteboard: u=user c=dashboard p= s=input.2013q2 → u=user c=dashboard p=1 s=input.2013q2
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.