Closed Bug 864423 Opened 11 years ago Closed 11 years ago

set up a robots.txt and humans.txt to allow google(etc)-crawling of our content

Categories

(Webmaker Graveyard :: Thimble, defect)

Product:
Webmaker Graveyard
Component:
Thimble
Platform:
x86
macOS
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED INVALID

People

(Reporter: michiel, Unassigned)

Details

with the ability to have stably saved pages rather than constant new publications, we can allow search engines to discover our content. We'll need to set up a robots.txt (and a humans.txt for the human factor) to indicate that doing so is okay.
Hi Pomax - just a note that I don't think we should place this as a high priority for this week
given the distributed nature of how we're now doing publishing, generating robot.txt files for each domain seems a bad plan. invalidating. We might eventually come up with an alternative, but for the moment these files are simply non-issues
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.