with the ability to have stably saved pages rather than constant new publications, we can allow search engines to discover our content. We'll need to set up a robots.txt (and a humans.txt for the human factor) to indicate that doing so is okay.
Hi Pomax - just a note that I don't think we should place this as a high priority for this week
given the distributed nature of how we're now doing publishing, generating robot.txt files for each domain seems a bad plan. invalidating. We might eventually come up with an alternative, but for the moment these files are simply non-issues
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.