Closed
Bug 568344
Opened 15 years ago
Closed 15 years ago
Fix robots.txt file on MDC
Categories
(mozilla.org Graveyard :: Server Operations, task)
mozilla.org Graveyard
Server Operations
Tracking
(Not tracked)
VERIFIED
FIXED
People
(Reporter: sheppy, Assigned: oremj)
Details
Currently, spiders are being asked to totally ignore MDC. This isn't exactly ideal. Instead, the robots.txt that MindTouch provided should be used. Pete should be sending that along shortly.
|
Reporter | |
Comment 1•15 years ago
|
||
User-Agent: *
Crawl-delay: 5
Sitemap: sitemap.xml
Request-rate: 1/5
Disallow: /@api/deki/*
Disallow: /*feed=rss
Disallow: /*type=feed
Disallow: /skins
Disallow: /template:
|
Assignee | |
Comment 2•15 years ago
|
||
Fixed.
Status: NEW → RESOLVED
Closed: 15 years ago
Resolution: --- → FIXED
|
||
Updated•15 years ago
|
Status: RESOLVED → VERIFIED
|
||
Updated•10 years ago
|
Product: mozilla.org → mozilla.org Graveyard
You need to log in
before you can comment on or make changes to this bug.
Description
•