Closed
Bug 521466
Opened 15 years ago
Closed 15 years ago
Stop search engines from indexing Contributors and Off Topic fora
Categories
(support.mozilla.org :: General, defect)
support.mozilla.org
General
Tracking
(Not tracked)
VERIFIED
FIXED
1.4.1
People
(Reporter: cilias, Assigned: jsocol)
References
()
Details
(Whiteboard: sumo_only)
Attachments
(1 file)
|
433 bytes,
patch
|
morgamic
:
review+
|
Details | Diff | Splinter Review |
It was determined last month that the misplaced traffic in the Contributors forum was caused by visitors arriving via Google search results. Now that we have pretty forum URLs, we should prevent Google from indexing the contributors forum.
James says this is super simple and almost zero risk so let's just do it in 1.4.1
Assignee: nobody → james
Target Milestone: 1.4.2 → 1.4.1
|
Assignee | |
Comment 2•15 years ago
|
||
Based on my research, Google, Yahoo and MSN/Bing (at least) support the wildcard matching. Bing's "Robots.txt validator" didn't complain about "Allow: " directives but it was not clear from other sources that Bing supports it. These lines should block the Contributors' and Off Topic forums in all locales in crawlers that support wildcards, and will hopefully be ignored by anything else.
Attachment #405571 -
Flags: review?(morgamic)
|
||
Updated•15 years ago
|
Attachment #405571 -
Flags: review?(morgamic) → review+
|
|
Assignee | |
Comment 3•15 years ago
|
||
r53170
Status: NEW → RESOLVED
Closed: 15 years ago
Resolution: --- → FIXED
Summary: Stop Google from indexing Contributors forum → Stop search engines from indexing Contributors and Off Topic fora
|
||
Comment 4•15 years ago
|
||
https://support-stage.mozilla.org/robots.txt matches the patch; verified FIXED based on code inspection (I don't think there's a way to tell until Google's index updates itself based on these rules in prod).
Status: RESOLVED → VERIFIED
|
|
Assignee | |
Updated•15 years ago
|
Whiteboard: sumo_only
You need to log in
before you can comment on or make changes to this bug.
Description
•