623939 - Add noindex meta tags

Status: VERIFIED FIXED

(addons.mozilla.org Graveyard :: Public Pages, defect)

Description

[Wil Clouser [:clouserw]]

We have a robots.txt that disallows crawlers, but Google still indexes the pages if anyone else links to them.  If we add a <meta> tag to the pages it will prevent them from going in google's index.  See http://www.google.com/support/webmasters/bin/answer.py?answer=93710 for details.

The meta tag should be in the main template and have an {% if not settings.ENGAGE_ROBOTS %} around it.

Marking this as a good first bug since it's a 3 line change and I've spelled it out pretty well (I hope).

Comment 1

[Wil Clouser [:clouserw]]

I think putting this in templates/base.html covers us.

Comment 2

[Paul Booker]

Thanks Wil, just wanted to let you know that i'm going to work on this now ..

Best,
Paul Booker
Appcoast

Comment 3

[Wil Clouser [:clouserw]]

https://github.com/jbalogh/zamboni/commit/f8a983e38a897859c537ddda683ba43a7a59ddfb

Comment 4

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified FIXED (code, not, obviously, the Google/bot-blocking behavior, yet):

<meta charset="utf-8">
              <meta name="robots" content="noindex">
          <title>Manage My Add-ons :: Developer Hub :: Add-ons</title>