759345 - [SEO] SuMoMo articles are not indexed by search engines (Google, Bing, Yahoo!, etc.)

Status: RESOLVED FIXED

(support.mozillamessaging.com Graveyard :: General, defect)

Description

[Kohei Yoshino [:kohei]]

I just realized that SuMoMo's knowledge base articles cannot be searched with Google...

https://www.google.comsearch?q=site:support.mozillamessaging.com
http://www.bing.com/search?q=site:support.mozillamessaging.com
http://search.yahoo.com/search?p=site:support.mozillamessaging.com

You are preventing crawlers from indexing those pages:

https://support.mozillamessaging.com/robots.txt

It should be fixed ASAP.

Comment 1

[James Socol [:jsocol, :james]]

Critical, but easy fix. Set ENGAGE_ROBOTS in settings_local to True.

Comment 2

[Kohei Yoshino [:kohei]]

A link in my comment 0 was wrong. It should be
https://www.google.com/search?q=site:support.mozillamessaging.com

Comment 3

[Roland Tanglao needinfo please :rolandtanglao, :mohnkuchen, :adobo, :sinigang, :roland]

i believe only :gozer can fix this on the server, so assigning to him

Comment 4

[Andrei Hajdukewycz [:sancus]]

Yeah, gozer needs to fix this.

Comment 5

[Philippe M. Chiasson (:gozer)]

Added to settings_local.py:

ENGAGE_ROBOTS = True

And restarted the server. Not sure how to verify it worked, however.

Comment 6

[Kohei Yoshino [:kohei]]

Looks good to me.
https://support.mozillamessaging.com/robots.txt

Comment 7

[James Socol [:jsocol, :james]]

https://support.mozillamessaging.com/robots.txt

With ENGAGE_ROBOTS = False, robots.txt is something more like

User-agent: *
Disallow: /

This is to prevent staging servers from being indexed.