607652 - Replace robots.txt after svn up on master.support and support-stage

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[James Socol [:jsocol, :james]]

Should be straight forward: after the svn update, we just need to replace the contents of robots.txt with

    User-agent: *
    Disallow: /

Something like:

    echo -e "User-agent: *\nDisallow: /" > /path/to/robots.txt

should do it.

Comment 1

[Reed Loden [:reed]]

or just add a rewriterule to point to an alternate robots.txt that's in svn (robots-stage.txt or something)... that might be better future-wise.

Comment 2

[James Socol [:jsocol, :james]]

(In reply to comment #1)
> that might be better future-wise.

Very soon we won't be using svn anymore, and we'll need to figure out a new way to do robots.txt. Nothing is future-proof right now.

Comment 3

[Reed Loden [:reed]]

(In reply to comment #2)
> Very soon we won't be using svn anymore, and we'll need to figure out a new way
> to do robots.txt. Nothing is future-proof right now.

Why does the VCS matter? s/svn/whatever vcs you're using at the time/

same difference, no?

Comment 4

[James Socol [:jsocol, :james]]

Can we please just do this for now? We won't be able to serve a physical file at /robots.txt for much longer, and will have to handle that in the new app. In the meantime, this is really an easy thing to do.

Comment 5

[Shyam Mani [:fox2mike]]

James, 

robots.txt already has Disallow: / as the first directive on support-stage, I just added it on master too. I think that'll do. 

http://master.support.mozilla.com/robots.txt
http://support-stage.mozilla.com/robots.txt