Closed Bug 1199046 Opened 9 years ago Closed 9 years ago

Add in-repo robots.txt to be served by WhiteNoise

Categories

(Tree Management :: Treeherder, defect, P3)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: emorley, Assigned: emorley)

References

Details

Attachments

(1 file)

Stage/prod have a generic robots.txt served by Apache (from /var/www/html/robots.txt), but Heroku has nothing: $ curl -I https://treeherder-heroku.herokuapp.com/robots.txt HTTP/1.1 404 NOT FOUND Connection: keep-alive Server: gunicorn/19.3.0 Date: Thu, 27 Aug 2015 00:59:35 GMT Transfer-Encoding: chunked Content-Type: text/html; charset=utf-8 Via: 1.1 vegur We should add one to `ui/` so it gets copied to `dist/` and served by WhiteNoise.
Once we fix this, we can request that the prod/stage Apache alias for the generic robots.txt be removed, making it easier for us to fix things like bug 1196997 ourselves, since it won't require puppet changes.
Blocks: 1196997
Attachment #8653500 - Flags: review?(cdawson) → review+
Commit pushed to master at https://github.com/mozilla/treeherder https://github.com/mozilla/treeherder/commit/f7b14e03e7d1edbd728077955029cc2c844299e4 Bug 1199046 - Add an in-repo robots.txt to be served by WhiteNoise Stage/prod's robots.txt is currently served by Apache, and is in the private puppet repo. Since Heroku can't use it, we need to check one into our repo, that will then be served by WhiteNoise along with the other static files. The robots.txt file here is a direct copy of that currently at: https://treeherder.mozilla.org/robots.txt Once this change is deployed, we can remove the alias in the stage/prod Apache config, leaving WhiteNoise to serve this file. As an added bonus, it will also then be much easier for us to alter it in the future.
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Blocks: 1201033
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: