616287 - Place RewriteRules under VCS

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Austin King [:ozten]]

After discussing upcoming L10n work, it's become apparent that we'll need to change the RewriteRules on an ongoing basis.

We'll want the change to a Django supported locale and the Apache config changes to be in the same commit/deployment.

We want to automate this process, so we don't have to file IT requests each time a locale is added to Django.

Can we:
1) breakout RewriteRules into a file
2) Source this file from current Apache Config
3) Add file to current git repo[1] which IT and webdev have access to
4) Update stage9 cron so that when this file changes, apache is reloaded
5) Update production deployment scripts to use this new file

[1] http://github.com/fwenzel/mdn

Comment 1

[Jeremy Orem [:oremj]]

Are we just going to do .htaccess here?

Comment 2

[Fred Wenzel [:wenzel]]

I am fine with that (if it works with wsgi, I've never combined the two). .htaccess is supposedly slower than dropping this into the confs directly, but considering mozilla.com can handle using it, I have no doubt MDN can as well.

Comment 3

[Austin King [:ozten]]

I haven't had time to setup the .htaccess, test it, and check it in.

This work needs to be done before I can get to the .htaccess work.

Comment 4

[Jeremy Orem [:oremj]]

(In reply to comment #2)
> I am fine with that (if it works with wsgi, I've never combined the two).
> .htaccess is supposedly slower than dropping this into the confs directly, but
> considering mozilla.com can handle using it, I have no doubt MDN can as well.

Yeah, the only difference is apache has to access the file, but compared to the speed of our php/python apps it doesn't really make a difference.

Assigning this to Austin for .htaccess work.

Comment 5

[Austin King [:ozten]]

Done in http://github.com/fwenzel/mdn/commit/5c78ae7

1) Can you copy/paste the existing Django ReWriteRules into this bug

2) Please delete old Django ReWriteRules

3) Please ensure that anytime configs/htaccess changes, it is copied into dekiwiki's root:
cp /path/to/mdn/configs/htaccess /var/www/dekiwiki/.htaccess

Maybe update bin/update_stage.sh?
Please note how this is done, we'll want to eventually update the push to production script as well.

4) Please make sure AllowOverride/Options will permit .htaccess

5) Any feedback or other things we should pull out of the Apache config and stick into this .htaccess file?

Note: this update includes the new /demos/ urls for Demo Room work.

Comment 6

[Jeremy Orem [:oremj]]

(In reply to comment #5)
> Done in http://github.com/fwenzel/mdn/commit/5c78ae7
> 
> 1) Can you copy/paste the existing Django ReWriteRules into this bug
    RewriteRule ^/((en-US|de|es|fr|ja|pl|sq|zh-TW|zh-CN)/?)$ /mwsgi/$1 [PT,L]
    RewriteRule ^/(((en-US|de|es|fr|ja|pl|sq|zh-TW|zh-CN)/)?addons/?)$ /mwsgi/$1 [PT,L]
    RewriteRule ^/(((en-US|de|es|fr|ja|pl|sq|zh-TW|zh-CN)/)?docs/?)$ /mwsgi/$1 [PT,L]
    RewriteRule ^/(((en-US|de|es|fr|ja|pl|sq|zh-TW|zh-CN)/)?mobile/?)$ /mwsgi/$1 [PT,L]
    RewriteRule ^/(((en-US|de|es|fr|ja|pl|sq|zh-TW|zh-CN)/)?mozilla/?)$ /mwsgi/$1 [PT,L]
    RewriteRule ^/(((en-US|de|es|fr|ja|pl|sq|zh-TW|zh-CN)/)?search/?)$ /mwsgi/$1 [PT,L]
    RewriteRule ^/(((en-US|de|es|fr|ja|pl|sq|zh-TW|zh-CN)/)?web/?)$ /mwsgi/$1 [PT,L]

> 
> 2) Please delete old Django ReWriteRules

Done.

> 
> 3) Please ensure that anytime configs/htaccess changes, it is copied into
> dekiwiki's root:
> cp /path/to/mdn/configs/htaccess /var/www/dekiwiki/.htaccess
> 
> Maybe update bin/update_stage.sh?
> Please note how this is done, we'll want to eventually update the push to
> production script as well.

I've just symlinked .htaccess from the django checkout to the dekiwiki docroot.
> 
> 4) Please make sure AllowOverride/Options will permit .htaccess
> 

Done.
> 5) Any feedback or other things we should pull out of the Apache config and
> stick into this .htaccess file?
I think this is fine for now.

Comment 7

[Austin King [:ozten]]

Apache's config is interfering. 

oremj will advise on next steps, we've discussed a few possibilities in IRC.

Comment 8

[Fred Wenzel [:wenzel]]

Do we have an update on this? I wanted to add Croatian (hr) today, and I am not sure what I need to do to get the rewrite rules updated.

Comment 9

[Jeremy Orem [:oremj]]

Do we just want to move all the deki rewrites to .htaccess?

Comment 10

[Austin King [:ozten]]

Yes please.

Comment 11

[Jay Patel [:jay]]

oremj:  Let us know when the rewrite rules are in .htaccess so we can test with adding the hr locale.  Thanks!

Comment 12

[Jeremy Orem [:oremj]]

Attachment: Devmo vhost

Comment 13

[Jeremy Orem [:oremj]]

I've attached the devmo vhost. Can you transpose all those rewrites and check them in to .htaccess?

Comment 14

[Austin King [:ozten]]

Luke will take this and commit to master. Will ship with Demo Room release.

Comment 15

[Luke Crouch [:groovecoder]]

Jeremy can you remove the rewrite stuff from the vhost file so this .htaccess will be definitive location?

Comment 16

[Luke Crouch [:groovecoder]]

oh, though I guess we should only do that when 0.9.3 goes live

Comment 17

[Jeremy Orem [:oremj]]

Yeah, I am I going to remove it when everything is ready to go.

Comment 18

[Jay Patel [:jay]]

Can we get these changes up on stage9 so we can enable the new locales there and give the l10n community a chance to see their localizations and verify the strings?

Comment 19

[Austin King [:ozten]]

We need /admin-media to serve up static content. According to Bug#628860 this is mostly setup. Is this something we can fix in the .htaccess ourselves? Or should we file a new IT bug?

Comment 20

[Jeremy Orem [:oremj]]

I don't think that is possible to do in .htaccess, but I have fixed the Aliases in the vhost.

Comment 21

[Jeremy Orem [:oremj]]

I've removed all the rewrites from the vhost on developer-stage9 and now the site seems to be broken. Did this work out on your dev deki/django set up?

Comment 22

[Luke Crouch [:groovecoder]]

I don't have a local rewrite setup, but ozten does. Ozten?

Comment 23

[Luke Crouch [:groovecoder]]

(In reply to comment #21)
> I've removed all the rewrites from the vhost on developer-stage9 and now the
> site seems to be broken. Did this work out on your dev deki/django set up?

Can I ssh into the stage9 box, and/or can you post the new vhost file?

Comment 24

[Jeremy Orem [:oremj]]

Attachment: rewriteless vhost

Comment 25

[Luke Crouch [:groovecoder]]

I notice in there the doc root is different?

/data/www/dekiwiki9 instead of /data/www/dekiwiki

Is that on purpose? Does /data/www/dekiwiki9 have the .htaccess link too?

Comment 26

[Jeremy Orem [:oremj]]

/data/www/dekiwiki is developer-stage and /data/www/dekiwiki9 is developer-stage9. .htaccess is now symlinked to both locations. Any rewrites that start with a '^/' will need to be altered since they will never match in the .htaccess context.

Comment 27

[Luke Crouch [:groovecoder]]

Sorry, my apache fu is weak. Why won't the ^/ rewrites match from within the .htaccess file? I thought Rewrite operated on the full url regardless of where it's defined.

On what URL will Rewrite operate in the new .htaccess file? e.g.:

RewriteRule ^/contests/$ http://labs.mozilla.com/contests/extendfirefox/ [R=302]

should change to what?

Comment 28

[Jeremy Orem [:oremj]]

http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html#rewriterule contains some info on per directory rewrites.

Comment 29

[Luke Crouch [:groovecoder]]

Sorry again, my apache fu is still weak. I tried both RewriteCond and taking out the leading slash ...

    RewriteCond %{REQUEST_URI} ^/contests/$
    RewriteRule ^/contests/$ http://labs.mozilla.com/contests/extendfirefox/ [R=302]
    RewriteRule ^contests/extendfirefox(/.*)? http://labs.mozilla.com/contests/extendfirefox$1 [R=302]

But https://developer-stage9.mozilla.org/contests/ still 404's? Unless the .htaccess file just hasn't updated yet. How often does stage9 poll & update from github?

Comment 30

[Jeremy Orem [:oremj]]

It should update every 5 to 10 min. I'd recommend working on it locally. Sounds hard to fix when changes only show up every x min.

Comment 31

[Luke Crouch [:groovecoder]]

Cleaned up the .htaccess file so all url's @ developer-stage9.mozilla.org should work.

Comment 32

[Austin King [:ozten]]

This may be the root cause of Bug#638914.

@oremj - Can you confirm what rules are left in our Apache's config? Did we move 100% of the rules into https://github.com/fwenzel/mdn/blob/master/configs/htaccess

Comment 33

[Jeremy Orem [:oremj]]

Bug 638914 is fixed. Can we close this as well?

Comment 34

[Jeremy Orem [:oremj]]

Reopen if anything else needs to be done.