1146915 - Stop mounting MDN in a subdirectory, fix Apache config

Status: RESOLVED INCOMPLETE

(Infrastructure & Operations Graveyard :: WebOps: Community Platform, task)

Description

[Jannis Leidel [:jezdez]]

MDN is currently mounted under the subdirectory "/mwsgi" in the Apache config (/etc/httpd/mozilla/domains/developer.mozilla.org.conf) and then we use the mod_rewrite engine to point requests to "/*" at it (https://github.com/mozilla/kuma/blob/c3ea7928fac2a4a22aa4d20e1f44273828515033/configs/htaccess#L138-L139). 

Since the rewrite engine doesn't come for free we should simply mount the WSGI app under the root. We already have plenty of aliases set up for other non-WSGI endpoints such as static files.

See https://code.google.com/p/modwsgi/wiki/QuickConfigurationGuide#Mounting_At_Root_Of_Site for more information for how this should be done instead.

In general we should start making the htaccess file less complex as well to make us independent from Apache for the move to AWS.

Comment 1

[C. Liang [:cyliang]]

jezdez: Is it sufficient to send a quick note out to the mdn-dev  mailing list to arrange a time & day this week to test this in stage?


For the test, I’d like to:
   - temporarily change the Apache config in stage to have:
      WSGIScriptAlias / /data/www/developer.mozilla.org/kuma/wsgi/kuma.wsgi
   - temporarily change the .htaccess file to remove “mwsgi” from certain rewrites [1]

I'd like to test this because I'm not sure if this will break processing of the .htaccess file, which is linked into the document root.


[1] https://github.com/mozilla/kuma/blob/c3ea7928fac2a4a22aa4d20e1f44273828515033/configs/htaccess#L126-L129 and https://github.com/mozilla/kuma/blob/c3ea7928fac2a4a22aa4d20e1f44273828515033/configs/htaccess#L139

Comment 2

[Jannis Leidel [:jezdez]]

Yep, sounds great, :cyliang.

Comment 3

[C. Liang [:cyliang]]

The changes did not break processing of the .htaccess file.  In testing, I found that some of the rewrites did not work correctly, but these may not be working independently of the mwsgi changes.


Before and after testing changes to mwsgi, I did empty out the load balancer caches for developer.mozilla.org


* Old landing pages 
(https://github.com/mozilla/kuma/blob/c3ea7928fac2a4a22aa4d20e1f44273828515033/configs/htaccess#L91-L95)

1. https://developer.allizom.org/en-US/mobile  -> https://developer.allizom.org/en-US/docs/Mozilla/Mobile (worked)
2. https://developer.allizom.org/en-US/mozilla -> https://developer.allizom.org/en-US/docs/Mozilla (worked)
3. https://developer.allizom.org/en-US/learn/html5 -> https://developer.allizom.org/en-US/learn/html5/ (FAILED, produced styled not found page)

Note that #3 worked when I reverted the mwsgi changes. 


* Section renames
(https://github.com/mozilla/kuma/blob/c3ea7928fac2a4a22aa4d20e1f44273828515033/configs/htaccess#L98-L104)

1. https://developer.allizom.org/En/JavaScript/Reference/Objects -> https://developer.allizom.org/en-US/JavaScript/Reference/Objects/ (FAILED (404), styled not found)
2. https://developer.allizom.org/En/JavaScript/Reference/Objects/Array -> https://developer.allizom.org/en-US/docs/JavaScript/Reference/Global_Objects/Array (worked)

#1 still failed after I reverted the mwsgi changes


* Legacy locales
(https://github.com/mozilla/kuma/blob/c3ea7928fac2a4a22aa4d20e1f44273828515033/configs/htaccess#L126-L129)
1. https://developer.allizom.org/en -> https://developer.allizom.org/en-US
2. https://developer.allizom.org/cn -> https://developer.allizom.org/en-US/cn (FAILED (404), styled not found)
3. https://developer.allizom.org/zh_cn -> https://developer.allizom.org/en-US/docs/zh_cn ( FAILED,  “This page was auto-generated because a user created a sub-page to this page.”  )
4. https://developer.allizom.org/zh_tw -> https://developer.allizom.org/en-US/zh_tw (FAILED, styled not found)

All failures still existed after I reverted the mwsgi changes.


I also did a little testing of the static samples/assets, but I’m not sure that I tested these correctly.  
  * In prod, https://developer.mozilla.org/patches -> https://developer.mozilla.org/en-US/data/www/patches/ (FAILED, styled not found)
    In stage, https://developer.allizom.org/patches -> https://developer.allizom.org/patches (unstyled 404)
  * In prod, https://developer.mozilla.org/samples/ -> https://developer.mozilla.org/samples/ (FAILED, unstyled forbidden)
    In stage, https://developer.allizom.org/samples/ -> https://developer.allizom.org/samples/ (FAILED, unstyled not found)

Failures in stage where there even after reverting the mwsgi changes.

Comment 4

[C. Liang [:cyliang]]

I did a fresh round of testing.  I've noted where the results were different with MDN mounted under the mwsgi than without.


https://developer.allizom.org/en-US/mobile  
   redirects to https://developer.allizom.org/en-US/docs/Mozilla/Mobile   

https://developer.allizom.org/en-US/mozilla
   redirects to https://developer.allizom.org/en-US/docs/Mozilla

https://developer.allizom.org/en-US/learn/html5
   does not redirect
   produces a styled 404
   [ if I revert my changes, it correctly redirects to https://developer.allizom.org/en-US/docs/Web/Guide/HTML/HTML5 ] 

https://developer.allizom.org/En/JavaScript/Reference/Objects
   does not redirect
   produces a styled 404
   [ if I revert my changes, it redirects to https://developer.allizom.org/en-US/JavaScript/Reference/Objects/, which produces a styled 404 ]

https://developer.allizom.org/En/JavaScript/Reference/Objects/Array
   redirects to https://developer.allizom.org/en-US/JavaScript/Reference/Objects/Array
   produces a styled 404

https://developer.allizom.org/en
   redirects to https://developer.allizom.org/en-US/

https://developer.allizom.org/cn
   redirects to https://developer.allizom.org/en-US/cn
   produces a styled 404 (in English)

https://developer.allizom.org/zh_cn
   redirects to https://developer.allizom.org/zh-CN/docs
   produces a styled 404 (in simplified Chinese)
   [ if I revert my changes, it redirects to https://developer.allizom.org/zh-CN/docs/Web ]

https://developer.allizom.org/zh_tw
   redirects to https://developer.allizom.org/en-US/zh_tw
   produces a styled 404 (in English)

https://developer.mozilla.org/patches
   redirects to https://developer.mozilla.org/en-US/data/www/patches/
   produces a styled 404

https://developer.mozilla.org/samples/  
   produces an unstyled forbidden error

Comment 5

[Jannis Leidel [:jezdez]]

Ok, I think that redirect problem stems from our homegrown, non-native URL resolver :(

We use the "SCRIPT_NAME" WSGI environ variable to prepend the URLs. When we remove that from the htaccess and therefor from the rewritten mod_wsgi URLs this would end up losing something I think.

Could you set the FORCE_SCRIPT_NAME setting in the local settings to an empty string to make Django not consider the script name at all?

  FORCE_SCRIPT_NAME = ''

is what'd we'd need to test this.

Once that's confirmed, the next step would be to not require that anymore in our code base.

Comment 6

[C. Liang [:cyliang]]

Unfortunately, adding FORCE_SCRIPT_NAME did not appear to fix the redirect issue.

We attempted to turn up the both the rewrite and wsgi logging in Apache.  Neither log produced any obvious indications of whether or not this is an issue in the url prefixer.  =\

Comment 7

[Shyam Mani [:fox2mike]]

Timeout, let's re-open this when there's someone on the MDN side to work on it.