793754 - [bedrock] /b/ redirect for /contribute/

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task, P3)

Description

[Anthony Ricaud (:rik)]

Please replace
    RewriteRule ^/en-US/contribute(/?)$ /b/en-US/contribute$1 [PT]
    RewriteRule ^/(\w{2,3}(?:-\w{2}(?:-mac)?)?/)?contribute(/?)$ /en-US/contribute/ [L,R=302]
    RewriteRule ^/en-US/contribute/event(/?)$ /b/en-US/contribute/event$1 [PT]
    RewriteRule ^/(\w{2,3}(?:-\w{2}(?:-mac)?)?/)?contribute/event(.*)$ /en-US/contribute/event$2 [L,R=302]


with
    RewriteRule ^/(\w{2,3}(?:-\w{2}(?:-mac)?)?/)?contribute(/?)$ /b/$1contribute/ [PT]
    RewriteRule ^/(\w{2,3}(?:-\w{2}(?:-mac)?)?/)?contribute/event(/?)$ /b/$1contribute/ [PT]

First on dev and stage.

Comment 1

[Brandon Burton [:solarce]]

This has been pushed to dev and stage

Comment 2

[Brandon Burton [:solarce]]

Is this ready to push?

Comment 3

[raymond [:retornam] (inactive)]

https://www.allizom.org/en-US/contribute/
https://www.allizom.org/en-US/contribute/event/ 


fixed on stage

Comment 4

[Anthony Ricaud (:rik)]

This regexp change will launch the contribute page in non en-US locales. So I'll defer to Pascal to know when it's ready to push.

Comment 5

[Pascal Chevrel:pascalc (PTO until April 26)]

The contribute page content changes too much currently to be able to say when we can push localized content, also, we won't push the contribute page until Bug 794059 is fixed.

Comment 6

[Brandon Burton [:solarce]]

Is this ready to be pushed?

Comment 7

[Pascal Chevrel:pascalc (PTO until April 26)]

yes please, now that Bug 794059 is fixed, the redirect can be put in place.

Comment 8

[Pascal Chevrel:pascalc (PTO until April 26)]

Can we get this redirect set up before the end of the week? I'd like to push a couple of locales before christmas, thanks!

Comment 9

[Brandon Burton [:solarce]]

I pushed :pascalc's needed fix to -dev and -stage

This is the current state of all the contribute related rewrites on -dev and -stage

    # bug 793754
    RewriteRule ^/(\w{2,3}(?:-\w{2}(?:-mac)?)?/)?contribute(/?)$ /b/$1contribute/ [PT]

    # still english only
    RewriteRule ^/en-US/contribute/page(/?)$ /b/en-US/contribute/page$1 [PT]
    RewriteRule ^/(\w{2,3}(?:-\w{2}(?:-mac)?)?/)?contribute/page(/?)$ /en-US/contribute/page/ [L,R=302]

    # bug 787953
    RewriteRule ^/en-US/contribute/event(/?)$ /b/en-US/contribute/event$1 [PT]
    RewriteRule ^/(\w{2,3}(?:-\w{2}(?:-mac)?)?/)?contribute/event(.*)$ /en-US/contribute/event$2 [L,R=302]

The rewrite works on -dev, not on -stage

https://www-dev.allizom.org/fr/contribute/ -> French content

https://www.allizom.org/fr/contribute/ -> https://www.allizom.org/en-US/contribute/

The locale checkouts from SVN appear to be the same


[[e-mail]1 locale]# svn info
Path: .
Working Copy Root Path: /data/bedrock-dev/src/www-dev.allizom.org-django/bedrock/locale
URL: http://svn.mozilla.org/projects/mozilla.com/trunk/locales
Repository Root: http://svn.mozilla.org
Repository UUID: 4eb1ac78-321c-0410-a911-ec516a8615a5
Revision: 111670
Node Kind: directory
Schedule: normal
Last Changed Author: [e-mail]
Last Changed Rev: 111663
Last Changed Date: 2012-12-18 08:06:16 -0800 (Tue, 18 Dec 2012)

[[e-mail]1 locale]# svn info /data/bedrock-stage/src/www.allizom.org-django/
bedrock/                commander_settings.py   commander_settings.pyc  update.py               update.pyc
[[e-mail]1 locale]# svn info /data/bedrock-stage/src/www.allizom.org-django/bedrock/locale/
Path: /data/bedrock-stage/src/www.allizom.org-django/bedrock/locale
Working Copy Root Path: /data/bedrock-stage/src/www.allizom.org-django/bedrock/locale
URL: http://svn.mozilla.org/projects/mozilla.com/trunk/locales
Repository Root: http://svn.mozilla.org
Repository UUID: 4eb1ac78-321c-0410-a911-ec516a8615a5
Revision: 111610
Node Kind: directory
Schedule: normal
Last Changed Author: [e-mail]
Last Changed Rev: 111569
Last Changed Date: 2012-12-13 07:29:07 -0800 (Thu, 13 Dec 2012)

Comment 10

[Brandon Burton [:solarce]]

Stage is currently on 92b1e85654f060b652d420642e1a63402fd28d11 for the Django code

Comment 11

[Brandon Burton [:solarce]]

Prod pushed

[[e-mail]1 air]# /data/genericrhel6/src/moztrap.mozilla.org/update 
Updating code...
Already up-to-date.
Updating vendor submodules...
syncing the dbs...
Syncing...
Creating tables ...
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)

Synced:
 > django.contrib.sessions
 > django.contrib.messages
 > django.contrib.staticfiles
 > django.contrib.contenttypes
 > django.contrib.auth
 > django.contrib.admin
 > django.contrib.humanize
 > south
 > registration
 > django_sha2
 > compressor
 > floppyforms
 > djangosecure
 > icanhaz
 > html5accordion
 > messages_ui
 > ajax_loading_overlay

Not synced (use migrations):
 - preferences
 - moztrap.model.core
 - moztrap.model.library
 - moztrap.model.environments
 - moztrap.model.execution
 - moztrap.model.attachments
 - moztrap.model.tags
(use ./manage.py migrate to migrate these)
Running migrations for preferences:
- Nothing to migrate.
 - Loading initial data for preferences.
Installed 0 object(s) from 0 fixture(s)
Running migrations for core:
- Nothing to migrate.
 - Loading initial data for core.
Installed 0 object(s) from 0 fixture(s)
Running migrations for library:
- Nothing to migrate.
 - Loading initial data for library.
Installed 0 object(s) from 0 fixture(s)
Running migrations for environments:
- Nothing to migrate.
 - Loading initial data for environments.
Installed 0 object(s) from 0 fixture(s)
Running migrations for execution:
- Nothing to migrate.
 - Loading initial data for execution.
Installed 0 object(s) from 0 fixture(s)
Running migrations for attachments:
- Nothing to migrate.
 - Loading initial data for attachments.
Installed 0 object(s) from 0 fixture(s)
Running migrations for tags:
- Nothing to migrate.
 - Loading initial data for tags.
Installed 0 object(s) from 0 fixture(s)
assets...

0 static files copied, 440 unmodified.
Invalid template at: /data/genericrhel6/src/moztrap.mozilla.org/moz-trap/requirements/vendor/lib/python/floppyforms/templates/floppyforms/gis/openlayers-sekizai.html
Found 'compress' tags in:
	/data/genericrhel6/src/moztrap.mozilla.org/moz-trap/templates/base.html
Compressing... done
Compressed 3 block(s) from 1 template(s).
Cleaning gitignore and pyc files...
setting revision
4642807f8bd47cb9294f63cc030f8d81c1ab999e
touching wsgi file
Deploying code...
done at Tue Dec 18 09:51:15 PST 2012
[2012-12-18 09:51:15] Running rsync_project
[2012-12-18 09:51:15] [localhost] running: /usr/bin/rsync -aq --include '.gitkeep' --exclude '.git*' --exclude '.hg*' --exclude '.svn*' --exclude 'CVS' --delete /data/genericrhel6/src/moztrap.mozilla.org/ /data/genericrhel6/www/moztrap.mozilla.org/
[2012-12-18 09:51:16] [localhost] finished: /usr/bin/rsync -aq --include '.gitkeep' --exclude '.git*' --exclude '.hg*' --exclude '.svn*' --exclude 'CVS' --delete /data/genericrhel6/src/moztrap.mozilla.org/ /data/genericrhel6/www/moztrap.mozilla.org/ (1.479s)
[2012-12-18 09:51:16] Finished rsync_project (1.480s)
[2012-12-18 09:51:16] Running commit_www
[2012-12-18 09:51:16] [localhost] running: cd /data/genericrhel6/www && /usr/bin/git add .; /usr/bin/git commit -a -m 'deploy ['moztrap.mozilla.org']'
[2012-12-18 09:51:47] [localhost] finished: cd /data/genericrhel6/www && /usr/bin/git add .; /usr/bin/git commit -a -m 'deploy ['moztrap.mozilla.org']' (30.743s)
[localhost] out: [master e516a95] deploy [moztrap.mozilla.org]
[localhost] out: 1 file changed, 4 insertions(+), 7 deletions(-)
[2012-12-18 09:51:47] Finished commit_www (30.743s)
[2012-12-18 09:51:47] Running push_www
[2012-12-18 09:51:47] [generic1.webapp.phx1.mozilla.com] running: /data/bin/update-www.sh moztrap.mozilla.org
[2012-12-18 09:51:47] [generic2.webapp.phx1.mozilla.com] running: /data/bin/update-www.sh moztrap.mozilla.org
[2012-12-18 09:51:47] [generic3.webapp.phx1.mozilla.com] running: /data/bin/update-www.sh moztrap.mozilla.org
[2012-12-18 09:51:47] [generic4.webapp.phx1.mozilla.com] running: /data/bin/update-www.sh moztrap.mozilla.org
[2012-12-18 09:51:47] [generic5.webapp.phx1.mozilla.com] running: /data/bin/update-www.sh moztrap.mozilla.org
[2012-12-18 09:51:47] [generic102.webapp.phx1.mozilla.com] running: /data/bin/update-www.sh moztrap.mozilla.org
[2012-12-18 09:51:53] [generic3.webapp.phx1.mozilla.com] finished: /data/bin/update-www.sh moztrap.mozilla.org (6.151s)
[generic3.webapp.phx1.mozilla.com] out: Not removing blog.mozilla.org/wp-content/cache/supercache/blog.mozilla.org/
[2012-12-18 09:51:53] [generic5.webapp.phx1.mozilla.com] finished: /data/bin/update-www.sh moztrap.mozilla.org (6.480s)
[2012-12-18 09:51:54] [generic4.webapp.phx1.mozilla.com] finished: /data/bin/update-www.sh moztrap.mozilla.org (6.543s)
[2012-12-18 09:51:54] [generic2.webapp.phx1.mozilla.com] finished: /data/bin/update-www.sh moztrap.mozilla.org (6.626s)
[generic2.webapp.phx1.mozilla.com] out: Not removing blog.mozilla.org/wp-content/cache/supercache/blog.mozilla.org/
[2012-12-18 09:51:54] [generic1.webapp.phx1.mozilla.com] finished: /data/bin/update-www.sh moztrap.mozilla.org (6.861s)
[generic1.webapp.phx1.mozilla.com] out: Not removing blog.mozilla.org/wp-content/cache/supercache/blog.mozilla.org/
[2012-12-18 09:51:54] [generic102.webapp.phx1.mozilla.com] finished: /data/bin/update-www.sh moztrap.mozilla.org (7.282s)
[generic102.webapp.phx1.mozilla.com] out: Not removing blog.mozilla.org/wp-content/cache/supercache/blog.mozilla.org/
[generic102.webapp.phx1.mozilla.com] out: Not removing mozillians.org/mozillians/apps/larper/
[generic102.webapp.phx1.mozilla.com] out: Not removing mozillians.org/mozillians/apps/taskboard/
[generic102.webapp.phx1.mozilla.com] out: Not removing wiki.mozilla.org/wiki/cache/8/
[2012-12-18 09:51:54] Finished push_www (7.291s)
[2012-12-18 09:51:54] [generic1.webapp.phx1.mozilla.com] running: touch /data/www/moztrap.mozilla.org/moz-trap/moztrap/deploy/vendor_wsgi.py
[2012-12-18 09:51:54] [generic2.webapp.phx1.mozilla.com] running: touch /data/www/moztrap.mozilla.org/moz-trap/moztrap/deploy/vendor_wsgi.py
[2012-12-18 09:51:54] [generic3.webapp.phx1.mozilla.com] running: touch /data/www/moztrap.mozilla.org/moz-trap/moztrap/deploy/vendor_wsgi.py
[2012-12-18 09:51:54] [generic4.webapp.phx1.mozilla.com] running: touch /data/www/moztrap.mozilla.org/moz-trap/moztrap/deploy/vendor_wsgi.py
[2012-12-18 09:51:54] [generic5.webapp.phx1.mozilla.com] running: touch /data/www/moztrap.mozilla.org/moz-trap/moztrap/deploy/vendor_wsgi.py
[2012-12-18 09:51:54] [generic102.webapp.phx1.mozilla.com] running: touch /data/www/moztrap.mozilla.org/moz-trap/moztrap/deploy/vendor_wsgi.py
[2012-12-18 09:51:55] [generic1.webapp.phx1.mozilla.com] finished: touch /data/www/moztrap.mozilla.org/moz-trap/moztrap/deploy/vendor_wsgi.py (0.218s)
[2012-12-18 09:51:55] [generic2.webapp.phx1.mozilla.com] finished: touch /data/www/moztrap.mozilla.org/moz-trap/moztrap/deploy/vendor_wsgi.py (0.429s)
[2012-12-18 09:51:55] [generic3.webapp.phx1.mozilla.com] finished: touch /data/www/moztrap.mozilla.org/moz-trap/moztrap/deploy/vendor_wsgi.py (0.430s)
[2012-12-18 09:51:55] [generic4.webapp.phx1.mozilla.com] finished: touch /data/www/moztrap.mozilla.org/moz-trap/moztrap/deploy/vendor_wsgi.py (0.435s)
[2012-12-18 09:51:55] [generic102.webapp.phx1.mozilla.com] finished: touch /data/www/moztrap.mozilla.org/moz-trap/moztrap/deploy/vendor_wsgi.py (0.449s)
[2012-12-18 09:51:55] [generic5.webapp.phx1.mozilla.com] finished: touch /data/www/moztrap.mozilla.org/moz-trap/moztrap/deploy/vendor_wsgi.py (0.520s)

Comment 12

[Brandon Burton [:solarce]]

Whoops, wrong bug

Comment 13

[Brandon Burton [:solarce]]

:jakem is looking into this further

Comment 14

[Jake Maul [:jakem]]

@pascalc: We are theorizing that this might have something to do with the PHP app... the .htaccess rules and such in there are relevant, and different from dev. Dev is on trunk, stage is on tags/stage... there are a good bit of differences between the .htaccess files, at least... maybe other stuff too?


[[e-mail]1 data]# svn info bedrock-dev/src/www-dev.allizom.org
URL: http://svn.mozilla.org/projects/mozilla.com/trunk
Repository UUID: 4eb1ac78-321c-0410-a911-ec516a8615a5
Revision: 111674
Last Changed Author: [e-mail]
Last Changed Rev: 111668
Last Changed Date: 2012-12-18 08:44:01 -0800 (Tue, 18 Dec 2012)

[[e-mail]1 data]# svn info bedrock-stage/src/www.allizom.org
URL: http://svn.mozilla.org/projects/mozilla.com/tags/stage
Repository UUID: 4eb1ac78-321c-0410-a911-ec516a8615a5
Revision: 111674
Last Changed Author: [e-mail]
Last Changed Rev: 111673
Last Changed Date: 2012-12-18 09:04:58 -0800 (Tue, 18 Dec 2012)


We're not sure this is the culprit, but it's the best lead we've got at the moment, and it's something you can look into while we look at the server-side stuff.

Comment 15

[Pascal Chevrel:pascalc (PTO until April 26)]

Attachment: htaccess diff between trunk and tags/stage

@jake that page/url did not exist on the PHP side, only on the Django side. I made a diff of the htaccess there for dev and stage and I don't see any rule difference that would catch request for the contribute page. I am attaching the htaccess diff for reference in case I missed something obvious in it.

Comment 16

[Pascal Chevrel:pascalc (PTO until April 26)]

I found out that contribute.lang (file containing the translated strings for the template) had a BOM, since the first line is where we have the tag to activate a page for a locale, maybe the parser doesn't like it, I removed the BOM. Is stage svn auto-updated or is it updated only when there is a code push? Could you check that it is at r111683 which has the updated file without the BOM? Thanks

Comment 17

[Daniel Maher [:phrawzty]]

(In reply to Pascal Chevrel:pascalc from comment #16)
> Could you check that it is at r111683 which has the updated file
> without the BOM? Thanks

[[e-mail]1 data]$ svn info bedrock-stage/src/www.allizom.org
Path: bedrock-stage/src/www.allizom.org
Working Copy Root Path: /data/bedrock-stage/src/www.allizom.org
URL: http://svn.mozilla.org/projects/mozilla.com/tags/stage
Repository Root: http://svn.mozilla.org
Repository UUID: 4eb1ac78-321c-0410-a911-ec516a8615a5
Revision: 111683
Node Kind: directory
Schedule: normal
Last Changed Author: [e-mail]
Last Changed Rev: 111683
Last Changed Date: 2012-12-19 01:04:38 -0800 (Wed, 19 Dec 2012)

Comment 18

[Pascal Chevrel:pascalc (PTO until April 26)]

Here is what I have investigated so far:

On the PHP side:
- no redirect for contribute .htaccess
- no redirect in prefetch.php

On the Python site:
- no redirect for contribute in apps/redirects/urls.py
- I grepped the whole repo for 'contribute' and didn't find code related to that url that would relate to a redirect
- The template path and the lang file path do match and there is no code calling a non default lang file (apps/mozorg/templates/mozorg/contribute.html and locale/fr/mozorg/contribute.lang)

On the localized files side:
- the page for the locale is activated (## active ## as first line)
- the contribute.lang file had a BOM that I removed and merged to stage in r111683, my local testing with DEV = False indicates that it was the cause for the redirect, if I remove the BOM the first line is correctly parsed by Bedrock.

That said, I am still seeing the redirect on allizom.org:

curl -I https://www.allizom.org/fr/contribute/
HTTP/1.1 302 FOUND
Server: Apache
X-Backend-Server: bedrock1.stage.webapp.phx1.mozilla.com
Vary: Accept-Encoding
Cache-Control: max-age=600
Content-Type: text/html; charset=utf-8
Date: Wed, 19 Dec 2012 10:44:43 GMT
Location: https://www.allizom.org/en-US/contribute/
Expires: Wed, 19 Dec 2012 10:54:43 GMT
Transfer-Encoding: chunked
Connection: Keep-Alive
x-frame-options: DENY
X-Cache-Info: caching


Apache caching of the redirect maybe ?

Comment 19

[Daniel Maher [:phrawzty]]

05:13:08 <@pascalc> Phrawzty, tu as fait un svn info sur :
05:13:09 <@pascalc> /data/bedrock-stage/src/www.allizom.org
05:13:11 <@pascalc> Mais en fait, je pense que ça devrait être sur :
05:13:13 <@pascalc> /data/bedrock-stage/src/www.allizom.org-django/bedrock/locale
05:13:14 <@pascalc> Tu pourrais refaire le svn info stp ? et si c'est pas à la dernière révision faire un svn up ?
05:14:19 <@pascalc> (le premier dépôt est celui des pages allizom.org encore en php)


[e-mail]1 ~]$ svn info /data/bedrock-stage/src/www.allizom.org
Path: /data/bedrock-stage/src/www.allizom.org
Working Copy Root Path: /data/bedrock-stage/src/www.allizom.org
URL: http://svn.mozilla.org/projects/mozilla.com/tags/stage
Repository Root: http://svn.mozilla.org
Repository UUID: 4eb1ac78-321c-0410-a911-ec516a8615a5
Revision: 111687
Node Kind: directory
Schedule: normal
Last Changed Author: [e-mail]
Last Changed Rev: 111686
Last Changed Date: 2012-12-19 03:17:46 -0800 (Wed, 19 Dec 2012)


[[e-mail]1 ~]$ svn info /data/bedrock-stage/src/www.allizom.org-django/bedrock/locale
Path: /data/bedrock-stage/src/www.allizom.org-django/bedrock/locale
Working Copy Root Path: /data/bedrock-stage/src/www.allizom.org-django/bedrock/locale
URL: http://svn.mozilla.org/projects/mozilla.com/trunk/locales
Repository Root: http://svn.mozilla.org
Repository UUID: 4eb1ac78-321c-0410-a911-ec516a8615a5
Revision: 111673
Node Kind: directory
Schedule: normal
Last Changed Author: [e-mail]
Last Changed Rev: 111663
Last Changed Date: 2012-12-18 08:06:16 -0800 (Tue, 18 Dec 2012)


[[e-mail]1 ~]$ cd /data/bedrock-stage/src/www.allizom.org
[[e-mail]1 www.allizom.org]$ sudo svn up
Updating '.':
Fetching external item into 'includes/product-details':
External at revision 111582.
Fetching external item into 'includes/certs':
External at revision 111688.
Fetching external item into 'en-US/firefox/webhero/fb/locale':
External at revision 111688.
Fetching external item into 'img/screenshots':
External at revision 111688.
Fetching external item into 'tabzilla':
External at revision 211.
At revision 111688.


[e-mail]1 www.allizom.org]$ cd /data/bedrock-stage/src/www.allizom.org-django/bedrock/locale
[[e-mail]1 locale]$ sudo svn up
Updating '.':
U    fr/mozorg/contribute.lang
U    nl/mozorg/contribute.lang
Updated to revision 111688.

Comment 20

[Pascal Chevrel:pascalc (PTO until April 26)]

ok, we can see from Daniel's last post that allizom.org does not autoupdate lang files used by django, I will file a separate bug about it. The files are now using the latest version with the active tag and no BOM, but the page are still redirected.

Can we get a diff of the .htaccess used in dev and on stage by Django and an apache restart on stage to make sure that the updated htaccess is taken into account? Also, maybe there are redirect rules on stage affecting /contribute/ in the Apache config files and not in htaccess?

Comment 21

[Pascal Chevrel:pascalc (PTO until April 26)]

ok, this is now working:
https://www.allizom.org/fr/contribute/

Comment 22

[Pascal Chevrel:pascalc (PTO until April 26)]

Please put that redirect in production :)

Comment 23

[Daniel Maher [:phrawzty]]

(In reply to Pascal Chevrel:pascalc from comment #22)
> Please put that redirect in production :)

07:17:44 < phrawzty> pascalc: alors tu veux que je deploy en prod ?
07:18:02 <@pascalc> phrawzty oui

Translation: Please push production. :)

Comment 24

[Daniel Maher [:phrawzty]]

This rewrite has been applied in production :

    # bug 793754
    RewriteRule ^/(\w{2,3}(?:-\w{2}(?:-mac)?)?/)?contribute(/?)$ /b/$1contribute/ [PT]

It has been pushed out, and :pascalc appears to be happy with the results (verified on IRC).  Closing bug. :)

Comment 25

[Pascal Chevrel:pascalc (PTO until April 26)]

thanks :)