523428 - [AMO] Update addons.mozilla.org on 10/20

Status: VERIFIED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Wil Clouser [:clouserw]]

This is a production AMO push for 5.2.  Steps are below, please paste the
output of all commands into this bug.  I expect there to be downtime so please
use a maintenance page.  Steps to follow:

1) There are 4 new defines in config.php.default starting with PERSONAS_.  They need to move to config.php and have values for the production personas database.  Please ensure the credentials you use have read access to the database and there are no other problems (eg. firewall) accessing the database from the AMO boxes.

2) Run the attached 5.2 SQL.  There is a query in here that might fail, so we need the output.  If it does fail, we'll write some SQL to fix it. (I was waiting to attach it, but it's too slow and I want to file this bug, so I'll attach it when it's done)

3) Sphinx - Do what was done on preview: disable the prime_sphinx_index.py script, its no longer needed.  Use the new sphinx.conf that requires a localsettings.py, see localsettings.py-dist for an example.  Run the indexer --all --rotate manually to verify things work and time it.  If it runs in under a few minutes change the cron from 1  hour to 5 minutes.

4) `svn up` addons.mozilla.org.  There are a couple local changes which need to be reverted.

5) Update services.addons.mozilla.org with `svn up`

6) Restart apache to pick up gettext changes

7) Clear all caches (Zeus, memcache, local cake cache) on both AMO and SAMO.

8) There are a few new cron jobs as well as changes to a lot of existing ones.  Please make the production crontab look like preview's.  A bunch of jobs switched from maintenance.php to maintenance.py.

I still need to tag the site so this bug is serving as advanced notice for now.

Comment 1

[Wil Clouser [:clouserw]]

Attachment: 5.2.sql

Comment 2

[Jeremy Orem [:oremj]]

Attachment: First set of queries.

The next set after the failed query is coming soon.

Comment 3

[chizu]

New time for sphinx indexing:
real	2m27.385s
user	0m14.050s
sys	0m0.990s

I lowered it to 15 minutes for now.

Comment 4

[Dave Dash [:davedash, :dd] (assign all bugs to mbrandt)]

awesome!

Comment 5

[Jeremy Orem [:oremj]]

steps 1, 3, 4, 5, 6, 7, 8 done.

Comment 6

[Jeremy Orem [:oremj]]

Attachment: Second set of queries.

Also:
mysql> delete from test_results where test_case_id=122;
Query OK, 4270 rows affected (1.09 sec)

mysql> delete from test_cases where id=122;
Query OK, 1 row affected (0.04 sec)

Comment 7

[Jeremy Orem [:oremj]]

I think that's everything.

Comment 8

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified FIXED; we ran Selenium, verified top fixes on production, and https://addons.mozilla.org/en-US/firefox/personas/ looks great now (we were waiting on script crons to finish running).