589715 - [AMO] Update addons.mozilla.org on 8/26 @ 1600

Status: VERIFIED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Wil Clouser [:clouserw]]

Steps to follow soon

Comment 1

[Justin Scott [:fligtar]]

26th, right? not 4 hours ago.

Comment 2

[Jeremy Orem [:oremj]]

Confirm the 26th?

Comment 3

[Wil Clouser [:clouserw]]

confirmed! thanks

Comment 4

[Wil Clouser [:clouserw]]

I was just talking to oremj about this.  I was thinking we could get away without downtime, but the database upgrades took me about 10 minutes (http://pastebin.mozilla.org/776404) so we should probably pull it down while that happens.

Comment 5

[Wil Clouser [:clouserw]]

Steps are here: http://etherpad.mozilla.org:9000/5-11-8-push

Comment 6

[Wil Clouser [:clouserw]]

0) Update settings_local.py:

- comment out the HERA line (we'll add it in below).  This is to avoid downtime.  If you can push the config and all file updates at the same time, just do step 7 now.

- add this below DOMAIN:   SERVICES_URL = 'https://services.%s' % DOMAIN

- Add these below UPLOADS_PATH:
  ADDON_ICONS_PATH = UPLOADS_PATH + '/addon_icons'
  COLLECTIONS_ICON_PATH = UPLOADS_PATH + '/collection_icons'
  PREVIEWS_PATH = UPLOADS_PATH + '/previews'

1) git fetch && git co -f origin/5.11.8

1.5) git config --remove-section submodule.locale && rm -rf locale

2) git submodule update --init

3) cd vendor; git co -f origin/5.11.8; git submodule update --init; cd ..

4) whack any .pyc files

5) ./manage.py compress_assets

6) schematic migrations

7) settings.HERA changed it's format (it's now a list of dictionaries -   see preview's config for an example).  This should be changed in   production and it should also be expanded to have credentials for all   the zeus boxes.

8) Push the new site to the webhead (AMO + SAMO)

9) Clear memcache

10) Restart celeryd

11) Wait a while for db/memcache to calm down

11.5) Go get a drink

12) Add these to apache's .conf:
    RewriteRule ^((?:(?:/\w{2,3}(?:-\w{2})?)(?:/(?:firefox|seamonkey|thunderbird|mobile|sunbird))?)?/tag/.*)$ /z$1 [PT]
    RewriteRule ^((?:(?:/\w{2,3}(?:-\w{2})?)(?:/(?:firefox|seamonkey|thunderbird|mobile|sunbird))?)?/tags/top)$ /z$1 [PT]
    RewriteRule
^((?:(?:/\w{2,3}(?:-\w{2})?)(?:/(?:firefox|seamonkey|thunderbird|mobile|sunbird))?)?/featured.*)$
/z$1 [PT]
RewriteRule ^((?:(?:/\w{2,3}(?:-\w{2})?)(?:/(?:firefox|seamonkey|thunderbird|mobile|sunbird))?)?/pages/appversions(/format:rss)?)$ /z$1 [PT]

13) import scripts/crontab/prod to crontab

14) Dump the `users` table to a file somewhere and email it to fligtar

15) Grab http://people.mozilla.com/~clouserw/temp/delete_these_users (that's 28M) and then run this job.  It will fill up rabbitmq.  Let's hope it doesn't crash:
    ./manage.py delete_users users_to_delete
    
16) Bring back redis (not critical): CACHE_MACHINE_NO_INVALIDATION = False

Comment 7

[Jeff Balogh (:jbalogh)]

(In reply to comment #6)
> ^((?:(?:/\w{2,3}(?:-\w{2})?)(?:/(?:firefox|seamonkey|thunderbird|mobile|sunbird))?)?/pages/appversions(/format:rss)?)$
> /z$1 [PT]

^((?:(?:/\w{2,3}(?:-\w{2})?)(?:/(?:firefox|seamonkey|thunderbird|mobile|sunbird))?)?/pages/appversions.*)$
/z$1 [PT]

Comment 8

[Jeremy Orem [:oremj]]

[root@mradm02 ~]# ./bin/zamboni_quick_update.sh  /data/amo_python/src/prod/zamboni 5.11.8 -v
+ git fetch -t origin
remote: Counting objects: 1963, done.
remote: Compressing objects: 100% (482/482), done.
remote: Total 1697 (delta 1257), reused 1509 (delta 1108)
Receiving objects: 100% (1697/1697), 321.05 KiB, done.
Resolving deltas: 100% (1257/1257), completed with 168 local objects.
From git://github.com/jbalogh/zamboni
 * [new tag]         5.11.8     -> 5.11.8
+ git checkout 5.11.8
M	media/img/amo2009
Previous HEAD position was dfcaa7e... no caching for fast_current_version
HEAD is now at 8b4bdd1... don't touch the modified date in collection migrations
+ git submodule update --init
Submodule 'locale' (git://github.com/jbalogh/amo-locales2) registered for path 'locale'
Cloning into locale...
remote: Counting objects: 877, done.
remote: Compressing objects: 100% (659/659), done.
remote: Total 877 (delta 182), reused 877 (delta 182)
Receiving objects: 100% (877/877), 11.94 MiB | 5.93 MiB/s, done.
Resolving deltas: 100% (182/182), done.
Submodule path 'locale': checked out '21521a822feac72bc6f3482173e5e7885549d7e6'
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 4 (delta 3), reused 0 (delta 0)
Unpacking objects: 100% (4/4), done.
From git://github.com/jbalogh/amo2009-img
   d8326ee..db07aea  master     -> origin/master
Submodule path 'media/img/amo2009': checked out 'db07aea0d7ad2abec38e0c0be868c16f0a8b8df6'
+ set +x
+ pushd vendor
+ git fetch -t origin
remote: Counting objects: 87, done.
remote: Compressing objects: 100% (76/76), done.
remote: Total 78 (delta 40), reused 0 (delta 0)
Unpacking objects: 100% (78/78), done.
From git://github.com/jbalogh/zamboni-lib
 * [new tag]         5.11.7.3   -> 5.11.7.3
 * [new tag]         5.11.8     -> 5.11.8
+ git checkout 5.11.8
M	src/bleach
M	src/commonware
M	src/django
M	src/django-cache-machine
M	src/django-multidb-router
M	src/django-queryset-transform
M	src/hera
M	src/jingo
M	src/tower
Previous HEAD position was c07bc3b... a switch to turn off invalidation
HEAD is now at 64d96a0... updating multidb-router
+ git submodule update --init
Submodule 'src/django-recaptcha' (git://github.com/mozilla/django-recaptcha.git) registered for path 'src/django-recaptcha'
Submodule 'src/happyforms' (git://github.com/mozilla/happyforms.git) registered for path 'src/happyforms'
Submodule 'src/redis' (git://github.com/jbalogh/redis-py) registered for path 'src/redis'
remote: Counting objects: 15, done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 9 (delta 6), reused 0 (delta 0)
Unpacking objects: 100% (9/9), done.
From git://github.com/jsocol/bleach
   e6225c2..d4bdcba  master     -> origin/master
Submodule path 'src/bleach': checked out 'd4bdcba5565d0b6071ad75fb5008bb835a0ff936'
remote: Counting objects: 53, done.
remote: Compressing objects: 100% (42/42), done.
remote: Total 42 (delta 29), reused 0 (delta 0)
Unpacking objects: 100% (42/42), done.
From git://github.com/jbalogh/django-cache-machine
   44a3b07..d03fa49  master     -> origin/master
   89ce18e..1eb9f2f  redis      -> origin/redis
Submodule path 'src/django-cache-machine': checked out '1eb9f2f7383bbfd56775a0219465e485847869d8'
remote: Counting objects: 28, done.
remote: Compressing objects: 100% (21/21), done.
remote: Total 23 (delta 8), reused 0 (delta 0)
Unpacking objects: 100% (23/23), done.
From git://github.com/jbalogh/django-multidb-router
   8eeef71..25ca0c6  master     -> origin/master
Submodule path 'src/django-multidb-router': checked out '25ca0c68929144ed9020d89f2ea1e941c7f9aeca'
Cloning into src/django-recaptcha...
remote: Counting objects: 69, done.
remote: Compressing objects: 100% (40/40), done.
remote: Total 69 (delta 33), reused 58 (delta 27)
Receiving objects: 100% (69/69), 10.62 KiB, done.
Resolving deltas: 100% (33/33), done.
Submodule path 'src/django-recaptcha': checked out 'bbe6a921e95e4961b028b7eb5ea197d3fb54aa9a'
Cloning into src/happyforms...
remote: Counting objects: 16, done.
remote: Compressing objects: 100% (11/11), done.
remote: Total 16 (delta 3), reused 0 (delta 0)
Receiving objects: 100% (16/16), done.
Resolving deltas: 100% (3/3), done.
Submodule path 'src/happyforms': checked out '729612c2a824a7e8283d416d2084bf506c671e24'
remote: Counting objects: 14, done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 9 (delta 5), reused 0 (delta 0)
Unpacking objects: 100% (9/9), done.
From git://github.com/clouserw/hera
   8c6a1db..1706c6e  master     -> origin/master
Submodule path 'src/hera': checked out '1706c6e8d58c805ccba42057f408b5820e079313'
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 4 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (4/4), done.
From git://github.com/jbalogh/jingo
   944a8c1..9c8fdfc  master     -> origin/master
Submodule path 'src/jingo': checked out '9c8fdfc63765db0e8b176444d9160499f58ebcea'
Cloning into src/redis...
remote: Counting objects: 526, done.
remote: Compressing objects: 100% (218/218), done.
remote: Total 526 (delta 312), reused 513 (delta 304)
Receiving objects: 100% (526/526), 92.81 KiB, done.
Resolving deltas: 100% (312/312), done.
Submodule path 'src/redis': checked out 'a655718c3fe5b6e01714b584268d4fa8d8cf5cc6'
+ popd
+ set +x
Compess assets? [y/n]: y

Comment 9

[Jeremy Orem [:oremj]]

Run schematic? [y/n]: y
Running migration 62:
CREATE INDEX type_idx ON collections (collection_type);
CREATE INDEX watcher_idx ON collections (weekly_subscribers);
CREATE INDEX rating_idx ON collections (rating);
CREATE INDEX created_idx ON collections (created);

That took 24.03 seconds
################################################## 

Running migration 63:
ALTER TABLE collections ADD UNIQUE (`author_id`, `slug`);

That took 6.91 seconds
################################################## 

Running migration 64:
-- This needs to be added on your dev copies, but justdave already ran it on
-- preview and production so I'm commenting it out here.

-- ALTER TABLE addonlogs ADD KEY `user_id` (`user_id`);

That took 0.01 seconds
################################################## 

Running migration 65:
DROP TABLE collections_categories;

That took 0.10 seconds
################################################## 

Running migration 66:
RENAME TABLE `users_blacklistednickname` TO `users_blacklistedusername`;
ALTER TABLE `users_blacklistedusername` CHANGE `nickname` `username` varchar(255) NOT NULL default '';
ALTER TABLE `users_blacklistedusername` DROP KEY `nickname`;
ALTER TABLE `users_blacklistedusername` ADD UNIQUE(`username`);

-- Will take 1-3 minutes
ALTER TABLE `users`
    ADD COLUMN `username` varchar(255) UNIQUE default NULL after `email`,
    ADD COLUMN `display_name` varchar(255) default NULL after `username`;

-- This needs to be run after the convert_user_fields management command.  Since
-- that's going in at the same time as this but has to be run manually, I'm
-- leaving this commented out.
-- Query OK, 859523 rows affected (2 min 24.52 sec)

-- ALTER TABLE users CHANGE COLUMN `username` `username` varchar(255) NOT NULL;


That took 366.66 seconds
##################################################

Comment 10

[Jeremy Orem [:oremj]]

-    RewriteRule ^((?:(?:/\w{2,3}(?:-\w{2})?)(?:/(?:firefox|seamonkey|thunderbird|mobile|sunbird))?)?/featured/?)$ /z$1 [PT]
     RewriteRule ^((?:(?:/\w{2,3}(?:-\w{2})?)(?:/(?:firefox|seamonkey|thunderbird|mobile|sunbird))?)?/recommended/?)$ /z$1 [PT]
 
+    #bug 589715
+    RewriteRule ^((?:(?:/\w{2,3}(?:-\w{2})?)(?:/(?:firefox|seamonkey|thunderbird|mobile|sunbird))?)?/tag/.*)$ /z$1 [PT]
+    RewriteRule ^((?:(?:/\w{2,3}(?:-\w{2})?)(?:/(?:firefox|seamonkey|thunderbird|mobile|sunbird))?)?/tags/top)$ /z$1 [PT]
+    RewriteRule ^((?:(?:/\w{2,3}(?:-\w{2})?)(?:/(?:firefox|seamonkey|thunderbird|mobile|sunbird))?)?/featured.*)$ /z$1 [PT]
+    RewriteRule ^((?:(?:/\w{2,3}(?:-\w{2})?)(?:/(?:firefox|seamonkey|thunderbird|mobile|sunbird))?)?/pages/appversions.*)$ /z$1 [PT]
+
+
+

Comment 11

[Jeremy Orem [:oremj]]

User stuff kicked off. I think this bug is complete.

Comment 12

[Stephen Donner [:stephend] Not actively reading bugmail]

Calling this VERIFIED; http://github.com/jbalogh/zamboni/commits/5.11.8.4 took care of things :-)