Closed Bug 599922 Opened 15 years ago Closed 15 years ago

Update addons.mozilla.org on 9/30 @ 1600

Categories

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Product:
Infrastructure & Operations Graveyard
Component:
WebOps: Other
Platform:
All
Other
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: clouserw, Assigned: oremj)

References

Details

(Whiteboard: 09/30/2010 @ 4pm)

Please update AMO and SAMO. Details to follow, no downtime expected on our end, but I think you wanted to flip a switch on zeus or something.
Depends on: 560659
Flags: needs-downtime+
Whiteboard: 09/30/2010 @ 4pm
Assignee: server-ops → jeremy.orem+bugs
Not user impacting right? Can we do the AMO DB master change at the same time without user impact (read-only site)?
No user impact expected. We should do the read only site first, so we can clear memcache after the upgrade without worry about stressing the slaves more. I also have some non-critical scripts that need to write to the db.
From http://etherpad.mozilla.org:9000/5-12-push: read_only_mode(globals()) "Do AMO DB master change" (599922#2) Undo read-only mode Update to 5.12 Update submodules Update vendor ./manage.py compress_assets ./schematic migrations cat migrations/7[56]* | mysql zamboni # migrations 75 and 76 were skipped in prod through a cherry-pick, pick them up Add MONGO_* variables to settings_local Make sure FILES_URL is using SITE_URL for both AMO and SAMO Push the site out Restart celeryd ./manage.py make_django_users <= makes a bunch of tasks, we don't need to wait for it ./manage.py fix_user_images_again Clear memcache Import new crontab
[root@mradm02 ~]# ./bin/zamboni_quick_update.sh 5.12 5.12 + git fetch -t origin remote: Counting objects: 1522, done. remote: Compressing objects: 100% (543/543), done. remote: Total 1265 (delta 815), reused 1072 (delta 664) Receiving objects: 100% (1265/1265), 472.97 KiB, done. Resolving deltas: 100% (815/815), completed with 127 local objects. From git://github.com/jbalogh/zamboni * [new tag] 5.12 -> 5.12 + git checkout 5.12 M lib/product_details/json M locale Previous HEAD position was bc80c3f... use https for the gd fonts HEAD is now at 37170e5... update locale (again!) + git submodule update --init remote: Counting objects: 223, done. remote: Compressing objects: 100% (117/117), done. remote: Total 216 (delta 194), reused 116 (delta 99) Receiving objects: 100% (216/216), 34.52 KiB, done. Resolving deltas: 100% (194/194), completed with 6 local objects. From git://github.com/jbalogh/mozilla-product-details 3e86dcc..d52db60 master -> origin/master Submodule path 'lib/product_details/json': checked out 'd52db607d3cd7317022a4bec2323829f6298691e' remote: Counting objects: 923, done. remote: Compressing objects: 100% (336/336), done. remote: Total 842 (delta 596), reused 705 (delta 491) Receiving objects: 100% (842/842), 16.59 MiB | 3.45 MiB/s, done. Resolving deltas: 100% (596/596), completed with 30 local objects. From git://github.com/jbalogh/amo-locales2 c4a89fe..b98e006 master -> origin/master Submodule path 'locale': checked out 'b98e006a2599044681da25905e2a894f2f9e29eb' + set +x + pushd vendor + git fetch -t origin remote: Counting objects: 979, done. remote: Compressing objects: 100% (774/774), done. remote: Total 932 (delta 187), reused 761 (delta 140) Receiving objects: 100% (932/932), 2.19 MiB | 1.36 MiB/s, done. Resolving deltas: 100% (187/187), completed with 29 local objects. From git://github.com/jbalogh/zamboni-lib * [new tag] 5.12 -> 5.12 + git checkout 5.12 M src/check M src/django-cache-machine M src/django-devserver M src/django-extensions M src/django-nose M src/django-piston M src/easy-thumbnails M src/jingo-minify M src/test-utils Previous HEAD position was fdc0fe1... update commonware HEAD is now at d13284f... add django-csp to zamboni.pth + git submodule update --init Submodule 'src/amo-validator' (git://github.com/mattbasta/amo-validator.git) registered for path 'src/amo-validator' Submodule 'src/django-csp' (git://github.com/mozilla/django-csp.git) registered for path 'src/django-csp' Submodule 'src/nuggets' (git://github.com/mozilla/nuggets.git) registered for path 'src/nuggets' Submodule 'src/pymongo' (git://github.com/mongodb/mongo-python-driver.git) registered for path 'src/pymongo' Cloning into src/amo-validator... remote: Counting objects: 1158, done. remote: Compressing objects: 100% (586/586), done. remote: Total 1158 (delta 798), reused 812 (delta 552) Receiving objects: 100% (1158/1158), 2.22 MiB | 1008 KiB/s, done. Resolving deltas: 100% (798/798), done. Submodule path 'src/amo-validator': checked out 'bbd35bdc177227496f8f4baca9f5ed232eafe1f9' remote: Counting objects: 5, done. remote: Compressing objects: 100% (3/3), done. remote: Total 3 (delta 2), reused 0 (delta 0) Unpacking objects: 100% (3/3), done. From git://github.com/jbalogh/check 1c73813..03b26bc master -> origin/master Submodule path 'src/check': checked out '03b26bc3f12c1a43ab1156d50225d955667a642b' remote: Counting objects: 7, done. remote: Compressing objects: 100% (4/4), done. remote: Total 4 (delta 3), reused 0 (delta 0) Unpacking objects: 100% (4/4), done. From git://github.com/jbalogh/django-cache-machine + 782e4c9...de9ed74 redis -> origin/redis (forced update) error: You have local changes to 'caching/invalidation.py'; cannot switch branches. Unable to checkout 'de9ed743862ff1e02c80cfd9b47857ce98912263' in submodule path 'src/django-cache-machine' + popd + set +x + git fetch -t origin remote: Counting objects: 1480, done. remote: Compressing objects: 100% (545/545), done. remote: Total 1224 (delta 782), reused 1025 (delta 624) Receiving objects: 100% (1224/1224), 466.98 KiB, done. Resolving deltas: 100% (782/782), completed with 121 local objects. From git://github.com/jbalogh/zamboni * [new tag] 5.12 -> 5.12 + git checkout 5.12 M lib/product_details/json M locale Previous HEAD position was bc80c3f... use https for the gd fonts HEAD is now at 37170e5... update locale (again!) + git submodule update --init remote: Counting objects: 223, done. remote: Compressing objects: 100% (117/117), done. remote: Total 216 (delta 194), reused 116 (delta 99) Receiving objects: 100% (216/216), 34.52 KiB, done. Resolving deltas: 100% (194/194), completed with 6 local objects. From git://github.com/jbalogh/mozilla-product-details 3e86dcc..d52db60 master -> origin/master Submodule path 'lib/product_details/json': checked out 'd52db607d3cd7317022a4bec2323829f6298691e' remote: Counting objects: 923, done. remote: Compressing objects: 100% (336/336), done. remote: Total 842 (delta 596), reused 705 (delta 491) Receiving objects: 100% (842/842), 16.59 MiB | 5.80 MiB/s, done. Resolving deltas: 100% (596/596), completed with 30 local objects. From git://github.com/jbalogh/amo-locales2 c4a89fe..b98e006 master -> origin/master Submodule path 'locale': checked out 'b98e006a2599044681da25905e2a894f2f9e29eb' + set +x + pushd vendor + git fetch -t origin remote: Counting objects: 979, done. remote: Compressing objects: 100% (774/774), done. remote: Total 932 (delta 187), reused 761 (delta 140) Receiving objects: 100% (932/932), 2.19 MiB | 1.44 MiB/s, done. Resolving deltas: 100% (187/187), completed with 29 local objects. From git://github.com/jbalogh/zamboni-lib * [new tag] 5.12 -> 5.12 + git checkout 5.12 M src/check M src/commonware M src/django M src/django-cache-machine M src/django-devserver M src/django-extensions M src/django-multidb-router M src/django-nose M src/django-piston M src/django-queryset-transform M src/easy-thumbnails M src/happyforms M src/hera M src/jingo-minify M src/test-utils M src/tower Previous HEAD position was fdc0fe1... update commonware HEAD is now at d13284f... add django-csp to zamboni.pth + git submodule update --init Submodule 'src/amo-validator' (git://github.com/mattbasta/amo-validator.git) registered for path 'src/amo-validator' Submodule 'src/django-csp' (git://github.com/mozilla/django-csp.git) registered for path 'src/django-csp' Submodule 'src/nuggets' (git://github.com/mozilla/nuggets.git) registered for path 'src/nuggets' Submodule 'src/pymongo' (git://github.com/mongodb/mongo-python-driver.git) registered for path 'src/pymongo' Cloning into src/amo-validator... remote: Counting objects: 1158, done. remote: Compressing objects: 100% (586/586), done. remote: Total 1158 (delta 798), reused 812 (delta 552) Receiving objects: 100% (1158/1158), 2.22 MiB | 1016 KiB/s, done. Resolving deltas: 100% (798/798), done. Submodule path 'src/amo-validator': checked out 'bbd35bdc177227496f8f4baca9f5ed232eafe1f9' remote: Counting objects: 5, done. remote: Compressing objects: 100% (3/3), done. remote: Total 3 (delta 2), reused 0 (delta 0) Unpacking objects: 100% (3/3), done. From git://github.com/jbalogh/check 1c73813..03b26bc master -> origin/master Submodule path 'src/check': checked out '03b26bc3f12c1a43ab1156d50225d955667a642b' remote: Counting objects: 7, done. remote: Compressing objects: 100% (4/4), done. remote: Total 4 (delta 3), reused 0 (delta 0) Unpacking objects: 100% (4/4), done. From git://github.com/jbalogh/django-cache-machine + 782e4c9...de9ed74 redis -> origin/redis (forced update) error: You have local changes to 'caching/invalidation.py'; cannot switch branches. Unable to checkout 'de9ed743862ff1e02c80cfd9b47857ce98912263' in submodule path 'src/django-cache-machine' + popd + set +x
Running migration 79: DROP TABLE IF EXISTS `file_uploads`; CREATE TABLE `file_uploads` ( `created` datetime NOT NULL, `modified` datetime NOT NULL, `uuid` char(32) NOT NULL PRIMARY KEY, `path` varchar(255) NOT NULL, `name` varchar(255) NOT NULL, `user_id` int(11) unsigned, `validation` longtext, `task_error` longtext ) ENGINE=InnoDB DEFAULT CHARSET=utf8; ALTER TABLE `file_uploads` ADD CONSTRAINT FOREIGN KEY (`user_id`) REFERENCES `users` (`id`); That took 0.50 seconds ################################################## mysql> CREATE TABLE `piston_nonce` ( -> `id` integer AUTO_INCREMENT NOT NULL PRIMARY KEY, -> `token_key` varchar(18) NOT NULL, -> `consumer_key` varchar(18) NOT NULL, -> `key` varchar(255) NOT NULL -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8; .com',NOW(),NOW()); Query OK, 0 rows affected (0.09 sec) mysql> CREATE TABLE `piston_consumer` ( -> `id` integer AUTO_INCREMENT NOT NULL PRIMARY KEY, -> `name` varchar(255) NOT NULL, -> `description` longtext NOT NULL, -> `key` varchar(18) NOT NULL, -> `secret` varchar(32) NOT NULL, -> `status` varchar(16) NOT NULL, -> `user_id` integer -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8; Query OK, 0 rows affected (0.11 sec) mysql> ALTER TABLE `piston_consumer` ADD CONSTRAINT `user_id_refs_id_aad30107` FOREIGN KEY (`user_id`) REFERENCES `auth_user` (`id`); Query OK, 0 rows affected (0.14 sec) Records: 0 Duplicates: 0 Warnings: 0 mysql> CREATE TABLE `piston_token` ( -> `id` integer AUTO_INCREMENT NOT NULL PRIMARY KEY, -> `key` varchar(18) NOT NULL, -> `secret` varchar(32) NOT NULL, -> `verifier` varchar(10) NOT NULL, -> `token_type` integer NOT NULL, -> `timestamp` integer NOT NULL, -> `is_approved` bool NOT NULL, -> `user_id` integer, -> `consumer_id` integer NOT NULL, -> `callback` varchar(255), -> `callback_confirmed` bool NOT NULL -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8; Query OK, 0 rows affected (0.12 sec) mysql> ALTER TABLE `piston_token` ADD CONSTRAINT `user_id_refs_id_efc02d17` FOREIGN KEY (`user_id`) REFERENCES `auth_user` (`id`); Query OK, 0 rows affected (0.14 sec) Records: 0 Duplicates: 0 Warnings: 0 mysql> ALTER TABLE `piston_token` ADD CONSTRAINT `consumer_id_refs_id_85f42355` FOREIGN KEY (`consumer_id`) REFERENCES `piston_consumer` (`id`); Query OK, 0 rows affected (0.06 sec) Records: 0 Duplicates: 0 Warnings: 0 mysql> CREATE INDEX `piston_consumer_fbfc09f1` ON `piston_consumer` (`user_id`); Query OK, 0 rows affected (0.00 sec) Records: 0 Duplicates: 0 Warnings: 0 mysql> CREATE INDEX `piston_token_fbfc09f1` ON `piston_token` (`user_id`); Query OK, 0 rows affected (0.01 sec) Records: 0 Duplicates: 0 Warnings: 0 mysql> CREATE INDEX `piston_token_6565fc20` ON `piston_token` (`consumer_id`); Query OK, 0 rows affected (0.00 sec) Records: 0 Duplicates: 0 Warnings: 0 mysql> CREATE TABLE `users_blacklistedemaildomain` ( -> `id` int(11) NOT NULL auto_increment, -> `domain` varchar(255) NOT NULL default '', -> `created` datetime NOT NULL, -> `modified` datetime NOT NULL, -> PRIMARY KEY (`id`), -> UNIQUE KEY `domain` (`domain`) -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8; Query OK, 0 rows affected (0.07 sec) mysql> mysql> INSERT INTO -> `users_blacklistedemaildomain` (`domain`, `created`, `modified`) -> VALUES -> ('mailinator.com',NOW(),NOW()), -> ('mailinator2.com',NOW(),NOW()), -> ('fuckingduh.com',NOW(),NOW()), -> ('sogetthis.com',NOW(),NOW()), -> ('mailin8r.com',NOW(),NOW()), -> ('mailinator.net',NOW(),NOW()), -> ('klassmaster.com',NOW(),NOW()), -> ('putthisinyourspamdatabase.com',NOW(),NOW()), -> ('thisisnotmyrealemail.com',NOW(),NOW()), -> ('binkmail.com',NOW(),NOW()), -> ('spamhereplease.com',NOW(),NOW()), -> ('spamherelots.com',NOW(),NOW()), -> ('sendspamhere.com',NOW(),NOW()), -> ('chogmail.com',NOW(),NOW()), -> ('spamthisplease.com',NOW(),NOW()), -> ('frapmail.com',NOW(),NOW()), -> ('obobbo.com',NOW(),NOW()), -> ('devnullmail.com',NOW(),NOW()), -> ('bumpymail.com',NOW(),NOW()), -> ('centermail.com',NOW(),NOW()), -> ('centermail.net',NOW(),NOW()), -> ('discardmail.com',NOW(),NOW()), -> ('dodgeit.com',NOW(),NOW()), -> ('e4ward.com',NOW(),NOW()), -> ('emailias.com',NOW(),NOW()), -> ('fakeinformation.com',NOW(),NOW()), -> ('front14.org',NOW(),NOW()), -> ('ghosttexter.de',NOW(),NOW()), -> ('jetable.net',NOW(),NOW()), -> ('kasmail.com',NOW(),NOW()), -> ('link2mail.net',NOW(),NOW()), -> ('mailexpire.com',NOW(),NOW()), -> ('mailmoat.com',NOW(),NOW()), -> ('messagebeamer.de',NOW(),NOW()), -> ('mytrashmail.com',NOW(),NOW()), -> ('nervmich.net',NOW(),NOW()), -> ('netmails.net',NOW(),NOW()), -> ('netzidiot.de',NOW(),NOW()), -> ('nurfuerspam.de',NOW(),NOW()), -> ('oneoffemail.com',NOW(),NOW()), -> ('pookmail.com',NOW(),NOW()), -> ('privacy.net',NOW(),NOW()), -> ('punkass.com',NOW(),NOW()), -> ('sneakemail.com',NOW(),NOW()), -> ('sofort-mail.de',NOW(),NOW()), -> ('spam.la',NOW(),NOW()), -> ('spambob.com',NOW(),NOW()), -> ('spambob.net',NOW(),NOW()), -> ('spambob.org',NOW(),NOW()), -> ('spamex.com',NOW(),NOW()), -> ('spamgourmet.com',NOW(),NOW()), -> ('spamhole.com',NOW(),NOW()), -> ('spaminator.de',NOW(),NOW()), -> ('spammotel.com',NOW(),NOW()), -> ('spamtrail.com',NOW(),NOW()), -> ('trash-mail.de',NOW(),NOW()), -> ('trashmail.net',NOW(),NOW()), -> ('wuzup.net',NOW(),NOW()), -> ('suremail.info',NOW(),NOW()), -> ('spammote.com',NOW(),NOW()); Query OK, 60 rows affected (0.00 sec) Records: 60 Duplicates: 0 Warnings: 0
[root@mradm02 zamboni]# python26 manage.py make_django_users Spawning tasks to convert 739261 users.
[root@mradm02 zamboni]# python26 manage.py fix_user_images_again Looking for images in /mnt/netapp_amo/addons.mozilla.org-remora/shared_storage/uploads/userpics... I found 19509 images. Sending IDs to celeryd... All done.
Status: NEW → RESOLVED
Closed: 15 years ago
Resolution: --- → FIXED
verified.
Status: RESOLVED → VERIFIED
Component: Server Operations: Web Operations → WebOps: Other
Product: mozilla.org → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.