[amo] Update zamboni to 6.1.2.2

RESOLVED FIXED

Status

--
critical
RESOLVED FIXED
7 years ago
5 years ago

People

(Reporter: jbalogh, Assigned: oremj)

Tracking

Details

(Reporter)

Description

7 years ago
Please update zamboni to fix an error that's blocking up our queues. This should not wait for oremj to come back, it's going to start hurting the site soon.

Here's the commands to run on mradm02:

cd /data/amo_python/src/prod/zamboni
commander /data/amo_python/src/prod/zamboni/scripts/deploy.py start_update:6.1.2.2,6.1.2
commander /data/amo_python/src/prod/zamboni/scripts/deploy.py update_amo

Thanks!
(Assignee)

Updated

7 years ago
Assignee: server-ops → jeremy.orem+bugs
(Assignee)

Comment 1

7 years ago
[root@mradm02 ~]# cd /data/amo_python/src/prod/zamboni
[root@mradm02 zamboni]# commander /data/amo_python/src/prod/zamboni/scripts/deploy.py start_update:6.1.2.2,6.1.2
Running 'start_update'
[localhost] running: mv /etc/cron.d/addons-prod-maint /tmp/addons-prod-maint
[localhost] finished: mv /etc/cron.d/addons-prod-maint /tmp/addons-prod-maint (0.051s)
[localhost] running: cd /data/amo_python/src/prod/zamboni && /usr/bin/git fetch -t origin
[localhost] finished: cd /data/amo_python/src/prod/zamboni && /usr/bin/git fetch -t origin (0.558s)
[localhost] err: From git://gitmirror.mozilla.org/jbalogh/zamboni
[localhost] err: * [new tag]         6.1.2.2    -> 6.1.2.2
[localhost] running: cd /data/amo_python/src/prod/zamboni && /usr/bin/git checkout 6.1.2.2
[localhost] finished: cd /data/amo_python/src/prod/zamboni && /usr/bin/git checkout 6.1.2.2 (0.213s)
[localhost] err: Previous HEAD position was 61c305c... turns out that did not work
[localhost] err: HEAD is now at 21a3049... only index if elastic is on
[localhost] running: cd /data/amo_python/src/prod/zamboni && /usr/bin/git submodule sync
[localhost] finished: cd /data/amo_python/src/prod/zamboni && /usr/bin/git submodule sync (0.118s)
[localhost] out: Synchronizing submodule url for 'lib/product_details/json'
[localhost] out: Synchronizing submodule url for 'media/css/global'
[localhost] out: Synchronizing submodule url for 'media/js/global'
[localhost] out: Synchronizing submodule url for 'templates/global'
[localhost] running: cd /data/amo_python/src/prod/zamboni && /usr/bin/git submodule update --init
[localhost] finished: cd /data/amo_python/src/prod/zamboni && /usr/bin/git submodule update --init (0.170s)
[localhost] running: cd /data/amo_python/src/prod/zamboni/vendor && /usr/bin/git fetch -t origin
[localhost] finished: cd /data/amo_python/src/prod/zamboni/vendor && /usr/bin/git fetch -t origin (5.280s)
[localhost] running: cd /data/amo_python/src/prod/zamboni/vendor && /usr/bin/git checkout 6.1.2
[localhost] finished: cd /data/amo_python/src/prod/zamboni/vendor && /usr/bin/git checkout 6.1.2 (1.023s)
[localhost] out: M	src/check
[localhost] out: M	src/django-mobility
[localhost] out: M	src/django-multidb-router
[localhost] out: M	src/django-piston
[localhost] out: M	src/django-queryset-transform
[localhost] out: M	src/django-quieter-formset
[localhost] out: M	src/easy-thumbnails
[localhost] out: M	src/elasticutils
[localhost] out: M	src/happyforms
[localhost] out: M	src/hera
[localhost] err: HEAD is now at 65a6056... update sdk docs to 1.0rc4
[localhost] running: cd /data/amo_python/src/prod/zamboni/vendor && /usr/bin/git submodule sync
[localhost] finished: cd /data/amo_python/src/prod/zamboni/vendor && /usr/bin/git submodule sync (1.002s)
[localhost] out: Synchronizing submodule url for 'src/addon-sdk-sdocs'
[localhost] out: Synchronizing submodule url for 'src/amo-validator'
[localhost] out: Synchronizing submodule url for 'src/bleach'
[localhost] out: Synchronizing submodule url for 'src/cache-panel'
[localhost] out: Synchronizing submodule url for 'src/check'
[localhost] out: Synchronizing submodule url for 'src/commonware'
[localhost] out: Synchronizing submodule url for 'src/django'
[localhost] out: Synchronizing submodule url for 'src/django-cache-machine'
[localhost] out: Synchronizing submodule url for 'src/django-cronjobs'
[localhost] out: Synchronizing submodule url for 'src/django-csp'
[localhost] out: Synchronizing submodule url for 'src/django-debug-toolbar'
[localhost] out: Synchronizing submodule url for 'src/django-devserver'
[localhost] out: Synchronizing submodule url for 'src/django-extensions'
[localhost] out: Synchronizing submodule url for 'src/django-fixture-magic'
[localhost] out: Synchronizing submodule url for 'src/django-mobility'
[localhost] out: Synchronizing submodule url for 'src/django-multidb-router'
[localhost] out: Synchronizing submodule url for 'src/django-nose'
[localhost] out: Synchronizing submodule url for 'src/django-piston'
[localhost] out: Synchronizing submodule url for 'src/django-queryset-transform'
[localhost] out: Synchronizing submodule url for 'src/django-quieter-formset'
[localhost] out: Synchronizing submodule url for 'src/django-qunit'
[localhost] out: Synchronizing submodule url for 'src/django-recaptcha'
[localhost] out: Synchronizing submodule url for 'src/django-tables'
[localhost] out: Synchronizing submodule url for 'src/django-waffle'
[localhost] out: Synchronizing submodule url for 'src/easy-thumbnails'
[localhost] out: Synchronizing submodule url for 'src/elasticutils'
[localhost] out: Synchronizing submodule url for 'src/fastchardet'
[localhost] out: Synchronizing submodule url for 'src/happyforms'
[localhost] out: Synchronizing submodule url for 'src/hera'
[localhost] out: Synchronizing submodule url for 'src/jingo'
[localhost] out: Synchronizing submodule url for 'src/jingo-minify'
[localhost] out: Synchronizing submodule url for 'src/jstestnetlib'
[localhost] out: Synchronizing submodule url for 'src/nuggets'
[localhost] out: Synchronizing submodule url for 'src/pymongo'
[localhost] out: Synchronizing submodule url for 'src/python-oauth2'
[localhost] out: Synchronizing submodule url for 'src/redis'
[localhost] out: Synchronizing submodule url for 'src/schematic'
[localhost] out: Synchronizing submodule url for 'src/statsd'
[localhost] out: Synchronizing submodule url for 'src/test-utils'
[localhost] out: Synchronizing submodule url for 'src/tower'
[localhost] running: cd /data/amo_python/src/prod/zamboni/vendor && /usr/bin/git submodule update --init
[localhost] finished: cd /data/amo_python/src/prod/zamboni/vendor && /usr/bin/git submodule update --init (1.187s)
[root@mradm02 zamboni]# commander /data/amo_python/src/prod/zamboni/scripts/deploy.py update_amo
Running 'update_amo'
[localhost] running: cd /data/amo_python/src/prod/zamboni/locale && svn revert -R .
[localhost] finished: cd /data/amo_python/src/prod/zamboni/locale && svn revert -R . (4.418s)
[localhost] running: cd /data/amo_python/src/prod/zamboni/locale && svn up
[localhost] finished: cd /data/amo_python/src/prod/zamboni/locale && svn up (1.810s)
[localhost] out: U    nl/LC_MESSAGES/messages.po
[localhost] out: Updated to revision 91335.
[localhost] running: cd /data/amo_python/src/prod/zamboni && python2.6 manage.py compress_assets
[localhost] finished: cd /data/amo_python/src/prod/zamboni && python2.6 manage.py compress_assets  (47.587s)
[localhost] running: cd /data/amo_python/src/prod/zamboni && python2.6 ./vendor/src/schematic/schematic migrations
[localhost] finished: cd /data/amo_python/src/prod/zamboni && python2.6 ./vendor/src/schematic/schematic migrations (3.895s)
[pm-app-amo02] running: service crond stop
[pm-app-amo03] running: service crond stop
[pm-app-amo04] running: service crond stop
[pm-app-amo05] running: service crond stop
[pm-app-amo07] running: service crond stop
[pm-app-amo10] running: service crond stop
[pm-app-amo11] running: service crond stop
[pm-app-amo12] running: service crond stop
[pm-app-amo13] running: service crond stop
[pm-app-amo14] running: service crond stop
[pm-app-amo15] running: service crond stop
[pm-app-amo16] running: service crond stop
[pm-app-amo17] running: service crond stop
[pm-app-amo18] running: service crond stop
[pm-app-amo19] running: service crond stop
[pm-app-amo20] running: service crond stop
[pm-app-amo21] running: service crond stop
[pm-app-amo24] running: service crond stop
[pm-app-amo25] running: service crond stop
[pm-app-amo27] running: service crond stop
[pm-app-amodev01] running: service crond stop
[pm-app-amo26] running: service crond stop
[pm-app-amo20] finished: service crond stop (1.266s)
[pm-app-amo20] out: Stopping crond: [  OK  ]
[pm-app-amodev01] finished: service crond stop (1.257s)
[pm-app-amodev01] out: Stopping crond: [  OK  ]
[pm-app-amo26] finished: service crond stop (1.258s)
[pm-app-amo26] out: Stopping crond: [  OK  ]
[pm-app-amo16] finished: service crond stop (1.371s)
[pm-app-amo16] out: Stopping crond: [  OK  ]
[pm-app-amo24] finished: service crond stop (1.357s)
[pm-app-amo24] out: Stopping crond: [  OK  ]
[pm-app-amo11] finished: service crond stop (1.447s)
[pm-app-amo11] out: Stopping crond: [  OK  ]
[pm-app-amo17] finished: service crond stop (1.435s)
[pm-app-amo17] out: Stopping crond: [  OK  ]
[pm-app-amo10] finished: service crond stop (1.518s)
[pm-app-amo10] out: Stopping crond: [  OK  ]
[pm-app-amo04] finished: service crond stop (1.640s)
[pm-app-amo04] out: Stopping crond: [  OK  ]
[pm-app-amo21] finished: service crond stop (1.655s)
[pm-app-amo21] out: Stopping crond: [  OK  ]
[pm-app-amo18] finished: service crond stop (1.819s)
[pm-app-amo18] out: Stopping crond: [  OK  ]
[pm-app-amo19] finished: service crond stop (1.865s)
[pm-app-amo19] out: Stopping crond: [  OK  ]
[pm-app-amo15] finished: service crond stop (2.516s)
[pm-app-amo15] out: Stopping crond: [  OK  ]
[pm-app-amo05] finished: service crond stop (3.213s)
[pm-app-amo05] out: Stopping crond: [  OK  ]
[pm-app-amo03] finished: service crond stop (3.343s)
[pm-app-amo03] out: Stopping crond: [  OK  ]
[pm-app-amo07] finished: service crond stop (3.625s)
[pm-app-amo07] out: Stopping crond: [  OK  ]
[pm-app-amo02] finished: service crond stop (4.010s)
[pm-app-amo02] out: Stopping crond: [  OK  ]
[pm-app-amo25] finished: service crond stop (4.291s)
[pm-app-amo25] out: Stopping crond: [  OK  ]
[pm-app-amo12] finished: service crond stop (4.844s)
[pm-app-amo12] out: Stopping crond: [  OK  ]
[pm-app-amo13] finished: service crond stop (6.136s)
[pm-app-amo13] out: Stopping crond: [  OK  ]
[pm-app-amo14] finished: service crond stop (10.768s)
[pm-app-amo14] out: Stopping crond: [  OK  ]
[pm-app-amo27] finished: service crond stop (12.839s)
[pm-app-amo27] out: Stopping crond: [  OK  ]
[localhost] running: cd /data/amo_python && /usr/bin/rsync -aq --exclude '.git*' --delete src/prod/zamboni/media/ www/prod/zamboni/media/
[localhost] finished: cd /data/amo_python && /usr/bin/rsync -aq --exclude '.git*' --delete src/prod/zamboni/media/ www/prod/zamboni/media/ (0.618s)
[localhost] running: cd /data/amo_python/www && /usr/bin/git add .
[localhost] finished: cd /data/amo_python/www && /usr/bin/git add . (2.662s)
[localhost] running: cd /data/amo_python/www && /usr/bin/git commit -q -a -m 'AMO PUSH'
[localhost] finished: cd /data/amo_python/www && /usr/bin/git commit -q -a -m 'AMO PUSH' (2.707s)
[localhost] out: # On branch master
[localhost] out: nothing to commit (working directory clean)
[pm-app-amo02] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo03] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo05] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo04] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo07] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo05] finished: /data/bin/libget/get-php5-www-git.sh (6.400s)
[pm-app-amo10] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo04] finished: /data/bin/libget/get-php5-www-git.sh (6.771s)
[pm-app-amo11] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo07] finished: /data/bin/libget/get-php5-www-git.sh (8.661s)
[pm-app-amo12] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo03] finished: /data/bin/libget/get-php5-www-git.sh (8.679s)
[pm-app-amo13] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo02] finished: /data/bin/libget/get-php5-www-git.sh (9.838s)
[pm-app-amo14] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo10] finished: /data/bin/libget/get-php5-www-git.sh (4.095s)
[pm-app-amo15] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo11] finished: /data/bin/libget/get-php5-www-git.sh (4.490s)
[pm-app-amo16] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo15] finished: /data/bin/libget/get-php5-www-git.sh (4.719s)
[pm-app-amo17] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo16] finished: /data/bin/libget/get-php5-www-git.sh (3.997s)
[pm-app-amo18] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo12] finished: /data/bin/libget/get-php5-www-git.sh (9.407s)
[pm-app-amo19] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo17] finished: /data/bin/libget/get-php5-www-git.sh (4.937s)
[pm-app-amo20] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo18] finished: /data/bin/libget/get-php5-www-git.sh (5.146s)
[pm-app-amo21] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo19] finished: /data/bin/libget/get-php5-www-git.sh (4.640s)
[pm-app-amo24] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo20] finished: /data/bin/libget/get-php5-www-git.sh (4.438s)
[pm-app-amo25] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo13] finished: /data/bin/libget/get-php5-www-git.sh (17.241s)
[pm-app-amo26] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo21] finished: /data/bin/libget/get-php5-www-git.sh (5.601s)
[pm-app-amo27] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo24] finished: /data/bin/libget/get-php5-www-git.sh (4.578s)
[pm-app-amo24] out: Not removing preview/zamboni/apps/cronjobs/
[pm-app-amo24] out: Not removing preview/zamboni/vendor/lib/python/carrot/
[pm-app-amodev01] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amo26] finished: /data/bin/libget/get-php5-www-git.sh (2.841s)
[pm-app-amo26] out: Not removing prod/zamboni/apps/cronjobs/
[pm-app-amo26] out: Not removing prod/zamboni/apps/nick/
[pm-app-amo26] out: Not removing prod/zamboni/vendor/lib/python/carrot/
[pm-gearman-amo01] running: /data/bin/libget/get-php5-www-git.sh
[pm-app-amodev01] finished: /data/bin/libget/get-php5-www-git.sh (2.885s)
[pm-app-amodev01] out: Not removing builder.next/flightdeck/apps/api/
[pm-app-amodev01] out: Not removing builder.preview/flightdeck/apps/api/
[pm-app-amodev01] out: Not removing builder/flightdeck/apps/api/
[pm-app-amodev01] out: Not removing next/zamboni/vendor/lib/python/carrot/
[pm-app-amodev01] out: Not removing preview/zamboni/apps/cronjobs/
[pm-app-amodev01] out: Not removing preview/zamboni/apps/nick/
[pm-app-amodev01] out: Not removing preview/zamboni/vendor/lib/python/carrot/
[pm-app-amodev01] out: Not removing prod/zamboni/apps/nick/
[pm-app-amodev01] out: Not removing prod/zamboni/vendor/lib/python/carrot/
[pm-gearman-amo02] running: /data/bin/libget/get-php5-www-git.sh
[pm-gearman-amo01] finished: /data/bin/libget/get-php5-www-git.sh (3.919s)
[pm-gearman-amo02] finished: /data/bin/libget/get-php5-www-git.sh (3.443s)
[pm-gearman-amo02] out: Not removing preview/zamboni/vendor/lib/python/carrot/
[pm-app-amo25] finished: /data/bin/libget/get-php5-www-git.sh (9.496s)
[pm-app-amo27] finished: /data/bin/libget/get-php5-www-git.sh (9.221s)

It's still restarting celery and whatnot, but looks like it will finish up soon.
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED
Component: Server Operations: Web Operations → WebOps: Other
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.