Closed Bug 719032 Opened 14 years ago Closed 14 years ago

Emergency SUMO push 2012-01-18

Categories

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Product:
Infrastructure & Operations Graveyard
Component:
WebOps: Other
Platform:
All
Other
Type:
task
Priority:
Not set
Severity:
critical

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: rrosario, Assigned: jason)

Details

2 bad regressions were found after yesterday's push. The tag is 2012-01-18. Normal push.
Assignee: server-ops → jthomas
Pushed to prod: [root@supportadm.private.phx1 support.mozilla.org]# ./update 2012-01-18 + cd /data/support/src/support.mozilla.org/kitsune + echo -e 'Fetching tags...' Fetching tags... + git fetch -t remote: Counting objects: 22, done. remote: Compressing objects: 100% (7/7), done. remote: Total 14 (delta 11), reused 10 (delta 7) Unpacking objects: 100% (14/14), done. From git://github.com/mozilla/kitsune * [new tag] 2012-01-18 -> 2012-01-18 + checkretval + retval=0 + [[ 0 -gt 0 ]] + echo -e 'Updating code...' Updating code... + git checkout 2012-01-18 Previous HEAD position was d708da6... [bug 717063] Use new vote macro in mobile document.html HEAD is now at baaeed1... [bug 718889] Correctly enable gallery radio buttons. + checkretval + retval=0 + [[ 0 -gt 0 ]] + echo -e 'Sync submodule...' Sync submodule... + git submodule sync Synchronizing submodule url for 'media/global' Synchronizing submodule url for 'templates/global' Synchronizing submodule url for 'vendor/src/bleach' Synchronizing submodule url for 'vendor/src/cache-panel' Synchronizing submodule url for 'vendor/src/celery' Synchronizing submodule url for 'vendor/src/check' Synchronizing submodule url for 'vendor/src/commonware' Synchronizing submodule url for 'vendor/src/django' Synchronizing submodule url for 'vendor/src/django-adminplus' Synchronizing submodule url for 'vendor/src/django-arecibo' Synchronizing submodule url for 'vendor/src/django-authority' Synchronizing submodule url for 'vendor/src/django-cache-machine' Synchronizing submodule url for 'vendor/src/django-celery' Synchronizing submodule url for 'vendor/src/django-cronjobs' Synchronizing submodule url for 'vendor/src/django-csp' Synchronizing submodule url for 'vendor/src/django-debug-toolbar' Synchronizing submodule url for 'vendor/src/django-extensions' Synchronizing submodule url for 'vendor/src/django-mobility' Synchronizing submodule url for 'vendor/src/django-mozilla-product-details' Synchronizing submodule url for 'vendor/src/django-multidb-router' Synchronizing submodule url for 'vendor/src/django-nose' Synchronizing submodule url for 'vendor/src/django-picklefield' Synchronizing submodule url for 'vendor/src/django-qunit' Synchronizing submodule url for 'vendor/src/django-ratelimit' Synchronizing submodule url for 'vendor/src/django-recaptcha' Synchronizing submodule url for 'vendor/src/django-session-csrf' Synchronizing submodule url for 'vendor/src/django-tidings' Synchronizing submodule url for 'vendor/src/django-timezones' Synchronizing submodule url for 'vendor/src/django-waffle' Synchronizing submodule url for 'vendor/src/elasticutils' Synchronizing submodule url for 'vendor/src/jingo' Synchronizing submodule url for 'vendor/src/jingo-minify' Synchronizing submodule url for 'vendor/src/jstestnetlib' Synchronizing submodule url for 'vendor/src/kombu' Synchronizing submodule url for 'vendor/src/oedipus' Synchronizing submodule url for 'vendor/src/py-wikimarkup' Synchronizing submodule url for 'vendor/src/pystatsd' Synchronizing submodule url for 'vendor/src/python-recaptcha' Synchronizing submodule url for 'vendor/src/redis-py' Synchronizing submodule url for 'vendor/src/schematic' Synchronizing submodule url for 'vendor/src/sphinxapi' Synchronizing submodule url for 'vendor/src/test-utils' Synchronizing submodule url for 'vendor/src/tower' Synchronizing submodule url for 'vendor/src/tweepy' + checkretval + retval=0 + [[ 0 -gt 0 ]] + echo -e 'Update submodule...' Update submodule... + git submodule update --init --recursive + checkretval + retval=0 + [[ 0 -gt 0 ]] + cd /data/support/src/support.mozilla.org/kitsune + find /data/support/src/support.mozilla.org/kitsune -name '*.pyc' -delete + python ./manage.py compress_assets To turn on cache busting, use settings.CACHEBUST_IMGS Minifying /data/support/src/support.mozilla.org/kitsune/media/js/wiki-all.js (using YUI Compressor) cat: /data/support/src/support.mozilla.org/kitsune/media/js/libs/highstock.src.js: No such file or directory Minifying /data/support/src/support.mozilla.org/kitsune/media/js/gallery-all.js (using YUI Compressor) Unchanged files skipped for minification: 36 + checkretval + retval=0 + [[ 0 -gt 0 ]] + python ./vendor/src/schematic/schematic migrations/ + checkretval + retval=0 + [[ 0 -gt 0 ]] + cd /data/support/src/support.mozilla.org/kitsune/locale + svn up At revision 100105. + checkretval + retval=0 + [[ 0 -gt 0 ]] + ./compile-mo.sh . + checkretval + retval=0 + [[ 0 -gt 0 ]] + cd /data/support/src/support.mozilla.org/kitsune/scripts/crontab + ./gen-crons.py -k /data/support/src/support.mozilla.org/kitsune -u apache + checkretval + retval=0 + [[ 0 -gt 0 ]] + cd /data/support/src/support.mozilla.org/kitsune + find /data/support/src/support.mozilla.org/kitsune -name '*.pyc' -delete + read -p 'Do you really want to push SUMO and reload apache? [y/n] ' YESNO Do you really want to push SUMO and reload apache? [y/n] y + [[ y != \y ]] + /data/support/deploy -n support.mozilla.org [2012-01-18 08:26:13] Running rsync_project [2012-01-18 08:26:13] [localhost] running: /usr/bin/rsync -aq --include '.gitkeep' --exclude '.git*' --exclude '.hg*' --delete /data/support/src/support.mozilla.org/ /data/support/www/support.mozilla.org/ [2012-01-18 08:26:20] [localhost] finished: /usr/bin/rsync -aq --include '.gitkeep' --exclude '.git*' --exclude '.hg*' --delete /data/support/src/support.mozilla.org/ /data/support/www/support.mozilla.org/ (7.388s) + issue-multi-command support /data/bin/update-www.sh [2012-01-18 08:26:20] [support1.webapp.phx1.mozilla.com] running: /data/bin/update-www.sh [2012-01-18 08:26:20] [support2.webapp.phx1.mozilla.com] running: /data/bin/update-www.sh [2012-01-18 08:26:20] [support3.webapp.phx1.mozilla.com] running: /data/bin/update-www.sh [2012-01-18 08:26:20] [support4.webapp.phx1.mozilla.com] running: /data/bin/update-www.sh [2012-01-18 08:26:21] [support5.webapp.phx1.mozilla.com] running: /data/bin/update-www.sh [2012-01-18 08:26:22] [support2.webapp.phx1.mozilla.com] finished: /data/bin/update-www.sh (1.456s) [2012-01-18 08:26:22] [support1.webapp.phx1.mozilla.com] finished: /data/bin/update-www.sh (1.767s) [2012-01-18 08:26:22] [support4.webapp.phx1.mozilla.com] finished: /data/bin/update-www.sh (1.929s) [2012-01-18 08:26:23] [support5.webapp.phx1.mozilla.com] finished: /data/bin/update-www.sh (2.021s) [2012-01-18 08:26:23] [support3.webapp.phx1.mozilla.com] finished: /data/bin/update-www.sh (2.056s) + issue-multi-command support-celery /data/bin/update-www.sh [2012-01-18 08:26:23] [sumocelery1.webapp.phx1.mozilla.com] running: /data/bin/update-www.sh [2012-01-18 08:26:24] [sumocelery1.webapp.phx1.mozilla.com] finished: /data/bin/update-www.sh (1.210s) + scp /data/support/src/support.mozilla.org/kitsune/configs/sphinx/sphinx.conf pp-app-sphinx01.phx.mozilla.com:/data/sphinx/support/sphinx.conf sphinx.conf 100% 12KB 12.3KB/s 00:00 + scp /data/support/src/support.mozilla.org/kitsune/configs/sphinx/sphinx.conf pp-app-sphinx02.phx.mozilla.com:/data/sphinx/support/sphinx.conf sphinx.conf 100% 12KB 12.3KB/s 00:00 + scp /data/support/src/support.mozilla.org/kitsune/configs/sphinx/stopwords.txt pp-app-sphinx01.phx.mozilla.com:/data/sphinx/support/stopwords.txt stopwords.txt 100% 954 0.9KB/s 00:00 + scp /data/support/src/support.mozilla.org/kitsune/configs/sphinx/stopwords.txt pp-app-sphinx02.phx.mozilla.com:/data/sphinx/support/stopwords.txt stopwords.txt 100% 954 0.9KB/s 00:00 + echo 'Restarting httpd...' Restarting httpd... + issue-multi-command support /sbin/service httpd graceful [2012-01-18 08:26:26] [support1.webapp.phx1.mozilla.com] running: /sbin/service httpd graceful [2012-01-18 08:26:26] [support2.webapp.phx1.mozilla.com] running: /sbin/service httpd graceful [2012-01-18 08:26:26] [support3.webapp.phx1.mozilla.com] running: /sbin/service httpd graceful [2012-01-18 08:26:26] [support4.webapp.phx1.mozilla.com] running: /sbin/service httpd graceful [2012-01-18 08:26:26] [support5.webapp.phx1.mozilla.com] running: /sbin/service httpd graceful [2012-01-18 08:26:27] [support1.webapp.phx1.mozilla.com] finished: /sbin/service httpd graceful (0.739s) [2012-01-18 08:26:27] [support5.webapp.phx1.mozilla.com] finished: /sbin/service httpd graceful (0.743s) [2012-01-18 08:26:27] [support2.webapp.phx1.mozilla.com] finished: /sbin/service httpd graceful (0.799s) [2012-01-18 08:26:27] [support3.webapp.phx1.mozilla.com] finished: /sbin/service httpd graceful (0.808s) [2012-01-18 08:26:27] [support4.webapp.phx1.mozilla.com] finished: /sbin/service httpd graceful (0.832s) + echo 'Restarting Celery...' Restarting Celery... + issue-multi-command support-celery /sbin/service celeryd-kitsune-prod restart [2012-01-18 08:26:27] [sumocelery1.webapp.phx1.mozilla.com] running: /sbin/service celeryd-kitsune-prod restart [2012-01-18 08:36:39] [sumocelery1.webapp.phx1.mozilla.com] finished: /sbin/service celeryd-kitsune-prod restart (612.360s) [sumocelery1.webapp.phx1.mozilla.com] out: Restarting celery-kitsune-prod: celery-kitsune-prod: stopped [sumocelery1.webapp.phx1.mozilla.com] out: celery-kitsune-prod: started [sumocelery1.webapp.phx1.mozilla.com] out: [ OK ]
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
Thanks!
Status: RESOLVED → VERIFIED
Component: Server Operations: Web Operations → WebOps: Other
Product: mozilla.org → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.