Closed Bug 913093 Opened 11 years ago Closed 11 years ago

push Socorro 58 release

Categories

(Infrastructure & Operations Graveyard :: WebOps: Socorro, task, P3)

Product:
Infrastructure & Operations Graveyard
Component:
WebOps: Socorro
Type:
task

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: lonnen, Assigned: bburton)

Details

(Whiteboard: [push interrupt])

Assignee: server-ops-webops → bburton
Status: NEW → ASSIGNED
Django pushed

[root@socorroadm.private.phx1 ~]# /data/crashstats/src/crash-stats.mozilla.org/update https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
DIR: /data/crashstats/src/crash-stats.mozilla.org
ENV: crashstats
SITE: crash-stats.mozilla.org
APP: socorro
Upgrading socorro-crashstats (django) prod
Upgrading socorro to https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
/data/crashstats/src/crash-stats.mozilla.org ~
Creating tables ...
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)
[2013-09-05 17:24:15] Running rsync_project
[2013-09-05 17:24:15] [localhost] running: /usr/bin/rsync -aq --include '.gitkeep' --exclude '.git*' --exclude '.hg*' --exclude '.svn*' --exclude 'CVS' --exclude '.bzr*' --delete /data/crashstats/src/crash-stats.mozilla.org/ /data/crashstats/www/crash-stats.mozilla.org/
[2013-09-05 17:24:20] [localhost] finished: /usr/bin/rsync -aq --include '.gitkeep' --exclude '.git*' --exclude '.hg*' --exclude '.svn*' --exclude 'CVS' --exclude '.bzr*' --delete /data/crashstats/src/crash-stats.mozilla.org/ /data/crashstats/www/crash-stats.mozilla.org/ (5.418s)
[2013-09-05 17:24:20] Finished rsync_project (5.419s)
[2013-09-05 17:24:20] Running commit_www
[2013-09-05 17:24:20] [localhost] running: cd /data/crashstats/www && /usr/bin/git add .; /usr/bin/git commit -a -m 'deploy ['crash-stats.mozilla.org']'
[2013-09-05 17:24:27] [localhost] finished: cd /data/crashstats/www && /usr/bin/git add .; /usr/bin/git commit -a -m 'deploy ['crash-stats.mozilla.org']' (6.541s)
[localhost] out: [master 86160ee] deploy [crash-stats.mozilla.org]
[localhost] out: 178 files changed, 2935 insertions(+), 569 deletions(-)
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/application/alembic/versions/2209ca57dcc6_bug_905346_timezone_.py
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/application/alembic/versions/4de11772649b_bug905346_fix_timezo.py
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/application/socorro/middleware/crashes_count_by_day_service.py
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/application/socorro/middleware/suspicious_service.py
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/application/tools/peep-0.4.tar.gz
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/socorro-virtualenv/bin/peep
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/socorro-virtualenv/lib/python2.6/site-packages/peep-0.4-py2.6.egg-info/PKG-INFO
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/socorro-virtualenv/lib/python2.6/site-packages/peep-0.4-py2.6.egg-info/SOURCES.txt
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/socorro-virtualenv/lib/python2.6/site-packages/peep-0.4-py2.6.egg-info/dependency_links.txt
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/socorro-virtualenv/lib/python2.6/site-packages/peep-0.4-py2.6.egg-info/entry_points.txt
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/socorro-virtualenv/lib/python2.6/site-packages/peep-0.4-py2.6.egg-info/installed-files.txt
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/socorro-virtualenv/lib/python2.6/site-packages/peep-0.4-py2.6.egg-info/requires.txt
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/socorro-virtualenv/lib/python2.6/site-packages/peep-0.4-py2.6.egg-info/top_level.txt
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/socorro-virtualenv/lib/python2.6/site-packages/peep/__init__.py
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/socorro-virtualenv/lib/python2.6/site-packages/wsgiref-0.1.2-py2.6.egg-info/PKG-INFO
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/socorro-virtualenv/lib/python2.6/site-packages/wsgiref-0.1.2-py2.6.egg-info/SOURCES.txt
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/socorro-virtualenv/lib/python2.6/site-packages/wsgiref-0.1.2-py2.6.egg-info/dependency_links.txt
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/socorro-virtualenv/lib/python2.6/site-packages/wsgiref-0.1.2-py2.6.egg-info/installed-files.txt
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/socorro-virtualenv/lib/python2.6/site-packages/wsgiref-0.1.2-py2.6.egg-info/top_level.txt
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/socorro-virtualenv/lib/python2.6/site-packages/wsgiref/__init__.py
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/socorro-virtualenv/lib/python2.6/site-packages/wsgiref/handlers.py
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/socorro-virtualenv/lib/python2.6/site-packages/wsgiref/headers.py
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/socorro-virtualenv/lib/python2.6/site-packages/wsgiref/simple_server.py
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/socorro-virtualenv/lib/python2.6/site-packages/wsgiref/util.py
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/socorro-virtualenv/lib/python2.6/site-packages/wsgiref/validate.py
[localhost] out: rewrite crash-stats.mozilla.org/socorro/webapp-django/nosetests.xml (65%)
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/static/CACHE/css/39e20fa5aaf8.css
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/static/CACHE/css/60e319f24233.css
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/static/CACHE/css/7e9656d27d93.css
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/static/CACHE/css/c83b241d863e.css
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/static/CACHE/css/e7acb85cd136.css
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/static/CACHE/css/eafba666fab4.css
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/static/CACHE/css/f7c0c799e34d.css
[localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/static/CACHE/css/fdea8e9bdfcc.css
[2013-09-05 17:24:27] Finished commit_www (6.543s)
[2013-09-05 17:24:27] Running push_www
[2013-09-05 17:24:27] [socorro1.webapp.phx1.mozilla.com] running: /data/bin/update-www.sh crash-stats.mozilla.org
[2013-09-05 17:24:27] [socorro2.webapp.phx1.mozilla.com] running: /data/bin/update-www.sh crash-stats.mozilla.org
[2013-09-05 17:24:27] [socorro3.webapp.phx1.mozilla.com] running: /data/bin/update-www.sh crash-stats.mozilla.org
[2013-09-05 17:24:27] [socorro4.webapp.phx1.mozilla.com] running: /data/bin/update-www.sh crash-stats.mozilla.org
[2013-09-05 17:24:45] [socorro1.webapp.phx1.mozilla.com] finished: /data/bin/update-www.sh crash-stats.mozilla.org (18.530s)
[socorro1.webapp.phx1.mozilla.com] out: Not removing crash-stats.mozilla.org/app/
[2013-09-05 17:24:46] [socorro4.webapp.phx1.mozilla.com] finished: /data/bin/update-www.sh crash-stats.mozilla.org (19.508s)
[2013-09-05 17:24:47] [socorro3.webapp.phx1.mozilla.com] finished: /data/bin/update-www.sh crash-stats.mozilla.org (20.024s)
[2013-09-05 17:24:47] [socorro2.webapp.phx1.mozilla.com] finished: /data/bin/update-www.sh crash-stats.mozilla.org (20.110s)
[2013-09-05 17:24:47] Finished push_www (20.113s)
[2013-09-05 17:24:47] [socorro1.webapp.phx1.mozilla.com] running: service httpd restart
[2013-09-05 17:24:47] [socorro2.webapp.phx1.mozilla.com] running: service httpd restart
[2013-09-05 17:24:47] [socorro3.webapp.phx1.mozilla.com] running: service httpd restart
[2013-09-05 17:24:47] [socorro4.webapp.phx1.mozilla.com] running: service httpd restart
[2013-09-05 17:24:49] [socorro1.webapp.phx1.mozilla.com] finished: service httpd restart (1.891s)
[socorro1.webapp.phx1.mozilla.com] out: Stopping httpd: [  OK  ]
[socorro1.webapp.phx1.mozilla.com] out: Starting httpd: [  OK  ]
[2013-09-05 17:24:49] [socorro3.webapp.phx1.mozilla.com] finished: service httpd restart (1.902s)
[socorro3.webapp.phx1.mozilla.com] out: Stopping httpd: [  OK  ]
[socorro3.webapp.phx1.mozilla.com] out: Starting httpd: [  OK  ]
[2013-09-05 17:24:49] [socorro4.webapp.phx1.mozilla.com] finished: service httpd restart (1.912s)
[socorro4.webapp.phx1.mozilla.com] out: Stopping httpd: [  OK  ]
[socorro4.webapp.phx1.mozilla.com] out: Starting httpd: [  OK  ]
[2013-09-05 17:24:49] [socorro2.webapp.phx1.mozilla.com] finished: service httpd restart (1.930s)
[socorro2.webapp.phx1.mozilla.com] out: Stopping httpd: [  OK  ]
[socorro2.webapp.phx1.mozilla.com] out: Starting httpd: [  OK  ]
[2013-09-05 17:24:49] [socorro-memcache1.webapp.phx1.mozilla.com] running: service memcached restart
[2013-09-05 17:24:49] [socorro-memcache2.webapp.phx1.mozilla.com] running: service memcached restart
[2013-09-05 17:24:50] [socorro-memcache2.webapp.phx1.mozilla.com] finished: service memcached restart (0.529s)
[socorro-memcache2.webapp.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[socorro-memcache2.webapp.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[2013-09-05 17:24:50] [socorro-memcache1.webapp.phx1.mozilla.com] finished: service memcached restart (0.638s)
[socorro-memcache1.webapp.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[socorro-memcache1.webapp.phx1.mozilla.com] out: Starting memcached: [  OK  ]
The rest of the push is complete

processors03 and 05 had to have their lock file manually cleaned up and be started manually, tp-socorro01-master02 script stuff is being dealt with in another bug

[root@sp-admin01.phx1 ~]# /root/bin/update-socorro-all.sh https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Upgrading socorro to https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Upgrading socorro on sp-collector01.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_10_26_07
Upgrading socorro on sp-collector02.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_17_26_13
Upgrading socorro on sp-collector03.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_17_26_19
Upgrading socorro on sp-collector04.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_17_26_39
Upgrading socorro on sp-collector05.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_17_26_45
Upgrading socorro on sp-collector06.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_17_27_16
Upgrading socorro on sp-mware01.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_10_27_22
Upgrading socorro on sp-mware02.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_10_27_25
Upgrading socorro on sp-mware03.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_10_27_31
Upgrading socorro on sp-processor01.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_10_27_35
Upgrading socorro on sp-processor02.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_10_27_42
Upgrading socorro on sp-processor03.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: Can't lock the lock file "/var/lock/socorro-processor.lock". Is another instance running?
[FAILED]
ERROR: could not start socorro-processor
Could not upgrade socorro on sp-processor03.phx1.mozilla.com, exit_code: 1
[root@sp-admin01.phx1 ~]# /root/bin/update-socorro-all.sh https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Upgrading socorro to https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Upgrading socorro on sp-collector01.phx1.mozilla.com
Upgrading socorro on sp-collector02.phx1.mozilla.com
Upgrading socorro on sp-collector03.phx1.mozilla.com
Upgrading socorro on sp-collector04.phx1.mozilla.com
Upgrading socorro on sp-collector05.phx1.mozilla.com
Upgrading socorro on sp-collector06.phx1.mozilla.com
Upgrading socorro on sp-mware01.phx1.mozilla.com
Upgrading socorro on sp-mware02.phx1.mozilla.com
Upgrading socorro on sp-mware03.phx1.mozilla.com
Upgrading socorro on sp-processor01.phx1.mozilla.com
Upgrading socorro on sp-processor02.phx1.mozilla.com
Upgrading socorro on sp-processor03.phx1.mozilla.com
Upgrading socorro on sp-processor04.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_10_29_21
Upgrading socorro on sp-processor05.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: Can't lock the lock file "/var/lock/socorro-processor.lock". Is another instance running?
[FAILED]
ERROR: could not start socorro-processor
Could not upgrade socorro on sp-processor05.phx1.mozilla.com, exit_code: 1
[root@sp-admin01.phx1 ~]# /root/bin/update-socorro-all.sh https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Upgrading socorro to https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Upgrading socorro on sp-collector01.phx1.mozilla.com
Upgrading socorro on sp-collector02.phx1.mozilla.com
Upgrading socorro on sp-collector03.phx1.mozilla.com
Upgrading socorro on sp-collector04.phx1.mozilla.com
Upgrading socorro on sp-collector05.phx1.mozilla.com
Upgrading socorro on sp-collector06.phx1.mozilla.com
Upgrading socorro on sp-mware01.phx1.mozilla.com
Upgrading socorro on sp-mware02.phx1.mozilla.com
Upgrading socorro on sp-mware03.phx1.mozilla.com
Upgrading socorro on sp-processor01.phx1.mozilla.com
Upgrading socorro on sp-processor02.phx1.mozilla.com
Upgrading socorro on sp-processor03.phx1.mozilla.com
Upgrading socorro on sp-processor04.phx1.mozilla.com
Upgrading socorro on sp-processor05.phx1.mozilla.com
Upgrading socorro on sp-processor06.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_10_30_43
Upgrading socorro on sp-processor07.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_10_31_05
Upgrading socorro on sp-processor08.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_10_31_27
Upgrading socorro on sp-processor09.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_10_31_53
Upgrading socorro on sp-processor10.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_10_32_16
Upgrading socorro on sp-web01.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: Warning: DocumentRoot [/data/socorro/htdocs] does not exist
Warning: DocumentRoot [/data/socorro/htdocs] does not exist
[  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_10_32_38
Upgrading socorro on sp-web02.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: Warning: DocumentRoot [/data/socorro/htdocs] does not exist
Warning: DocumentRoot [/data/socorro/htdocs] does not exist
[  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_10_32_41
Upgrading socorro on sp-web03.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: Warning: DocumentRoot [/data/socorro/htdocs] does not exist
Warning: DocumentRoot [/data/socorro/htdocs] does not exist
[  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_10_32_45
Upgrading socorro on sp-web04.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: Warning: DocumentRoot [/data/socorro/htdocs] does not exist
Warning: DocumentRoot [/data/socorro/htdocs] does not exist
[  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_10_32_49
Upgrading socorro on sp-web05.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: Warning: DocumentRoot [/data/socorro/htdocs] does not exist
Warning: DocumentRoot [/data/socorro/htdocs] does not exist
[  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_10_32_53
Upgrading socorro on sp-admin01.phx1.mozilla.com
Stopping socorro-monitor: [  OK  ]
Starting socorro-monitor: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_10_32_57
Upgrading socorro on tp-socorro01-master01.phx1.mozilla.com
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_10_33_14
Upgrading socorro on tp-socorro01-master02.phx1.mozilla.com
sudo: /data/bin/update-socorro.sh: command not found
Could not upgrade socorro on tp-socorro01-master02.phx1.mozilla.com, exit_code: 1
[root@sp-admin01.phx1 ~]# scp /data/bin/update-socorro.sh tp-socorro01-master02.phx1.mozilla.com:/data/bin/
update-socorro.sh                                                                                                                               100% 2396     2.3KB/s   00:00
[root@sp-admin01.phx1 ~]# /root/bin/update-socorro-all.sh https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Upgrading socorro to https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Upgrading socorro on sp-collector01.phx1.mozilla.com
Upgrading socorro on sp-collector02.phx1.mozilla.com
Upgrading socorro on sp-collector03.phx1.mozilla.com
Upgrading socorro on sp-collector04.phx1.mozilla.com
Upgrading socorro on sp-collector05.phx1.mozilla.com
Upgrading socorro on sp-collector06.phx1.mozilla.com
Upgrading socorro on sp-mware01.phx1.mozilla.com
Upgrading socorro on sp-mware02.phx1.mozilla.com
Upgrading socorro on sp-mware03.phx1.mozilla.com
Upgrading socorro on sp-processor01.phx1.mozilla.com
Upgrading socorro on sp-processor02.phx1.mozilla.com
Upgrading socorro on sp-processor03.phx1.mozilla.com
Upgrading socorro on sp-processor04.phx1.mozilla.com
Upgrading socorro on sp-processor05.phx1.mozilla.com
Upgrading socorro on sp-processor06.phx1.mozilla.com
Upgrading socorro on sp-processor07.phx1.mozilla.com
Upgrading socorro on sp-processor08.phx1.mozilla.com
Upgrading socorro on sp-processor09.phx1.mozilla.com
Upgrading socorro on sp-processor10.phx1.mozilla.com
Upgrading socorro on sp-web01.phx1.mozilla.com
Upgrading socorro on sp-web02.phx1.mozilla.com
Upgrading socorro on sp-web03.phx1.mozilla.com
Upgrading socorro on sp-web04.phx1.mozilla.com
Upgrading socorro on sp-web05.phx1.mozilla.com
Upgrading socorro on sp-admin01.phx1.mozilla.com
Upgrading socorro on tp-socorro01-master01.phx1.mozilla.com
Upgrading socorro on tp-socorro01-master02.phx1.mozilla.com
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/513/artifact/socorro.tar.gz
Checksum: 18f1ea14415c3a85333db7fe821470c6
Backed up original to /data/socorro.05-09-2013_10_35_25
[sp-web01.phx1.mozilla.com] running: service memcached restart
[sp-web02.phx1.mozilla.com] running: service memcached restart
[sp-web03.phx1.mozilla.com] running: service memcached restart
[sp-web04.phx1.mozilla.com] running: service memcached restart
[sp-web05.phx1.mozilla.com] running: service memcached restart
[sp-web02.phx1.mozilla.com] finished: service memcached restart (0.456s)
[sp-web02.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web02.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web03.phx1.mozilla.com] finished: service memcached restart (0.543s)
[sp-web03.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web03.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web01.phx1.mozilla.com] finished: service memcached restart (0.548s)
[sp-web01.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web01.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web04.phx1.mozilla.com] finished: service memcached restart (0.552s)
[sp-web04.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web04.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web05.phx1.mozilla.com] finished: service memcached restart (0.576s)
[sp-web05.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web05.phx1.mozilla.com] out: Starting memcached: [  OK  ]
Status: ASSIGNED → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
QA verified -  thank you all for a seamless and quick release.
- Automation run - one failure that is being investigated - bug 913126
- Manual smoke tests pass
Status: RESOLVED → VERIFIED
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.