Closed Bug 925032 Opened 12 years ago Closed 12 years ago

Push Socorro 62 release

Categories

(Infrastructure & Operations Graveyard :: WebOps: Socorro, task)

Product:
Infrastructure & Operations Graveyard
Component:
WebOps: Socorro
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: lonnen, Assigned: bburton)

Details

Assignee: server-ops-webops → bburton
Django push complete [root@socorroadm.private.phx1 ~]# /data/crashstats/src/crash-stats.mozilla.org/update https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz DIR: /data/crashstats/src/crash-stats.mozilla.org ENV: crashstats SITE: crash-stats.mozilla.org APP: socorro Upgrading socorro-crashstats (django) prod Upgrading socorro to https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz /data/crashstats/src/crash-stats.mozilla.org ~ Creating tables ... Installing custom SQL ... Installing indexes ... Installed 0 object(s) from 0 fixture(s) [2013-10-09 20:08:59] Running rsync_project [2013-10-09 20:08:59] [localhost] running: /usr/bin/rsync -aq --include '.gitkeep' --exclude '.git*' --exclude '.hg*' --exclude '.svn*' --exclude 'CVS' --exclude '.bzr*' --delete /data/crashstats/src/crash-stats.mozilla.org/ /data/crashstats/www/crash-stats.mozilla.org/ [2013-10-09 20:09:04] [localhost] finished: /usr/bin/rsync -aq --include '.gitkeep' --exclude '.git*' --exclude '.hg*' --exclude '.svn*' --exclude 'CVS' --exclude '.bzr*' --delete /data/crashstats/src/crash-stats.mozilla.org/ /data/crashstats/www/crash-stats.mozilla.org/ (5.252s) [2013-10-09 20:09:04] Finished rsync_project (5.252s) [2013-10-09 20:09:04] Running commit_www [2013-10-09 20:09:04] [localhost] running: cd /data/crashstats/www && /usr/bin/git add .; /usr/bin/git commit -a -m 'deploy ['crash-stats.mozilla.org']' [2013-10-09 20:09:11] [localhost] finished: cd /data/crashstats/www && /usr/bin/git add .; /usr/bin/git commit -a -m 'deploy ['crash-stats.mozilla.org']' (6.743s) [localhost] out: [master e983815] deploy [crash-stats.mozilla.org] [localhost] out: 105 files changed, 4940 insertions(+), 2125 deletions(-) [localhost] out: rewrite crash-stats.mozilla.org/socorro/webapp-django/nosetests.xml (86%) [localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/static/CACHE/css/96afb03efb3f.css [localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/static/CACHE/js/4c130b8fe834.js [localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/static/CACHE/js/549dad23584e.js [localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/static/browserid/persona-buttons.css [localhost] out: rewrite crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/base.py (83%) [localhost] out: rewrite crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/context_processors.py (83%) [localhost] out: rewrite crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/forms.py (66%) [localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/helpers.py [localhost] out: rewrite crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/signals.py (80%) [localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/static/browserid/persona-buttons.css [localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/templates/browserid/button.html [localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/templates/browserid/info.html [localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/templatetags/__init__.py [localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/templatetags/browserid.py [localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/tests/models.py [localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/tests/test_base.py [localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/tests/test_forms.py [localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/tests/test_helpers.py [localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/tests/test_urls.py [localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/tests/test_util.py [localhost] out: rewrite crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/tests/test_views.py (64%) [localhost] out: rewrite crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/tests/urls.py (74%) [localhost] out: rewrite crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/urls.py (92%) [localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/util.py [localhost] out: rewrite crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/django_browserid/views.py (64%) [localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/fancy_tag/__init__.py [localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/vendor/lib/python/six.py [localhost] out: create mode 100755 crash-stats.mozilla.org/socorro/webapp-django/vendor/src/funfactory/tests/test_middleware.py [2013-10-09 20:09:11] Finished commit_www (6.745s) [2013-10-09 20:09:11] Running push_www [2013-10-09 20:09:11] [socorro1.webapp.phx1.mozilla.com] running: /data/bin/update-www.sh crash-stats.mozilla.org [2013-10-09 20:09:11] [socorro2.webapp.phx1.mozilla.com] running: /data/bin/update-www.sh crash-stats.mozilla.org [2013-10-09 20:09:11] [socorro3.webapp.phx1.mozilla.com] running: /data/bin/update-www.sh crash-stats.mozilla.org [2013-10-09 20:09:11] [socorro4.webapp.phx1.mozilla.com] running: /data/bin/update-www.sh crash-stats.mozilla.org [2013-10-09 20:09:38] [socorro2.webapp.phx1.mozilla.com] finished: /data/bin/update-www.sh crash-stats.mozilla.org (27.514s) [2013-10-09 20:09:38] [socorro3.webapp.phx1.mozilla.com] finished: /data/bin/update-www.sh crash-stats.mozilla.org (27.570s) [2013-10-09 20:09:39] [socorro1.webapp.phx1.mozilla.com] finished: /data/bin/update-www.sh crash-stats.mozilla.org (27.969s) [socorro1.webapp.phx1.mozilla.com] out: Not removing crash-stats.mozilla.org/app/ [2013-10-09 20:09:39] [socorro4.webapp.phx1.mozilla.com] finished: /data/bin/update-www.sh crash-stats.mozilla.org (27.973s) [2013-10-09 20:09:39] Finished push_www (27.982s) [2013-10-09 20:09:39] [socorro1.webapp.phx1.mozilla.com] running: service httpd restart [2013-10-09 20:09:39] [socorro2.webapp.phx1.mozilla.com] running: service httpd restart [2013-10-09 20:09:39] [socorro3.webapp.phx1.mozilla.com] running: service httpd restart [2013-10-09 20:09:39] [socorro4.webapp.phx1.mozilla.com] running: service httpd restart [2013-10-09 20:09:40] [socorro1.webapp.phx1.mozilla.com] finished: service httpd restart (1.586s) [socorro1.webapp.phx1.mozilla.com] out: Stopping httpd: [ OK ] [socorro1.webapp.phx1.mozilla.com] out: Starting httpd: [ OK ] [2013-10-09 20:09:41] [socorro2.webapp.phx1.mozilla.com] finished: service httpd restart (1.666s) [socorro2.webapp.phx1.mozilla.com] out: Stopping httpd: [ OK ] [socorro2.webapp.phx1.mozilla.com] out: Starting httpd: [ OK ] [2013-10-09 20:09:41] [socorro3.webapp.phx1.mozilla.com] finished: service httpd restart (1.680s) [socorro3.webapp.phx1.mozilla.com] out: Stopping httpd: [ OK ] [socorro3.webapp.phx1.mozilla.com] out: Starting httpd: [ OK ] [2013-10-09 20:09:41] [socorro4.webapp.phx1.mozilla.com] finished: service httpd restart (1.679s) [socorro4.webapp.phx1.mozilla.com] out: Stopping httpd: [ OK ] [socorro4.webapp.phx1.mozilla.com] out: Starting httpd: [ OK ] [2013-10-09 20:09:41] [socorro-memcache1.webapp.phx1.mozilla.com] running: service memcached restart [2013-10-09 20:09:41] [socorro-memcache2.webapp.phx1.mozilla.com] running: service memcached restart [2013-10-09 20:09:41] [socorro-memcache1.webapp.phx1.mozilla.com] finished: service memcached restart (0.413s) [socorro-memcache1.webapp.phx1.mozilla.com] out: Stopping memcached: [ OK ] [socorro-memcache1.webapp.phx1.mozilla.com] out: Starting memcached: [ OK ] [2013-10-09 20:09:41] [socorro-memcache2.webapp.phx1.mozilla.com] finished: service memcached restart (0.552s) [socorro-memcache2.webapp.phx1.mozilla.com] out: Stopping memcached: [ OK ] [socorro-memcache2.webapp.phx1.mozilla.com] out: Starting memcached: [ OK ]
Status: NEW → ASSIGNED
Push complete [root@sp-admin01.phx1 ~]# issue-multi-command processors service socorro-processor status^C [root@sp-admin01.phx1 ~]# /root/bin/update-socorro-all.sh https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Upgrading socorro to https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Upgrading socorro on sp-collector01.phx1.mozilla.com Stopping socorro-crashmover: [ OK ] Starting socorro-crashmover: [ OK ] Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Checksum: 026ae8fb5d194c573ea3dade01eb93a9 Backed up original to /data/socorro.09-10-2013_13_11_27 Upgrading socorro on sp-collector02.phx1.mozilla.com Stopping socorro-crashmover: [ OK ] Starting socorro-crashmover: [ OK ] Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Checksum: 026ae8fb5d194c573ea3dade01eb93a9 Backed up original to /data/socorro.09-10-2013_20_11_51 Upgrading socorro on sp-collector03.phx1.mozilla.com Stopping socorro-crashmover: [ OK ] Starting socorro-crashmover: [ OK ] Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Checksum: 026ae8fb5d194c573ea3dade01eb93a9 Backed up original to /data/socorro.09-10-2013_20_12_01 Upgrading socorro on sp-collector04.phx1.mozilla.com Stopping socorro-crashmover: [ OK ] Starting socorro-crashmover: [ OK ] Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Checksum: 026ae8fb5d194c573ea3dade01eb93a9 Backed up original to /data/socorro.09-10-2013_20_12_11 Upgrading socorro on sp-collector05.phx1.mozilla.com Stopping socorro-crashmover: [ OK ] Starting socorro-crashmover: [ OK ] Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Checksum: 026ae8fb5d194c573ea3dade01eb93a9 Backed up original to /data/socorro.09-10-2013_20_12_19 Upgrading socorro on sp-collector06.phx1.mozilla.com Stopping socorro-crashmover: [ OK ] Starting socorro-crashmover: [ OK ] Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Checksum: 026ae8fb5d194c573ea3dade01eb93a9 Backed up original to /data/socorro.09-10-2013_20_12_29 Upgrading socorro on sp-mware01.phx1.mozilla.com Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Checksum: 026ae8fb5d194c573ea3dade01eb93a9 Backed up original to /data/socorro.09-10-2013_13_12_37 Upgrading socorro on sp-mware02.phx1.mozilla.com Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Checksum: 026ae8fb5d194c573ea3dade01eb93a9 Backed up original to /data/socorro.09-10-2013_13_12_48 Upgrading socorro on sp-mware03.phx1.mozilla.com Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Checksum: 026ae8fb5d194c573ea3dade01eb93a9 Backed up original to /data/socorro.09-10-2013_13_12_54 Upgrading socorro on sp-processor01.phx1.mozilla.com Stopping socorro-processor: [ OK ] Starting socorro-processor: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Checksum: 026ae8fb5d194c573ea3dade01eb93a9 Backed up original to /data/socorro.09-10-2013_13_13_03 Upgrading socorro on sp-processor02.phx1.mozilla.com Stopping socorro-processor: [ OK ] Starting socorro-processor: Can't lock the lock file "/var/lock/socorro-processor.lock". Is another instance running? [FAILED] ERROR: could not start socorro-processor Could not upgrade socorro on sp-processor02.phx1.mozilla.com, exit_code: 1 [root@sp-admin01.phx1 ~]# /root/bin/update-socorro-all.sh https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Upgrading socorro to https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Upgrading socorro on sp-collector01.phx1.mozilla.com Upgrading socorro on sp-collector02.phx1.mozilla.com Upgrading socorro on sp-collector03.phx1.mozilla.com Upgrading socorro on sp-collector04.phx1.mozilla.com Upgrading socorro on sp-collector05.phx1.mozilla.com Upgrading socorro on sp-collector06.phx1.mozilla.com Upgrading socorro on sp-mware01.phx1.mozilla.com Upgrading socorro on sp-mware02.phx1.mozilla.com Upgrading socorro on sp-mware03.phx1.mozilla.com Upgrading socorro on sp-processor01.phx1.mozilla.com Upgrading socorro on sp-processor02.phx1.mozilla.com Upgrading socorro on sp-processor03.phx1.mozilla.com Stopping socorro-processor: [ OK ] Starting socorro-processor: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Checksum: 026ae8fb5d194c573ea3dade01eb93a9 Backed up original to /data/socorro.09-10-2013_13_14_55 Upgrading socorro on sp-processor04.phx1.mozilla.com Stopping socorro-processor: [ OK ] Starting socorro-processor: Can't lock the lock file "/var/lock/socorro-processor.lock". Is another instance running? [FAILED] ERROR: could not start socorro-processor Could not upgrade socorro on sp-processor04.phx1.mozilla.com, exit_code: 1 [root@sp-admin01.phx1 ~]# /root/bin/update-socorro-all.sh https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Upgrading socorro to https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Upgrading socorro on sp-collector01.phx1.mozilla.com Upgrading socorro on sp-collector02.phx1.mozilla.com Upgrading socorro on sp-collector03.phx1.mozilla.com Upgrading socorro on sp-collector04.phx1.mozilla.com Upgrading socorro on sp-collector05.phx1.mozilla.com Upgrading socorro on sp-collector06.phx1.mozilla.com Upgrading socorro on sp-mware01.phx1.mozilla.com Upgrading socorro on sp-mware02.phx1.mozilla.com Upgrading socorro on sp-mware03.phx1.mozilla.com Upgrading socorro on sp-processor01.phx1.mozilla.com Upgrading socorro on sp-processor02.phx1.mozilla.com Upgrading socorro on sp-processor03.phx1.mozilla.com Upgrading socorro on sp-processor04.phx1.mozilla.com Upgrading socorro on sp-processor05.phx1.mozilla.com Stopping socorro-processor: [ OK ] Starting socorro-processor: Can't lock the lock file "/var/lock/socorro-processor.lock". Is another instance running? [FAILED] ERROR: could not start socorro-processor Could not upgrade socorro on sp-processor05.phx1.mozilla.com, exit_code: 1 [root@sp-admin01.phx1 ~]# /root/bin/update-socorro-all.sh https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Upgrading socorro to https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Upgrading socorro on sp-collector01.phx1.mozilla.com Upgrading socorro on sp-collector02.phx1.mozilla.com Upgrading socorro on sp-collector03.phx1.mozilla.com Upgrading socorro on sp-collector04.phx1.mozilla.com Upgrading socorro on sp-collector05.phx1.mozilla.com Upgrading socorro on sp-collector06.phx1.mozilla.com Upgrading socorro on sp-mware01.phx1.mozilla.com Upgrading socorro on sp-mware02.phx1.mozilla.com Upgrading socorro on sp-mware03.phx1.mozilla.com Upgrading socorro on sp-processor01.phx1.mozilla.com Upgrading socorro on sp-processor02.phx1.mozilla.com Upgrading socorro on sp-processor03.phx1.mozilla.com Upgrading socorro on sp-processor04.phx1.mozilla.com Upgrading socorro on sp-processor05.phx1.mozilla.com Upgrading socorro on sp-processor06.phx1.mozilla.com Stopping socorro-processor: [ OK ] Starting socorro-processor: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Checksum: 026ae8fb5d194c573ea3dade01eb93a9 Backed up original to /data/socorro.09-10-2013_13_18_41 Upgrading socorro on sp-processor07.phx1.mozilla.com Stopping socorro-processor: [ OK ] Starting socorro-processor: Can't lock the lock file "/var/lock/socorro-processor.lock". Is another instance running? [FAILED] ERROR: could not start socorro-processor Could not upgrade socorro on sp-processor07.phx1.mozilla.com, exit_code: 1 [root@sp-admin01.phx1 ~]# /root/bin/update-socorro-all.sh https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Upgrading socorro to https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Upgrading socorro on sp-collector01.phx1.mozilla.com Upgrading socorro on sp-collector02.phx1.mozilla.com Upgrading socorro on sp-collector03.phx1.mozilla.com Upgrading socorro on sp-collector04.phx1.mozilla.com Upgrading socorro on sp-collector05.phx1.mozilla.com Upgrading socorro on sp-collector06.phx1.mozilla.com Upgrading socorro on sp-mware01.phx1.mozilla.com Upgrading socorro on sp-mware02.phx1.mozilla.com Upgrading socorro on sp-mware03.phx1.mozilla.com Upgrading socorro on sp-processor01.phx1.mozilla.com Upgrading socorro on sp-processor02.phx1.mozilla.com Upgrading socorro on sp-processor03.phx1.mozilla.com Upgrading socorro on sp-processor04.phx1.mozilla.com Upgrading socorro on sp-processor05.phx1.mozilla.com Upgrading socorro on sp-processor06.phx1.mozilla.com Upgrading socorro on sp-processor07.phx1.mozilla.com Upgrading socorro on sp-processor08.phx1.mozilla.com Stopping socorro-processor: [ OK ] Starting socorro-processor: Can't lock the lock file "/var/lock/socorro-processor.lock". Is another instance running? [FAILED] ERROR: could not start socorro-processor Could not upgrade socorro on sp-processor08.phx1.mozilla.com, exit_code: 1 [root@sp-admin01.phx1 ~]# /root/bin/update-socorro-all.sh https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Upgrading socorro to https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Upgrading socorro on sp-collector01.phx1.mozilla.com Upgrading socorro on sp-collector02.phx1.mozilla.com Upgrading socorro on sp-collector03.phx1.mozilla.com Upgrading socorro on sp-collector04.phx1.mozilla.com Upgrading socorro on sp-collector05.phx1.mozilla.com Upgrading socorro on sp-collector06.phx1.mozilla.com Upgrading socorro on sp-mware01.phx1.mozilla.com Upgrading socorro on sp-mware02.phx1.mozilla.com Upgrading socorro on sp-mware03.phx1.mozilla.com Upgrading socorro on sp-processor01.phx1.mozilla.com Upgrading socorro on sp-processor02.phx1.mozilla.com Upgrading socorro on sp-processor03.phx1.mozilla.com Upgrading socorro on sp-processor04.phx1.mozilla.com Upgrading socorro on sp-processor05.phx1.mozilla.com Upgrading socorro on sp-processor06.phx1.mozilla.com Upgrading socorro on sp-processor07.phx1.mozilla.com Upgrading socorro on sp-processor08.phx1.mozilla.com Upgrading socorro on sp-processor09.phx1.mozilla.com Stopping socorro-processor: [ OK ] Starting socorro-processor: Can't lock the lock file "/var/lock/socorro-processor.lock". Is another instance running? [FAILED] ERROR: could not start socorro-processor Could not upgrade socorro on sp-processor09.phx1.mozilla.com, exit_code: 1 [root@sp-admin01.phx1 ~]# /root/bin/update-socorro-all.sh https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Upgrading socorro to https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Upgrading socorro on sp-collector01.phx1.mozilla.com Upgrading socorro on sp-collector02.phx1.mozilla.com Upgrading socorro on sp-collector03.phx1.mozilla.com Upgrading socorro on sp-collector04.phx1.mozilla.com Upgrading socorro on sp-collector05.phx1.mozilla.com Upgrading socorro on sp-collector06.phx1.mozilla.com Upgrading socorro on sp-mware01.phx1.mozilla.com Upgrading socorro on sp-mware02.phx1.mozilla.com Upgrading socorro on sp-mware03.phx1.mozilla.com Upgrading socorro on sp-processor01.phx1.mozilla.com Upgrading socorro on sp-processor02.phx1.mozilla.com Upgrading socorro on sp-processor03.phx1.mozilla.com Upgrading socorro on sp-processor04.phx1.mozilla.com Upgrading socorro on sp-processor05.phx1.mozilla.com Upgrading socorro on sp-processor06.phx1.mozilla.com Upgrading socorro on sp-processor07.phx1.mozilla.com Upgrading socorro on sp-processor08.phx1.mozilla.com Upgrading socorro on sp-processor09.phx1.mozilla.com Upgrading socorro on sp-processor10.phx1.mozilla.com Stopping socorro-processor: [ OK ] Starting socorro-processor: Can't lock the lock file "/var/lock/socorro-processor.lock". Is another instance running? [FAILED] ERROR: could not start socorro-processor Could not upgrade socorro on sp-processor10.phx1.mozilla.com, exit_code: 1 [root@sp-admin01.phx1 ~]# /root/bin/update-socorro-all.sh https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Upgrading socorro to https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Upgrading socorro on sp-collector01.phx1.mozilla.com Upgrading socorro on sp-collector02.phx1.mozilla.com Upgrading socorro on sp-collector03.phx1.mozilla.com Upgrading socorro on sp-collector04.phx1.mozilla.com Upgrading socorro on sp-collector05.phx1.mozilla.com Upgrading socorro on sp-collector06.phx1.mozilla.com Upgrading socorro on sp-mware01.phx1.mozilla.com Upgrading socorro on sp-mware02.phx1.mozilla.com Upgrading socorro on sp-mware03.phx1.mozilla.com Upgrading socorro on sp-processor01.phx1.mozilla.com Upgrading socorro on sp-processor02.phx1.mozilla.com Upgrading socorro on sp-processor03.phx1.mozilla.com Upgrading socorro on sp-processor04.phx1.mozilla.com Upgrading socorro on sp-processor05.phx1.mozilla.com Upgrading socorro on sp-processor06.phx1.mozilla.com Upgrading socorro on sp-processor07.phx1.mozilla.com Upgrading socorro on sp-processor08.phx1.mozilla.com Upgrading socorro on sp-processor09.phx1.mozilla.com Upgrading socorro on sp-processor10.phx1.mozilla.com Upgrading socorro on sp-web01.phx1.mozilla.com Stopping httpd: [ OK ] Starting httpd: Warning: DocumentRoot [/data/socorro/htdocs] does not exist Warning: DocumentRoot [/data/socorro/htdocs] does not exist [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Checksum: 026ae8fb5d194c573ea3dade01eb93a9 Backed up original to /data/socorro.09-10-2013_13_25_36 Upgrading socorro on sp-web02.phx1.mozilla.com Stopping httpd: [ OK ] Starting httpd: Warning: DocumentRoot [/data/socorro/htdocs] does not exist Warning: DocumentRoot [/data/socorro/htdocs] does not exist [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Checksum: 026ae8fb5d194c573ea3dade01eb93a9 Backed up original to /data/socorro.09-10-2013_13_25_43 Upgrading socorro on sp-web03.phx1.mozilla.com Stopping httpd: [ OK ] Starting httpd: Warning: DocumentRoot [/data/socorro/htdocs] does not exist Warning: DocumentRoot [/data/socorro/htdocs] does not exist [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Checksum: 026ae8fb5d194c573ea3dade01eb93a9 Backed up original to /data/socorro.09-10-2013_13_26_02 Upgrading socorro on sp-web04.phx1.mozilla.com Stopping httpd: [ OK ] Starting httpd: Warning: DocumentRoot [/data/socorro/htdocs] does not exist Warning: DocumentRoot [/data/socorro/htdocs] does not exist [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Checksum: 026ae8fb5d194c573ea3dade01eb93a9 Backed up original to /data/socorro.09-10-2013_13_26_12 Upgrading socorro on sp-web05.phx1.mozilla.com Stopping httpd: [ OK ] Starting httpd: Warning: DocumentRoot [/data/socorro/htdocs] does not exist Warning: DocumentRoot [/data/socorro/htdocs] does not exist [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Checksum: 026ae8fb5d194c573ea3dade01eb93a9 Backed up original to /data/socorro.09-10-2013_13_26_21 Upgrading socorro on sp-admin01.phx1.mozilla.com Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Checksum: 026ae8fb5d194c573ea3dade01eb93a9 Backed up original to /data/socorro.09-10-2013_13_26_29 Upgrading socorro on tp-socorro01-master01.phx1.mozilla.com Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Checksum: 026ae8fb5d194c573ea3dade01eb93a9 Backed up original to /data/socorro.09-10-2013_13_26_50 Upgrading socorro on tp-socorro01-master02.phx1.mozilla.com Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/602/artifact/socorro.tar.gz Checksum: 026ae8fb5d194c573ea3dade01eb93a9 Backed up original to /data/socorro.09-10-2013_13_27_02
Processor cleanup bburton@althalus [01:14:26] [~] -> % ssh sp-processor02.phx1.mozilla.com sudo rm -rf /var/lock/socorro-processor.lock Connection to sp-processor02.phx1.mozilla.com closed. bburton@althalus [01:14:40] [~] -> % ssh sp-processor02.phx1.mozilla.com sudo /sbin/service socorro-processor start Starting socorro-processor: [ OK ] Connection to sp-processor02.phx1.mozilla.com closed. bburton@althalus [01:14:52] [~] -> % ssh sp-processor04.phx1.mozilla.com sudo rm -rf /var/lock/socorro-processor.lock Connection to sp-processor04.phx1.mozilla.com closed. bburton@althalus [01:16:20] [~] -> % ssh sp-processor04.phx1.mozilla.com sudo /sbin/service socorro-processor start Starting socorro-processor: [ OK ] Connection to sp-processor04.phx1.mozilla.com closed. bburton@althalus [01:16:27] [~] -> % ssh sp-processor05.phx1.mozilla.com sudo rm -rf /var/lock/socorro-processor.lock Connection to sp-processor05.phx1.mozilla.com closed. bburton@althalus [01:17:42] [~] -> % ssh sp-processor05.phx1.mozilla.com sudo /sbin/service socorro-processor start Starting socorro-processor: [ OK ] Connection to sp-processor05.phx1.mozilla.com closed. bburton@althalus [01:17:58] [~] -> % ssh sp-processor07.phx1.mozilla.com sudo rm -rf /var/lock/socorro-processor.lock Connection to sp-processor07.phx1.mozilla.com closed. bburton@althalus [01:19:31] [~] -> % ssh sp-processor07.phx1.mozilla.com sudo /sbin/service socorro-processor start Starting socorro-processor: [ OK ] Connection to sp-processor07.phx1.mozilla.com closed. bburton@althalus [01:19:38] [~] -> % ssh sp-processor08.phx1.mozilla.com sudo rm -rf /var/lock/socorro-processor.lock Connection to sp-processor08.phx1.mozilla.com closed. bburton@althalus [01:21:08] [~] -> % ssh sp-processor08.phx1.mozilla.com sudo /sbin/service socorro-processor start Starting socorro-processor: [ OK ] Connection to sp-processor08.phx1.mozilla.com closed. bburton@althalus [01:21:14] [~] -> % ssh sp-processor09.phx1.mozilla.com sudo rm -rf /var/lock/socorro-processor.lock Connection to sp-processor09.phx1.mozilla.com closed. bburton@althalus [01:22:34] [~] -> % ssh sp-processor09.phx1.mozilla.com sudo /sbin/service socorro-processor start Starting socorro-processor: [ OK ] Connection to sp-processor09.phx1.mozilla.com closed. bburton@althalus [01:22:40] [~] -> % ssh sp-processor10.phx1.mozilla.com sudo rm -rf /var/lock/socorro-processor.lock Connection to sp-processor10.phx1.mozilla.com closed. bburton@althalus [01:24:38] [~] -> % ssh sp-processor10.phx1.mozilla.com sudo /sbin/service socorro-processor start Starting socorro-processor: [ OK ] Connection to sp-processor10.phx1.mozilla.com closed.
Status: ASSIGNED → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
QA verified: - automation passes - http://qa-selenium.mv.mozilla.com:8080/view/Socorro/job/socorro.prod/219/ - manual bfts pass - all push bugs were [qa-]; nothing to spot check
Status: RESOLVED → VERIFIED
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.