Closed Bug 882397 Opened 11 years ago Closed 11 years ago

push Socorro 50 release

Categories

(Infrastructure & Operations Graveyard :: WebOps: Other, task, P3)

Product:
Infrastructure & Operations Graveyard
Component:
WebOps: Other
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: lonnen, Assigned: bburton)

References

Details

(Whiteboard: [push interrupt])

No longer depends on: 879514
due to an extended db migration we want to run this on 06-13-13 @10:30 am PST
for once we're ahead of schedule. ready anytime.
Assignee: server-ops-webops → bburton
sp-collector08 had a sad the first time, but was happy after that

[root@sp-admin01.phx1 ~]# /root/bin/update-socorro-all.sh https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Upgrading socorro to https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Upgrading socorro on sp-collector01.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_24_50
Upgrading socorro on sp-collector02.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_17_24_54
Upgrading socorro on sp-collector03.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_17_24_58
Upgrading socorro on sp-collector04.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_17_25_02
Upgrading socorro on sp-collector05.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_17_25_05
Upgrading socorro on sp-collector06.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_17_25_09
Upgrading socorro on sp-mware01.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_25_13
Upgrading socorro on sp-mware02.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_25_15
Upgrading socorro on sp-mware03.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_25_16
Upgrading socorro on sp-processor01.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_25_18
Upgrading socorro on sp-processor02.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_25_23
Upgrading socorro on sp-processor03.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_25_43
Upgrading socorro on sp-processor04.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_26_02
Upgrading socorro on sp-processor05.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_26_11
Upgrading socorro on sp-processor06.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_26_16
Upgrading socorro on sp-processor07.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_26_20
Upgrading socorro on sp-processor08.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: Can't lock the lock file "/var/lock/socorro-processor.lock". Is another instance running?
[FAILED]
ERROR: could not start socorro-processor
Could not upgrade socorro on sp-processor08.phx1.mozilla.com, exit_code: 1
[root@sp-admin01.phx1 ~]# /root/bin/update-socorro-all.sh https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Upgrading socorro to https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Upgrading socorro on sp-collector01.phx1.mozilla.com
Upgrading socorro on sp-collector02.phx1.mozilla.com
Upgrading socorro on sp-collector03.phx1.mozilla.com
Upgrading socorro on sp-collector04.phx1.mozilla.com
Upgrading socorro on sp-collector05.phx1.mozilla.com
Upgrading socorro on sp-collector06.phx1.mozilla.com
Upgrading socorro on sp-mware01.phx1.mozilla.com
Upgrading socorro on sp-mware02.phx1.mozilla.com
Upgrading socorro on sp-mware03.phx1.mozilla.com
Upgrading socorro on sp-processor01.phx1.mozilla.com
Upgrading socorro on sp-processor02.phx1.mozilla.com
Upgrading socorro on sp-processor03.phx1.mozilla.com
Upgrading socorro on sp-processor04.phx1.mozilla.com
Upgrading socorro on sp-processor05.phx1.mozilla.com
Upgrading socorro on sp-processor06.phx1.mozilla.com
Upgrading socorro on sp-processor07.phx1.mozilla.com
Upgrading socorro on sp-processor08.phx1.mozilla.com
Upgrading socorro on sp-processor09.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_28_13
Upgrading socorro on sp-processor10.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_28_17
Upgrading socorro on sp-web01.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_28_21
Upgrading socorro on sp-web02.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_28_23
Upgrading socorro on sp-web03.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_28_25
Upgrading socorro on sp-web04.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_28_26
Upgrading socorro on sp-web05.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_28_28
Upgrading socorro on sp-admin01.phx1.mozilla.com
Stopping socorro-monitor: [  OK  ]
Starting socorro-monitor: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_28_30
Upgrading socorro on tp-socorro01-master01.phx1.mozilla.com
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_28_45
Upgrading socorro on tp-socorro01-master02.phx1.mozilla.com
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/391/artifact/socorro.tar.gz
Checksum: 3dac1796f79715656a5f482fbf2a9b3e
Backed up original to /data/socorro.13-06-2013_10_28_48
[sp-web01.phx1.mozilla.com] running: service memcached restart
[sp-web02.phx1.mozilla.com] running: service memcached restart
[sp-web03.phx1.mozilla.com] running: service memcached restart
[sp-web04.phx1.mozilla.com] running: service memcached restart
[sp-web05.phx1.mozilla.com] running: service memcached restart
[sp-web02.phx1.mozilla.com] finished: service memcached restart (0.532s)
[sp-web02.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web02.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web01.phx1.mozilla.com] finished: service memcached restart (0.533s)
[sp-web01.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web01.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web04.phx1.mozilla.com] finished: service memcached restart (0.531s)
[sp-web04.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web04.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web03.phx1.mozilla.com] finished: service memcached restart (0.534s)
[sp-web03.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web03.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web05.phx1.mozilla.com] finished: service memcached restart (0.536s)
[sp-web05.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web05.phx1.mozilla.com] out: Starting memcached: [  OK  ]
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
for posterity: processor-08 was acting up and needed to be rebooted afterwards.
Blocks: 887448
No longer blocks: 887448
Blocks: 892064
Blocks: 892171
No longer blocks: 892171
Blocks: 895130
Component: Server Operations: Web Operations → WebOps: Other
Product: mozilla.org → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.