Closed Bug 810298 Opened 12 years ago Closed 12 years ago

re-push Socorro 25 to prod

Categories

(Infrastructure & Operations Graveyard :: WebOps: Other, task, P1)

Product:
Infrastructure & Operations Graveyard
Component:
WebOps: Other
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: lonnen, Assigned: fox2mike)

Details

There is a serious regression in Socorro 26. We need to push 25 back into production while we figure out what to do about 26.

Build: https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
This is urgent.
Severity: normal → blocker
Priority: P3 → P1
Assignee: server-ops-webops → shyam
Push complete :

[shyam@sp-admin01.phx1 ~]$ sudo /root/bin/update-socorro-all.sh https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Upgrading socorro to https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Upgrading socorro on sp-collector01.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_21_11
Upgrading socorro on sp-collector02.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_21_16
Upgrading socorro on sp-collector03.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_21_21
Upgrading socorro on sp-collector04.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_21_28
Upgrading socorro on sp-collector05.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_21_50
Upgrading socorro on sp-collector06.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_21_55
Upgrading socorro on sp-mware01.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_22_11
Upgrading socorro on sp-mware02.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_22_27
Upgrading socorro on sp-mware03.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_22_29
Upgrading socorro on sp-processor01.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_22_30
Upgrading socorro on sp-processor02.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_22_50
Upgrading socorro on sp-processor03.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_22_54
Upgrading socorro on sp-processor04.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_22_59
Upgrading socorro on sp-processor05.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_23_02
Upgrading socorro on sp-processor06.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_23_06
Upgrading socorro on sp-processor07.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_23_11
Upgrading socorro on sp-processor08.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_23_30
Upgrading socorro on sp-processor09.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_23_35
Upgrading socorro on sp-processor10.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_23_54
Upgrading socorro on sp-web01.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_23_57
Upgrading socorro on sp-web02.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_23_59
Upgrading socorro on sp-web03.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_24_03
Upgrading socorro on sp-web04.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_24_05
Upgrading socorro on sp-web05.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_24_19
Upgrading socorro on sp-admin01.phx1.mozilla.com
Stopping socorro-monitor: [  OK  ]
Starting socorro-monitor: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_24_20
Upgrading socorro on tp-socorro01-master01.phx1.mozilla.com
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_24_27
Upgrading socorro on tp-socorro01-master02.phx1.mozilla.com
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/256/artifact/socorro.tar.gz
Checksum: 1ae3b7a90689b663ee8fac542041e4c9
Backed up original to /data/socorro.09-11-2012_07_24_29
[sp-web01.phx1.mozilla.com] running: service memcached restart
[sp-web02.phx1.mozilla.com] running: service memcached restart
[sp-web03.phx1.mozilla.com] running: service memcached restart
[sp-web04.phx1.mozilla.com] running: service memcached restart
[sp-web05.phx1.mozilla.com] running: service memcached restart
[sp-web05.phx1.mozilla.com] finished: service memcached restart (0.251s)
[sp-web05.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web05.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web02.phx1.mozilla.com] finished: service memcached restart (0.259s)
[sp-web02.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web02.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web01.phx1.mozilla.com] finished: service memcached restart (0.262s)
[sp-web01.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web01.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web03.phx1.mozilla.com] finished: service memcached restart (0.260s)
[sp-web03.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web03.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web04.phx1.mozilla.com] finished: service memcached restart (0.258s)
[sp-web04.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web04.phx1.mozilla.com] out: Starting memcached: [  OK  ]
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
from QA: Our test automation is passing.
Component: Server Operations: Web Operations → WebOps: Other
Product: mozilla.org → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.