If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Status

Infrastructure & Operations
WebOps: Other
P3
normal
RESOLVED FIXED
5 years ago
4 years ago

People

(Reporter: lonnen, Assigned: cturra)

Tracking

Details

(Whiteboard: [push interrupt])

(Reporter)

Description

5 years ago
Build: https://ci.mozilla.org/job/socorro-release/379/
(Reporter)

Comment 1

5 years ago
hold on a minute. there's a packaging issue with the build
(Reporter)

Comment 2

5 years ago
new Build: https://ci.mozilla.org/job/socorro-release/380/

we're good to go
(Assignee)

Comment 3

5 years ago
push complete! there were two processor nodes that needed some manual tlc tho.

[cturra@sp-admin01.phx1 ~]$ sudo /root/bin/update-socorro-all.sh https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Upgrading socorro to https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Upgrading socorro on sp-collector01.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_14_57_24
Upgrading socorro on sp-collector02.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_21_57_29
Upgrading socorro on sp-collector03.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_21_57_44
Upgrading socorro on sp-collector04.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_21_57_49
Upgrading socorro on sp-collector05.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_21_57_52
Upgrading socorro on sp-collector06.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_21_57_57
Upgrading socorro on sp-mware01.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_14_58_02
Upgrading socorro on sp-mware02.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_14_58_10
Upgrading socorro on sp-mware03.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_14_58_12
Upgrading socorro on sp-processor01.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: Can't lock the lock file "/var/lock/socorro-processor.lock". Is another instance running?
[FAILED]
ERROR: could not start socorro-processor
Could not upgrade socorro on sp-processor01.phx1.mozilla.com, exit_code: 1
... manual tlc...

[cturra@sp-admin01.phx1 ~]$ sudo /root/bin/update-socorro-cturra.sh https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Upgrading socorro to https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Upgrading socorro on sp-processor02.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_15_01_14
Upgrading socorro on sp-processor03.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_15_01_29
Upgrading socorro on sp-processor04.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_15_01_49
Upgrading socorro on sp-processor05.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_15_02_20
Upgrading socorro on sp-processor06.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_15_02_40
Upgrading socorro on sp-processor07.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_15_03_01
Upgrading socorro on sp-processor08.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_15_03_11
Upgrading socorro on sp-processor09.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: Can't lock the lock file "/var/lock/socorro-processor.lock". Is another instance running?
[FAILED]
ERROR: could not start socorro-processor
Could not upgrade socorro on sp-processor09.phx1.mozilla.com, exit_code: 1
... manual tlc...

[cturra@sp-admin01.phx1 ~]$ sudo /root/bin/update-socorro-cturra.sh https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Upgrading socorro to https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Upgrading socorro on sp-processor10.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_15_06_21
Upgrading socorro on sp-web01.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_15_06_26
Upgrading socorro on sp-web02.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_15_06_35
Upgrading socorro on sp-web03.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_15_06_39
Upgrading socorro on sp-web04.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_15_06_46
Upgrading socorro on sp-web05.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_15_06_55
Upgrading socorro on sp-admin01.phx1.mozilla.com
Stopping socorro-monitor: [  OK  ]
Starting socorro-monitor: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_15_06_58
Upgrading socorro on tp-socorro01-master01.phx1.mozilla.com
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_15_07_24
Upgrading socorro on tp-socorro01-master02.phx1.mozilla.com
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/380/artifact/socorro.tar.gz
Checksum: ae9282ce956077386f132541712515bb
Backed up original to /data/socorro.01-05-2013_15_07_32
[sp-web01.phx1.mozilla.com] running: service memcached restart
[sp-web02.phx1.mozilla.com] running: service memcached restart
[sp-web03.phx1.mozilla.com] running: service memcached restart
[sp-web04.phx1.mozilla.com] running: service memcached restart
[sp-web05.phx1.mozilla.com] running: service memcached restart
[sp-web01.phx1.mozilla.com] finished: service memcached restart (5.615s)
[sp-web01.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web01.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web03.phx1.mozilla.com] finished: service memcached restart (5.649s)
[sp-web03.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web03.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web02.phx1.mozilla.com] finished: service memcached restart (5.651s)
[sp-web02.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web02.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web04.phx1.mozilla.com] finished: service memcached restart (5.695s)
[sp-web04.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web04.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web05.phx1.mozilla.com] finished: service memcached restart (5.695s)
[sp-web05.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web05.phx1.mozilla.com] out: Starting memcached: [  OK  ]
Assignee: server-ops-webops → cturra
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
Component: Server Operations: Web Operations → WebOps: Other
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.