Closed Bug 778972 Opened 12 years ago Closed 12 years ago

push Socorro 17.3 release

Categories

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Product:
Infrastructure & Operations Graveyard
Component:
WebOps: Other
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: rhelmer, Assigned: bburton)

References

(
URL
)

Details

(Whiteboard: ETA: Mon Jul 30 4:00 PM PDT) 

We need to fix a few regressions and also make a crontab change.

After the release goes out please add this to /etc/cron.d/socorro:

# generate nightly reports for crash-stats UI
00 02 * * * socorro /data/socorro/application/scripts/crons/cron_daily_matviews.sh
Whiteboard: ETA: Mon Jul 30 4:00 PM PDT
Pushed to stage

[root@socorroadm.stage.private.phx1 socorro]# /root/bin/auto-update.sh
Installing b0ee6f688ec3c4f373a4e2fd8e0ab646
Stopping socorro-monitor:                                  [  OK  ]
Starting socorro-monitor:                                  [  OK  ]
Updating and restarting services on socorro-processor1.stage.metrics.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Updating and restarting services on socorro-processor2.stage.metrics.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Updating and restarting services on socorro-web1.stage.webapp.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Updating and restarting services on socorro-web2.stage.webapp.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Updating and restarting services on socorro-collector1.stage.webapp.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Updating and restarting services on socorro-collector2.stage.webapp.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Updating and restarting services on socorro-mware1.stage.webapp.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Updating and restarting services on socorro-mware2.stage.webapp.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Updating and restarting services on socorro1.stage.db.phx1.mozilla.com
Updating and restarting services on socorro1.dev.db.phx1.mozilla.com
Restarting memcache on socorro-web1.stage.webapp.phx1.mozilla.com
Shutting down Distributed memory caching (memcached): [  OK  ]
Starting Distributed memory caching (memcached): [  OK  ]
Restarting memcache on socorro-web2.stage.webapp.phx1.mozilla.com
Shutting down Distributed memory caching (memcached): [  OK  ]
Starting Distributed memory caching (memcached): [  OK  ]
Assignee: server-ops-webops → bburton
Status: NEW → ASSIGNED
[root@sp-admin01.phx1 ~]# /root/bin/update-socorro-all.sh https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Upgrading socorro to https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Upgrading socorro on sp-collector01.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_16_59_59
Upgrading socorro on sp-collector02.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_00_04
Upgrading socorro on sp-collector03.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_00_08
Upgrading socorro on sp-collector04.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_00_14
Upgrading socorro on sp-collector05.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_00_19
Upgrading socorro on sp-collector06.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_00_24
Upgrading socorro on sp-mware01.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_00_29
Upgrading socorro on sp-mware02.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_00_31
Upgrading socorro on sp-mware03.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_00_33
Upgrading socorro on sp-processor01.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_00_49
Upgrading socorro on sp-processor02.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_00_53
Upgrading socorro on sp-processor03.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_00_57
Upgrading socorro on sp-processor04.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_01_01
Upgrading socorro on sp-processor05.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_01_05
Upgrading socorro on sp-processor06.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: Can't lock the lock file "/var/lock/socorro-processor.lock". Is another instance running?
[FAILED]
ERROR: could not start socorro-processor
Could not upgrade socorro on sp-processor06.phx1.mozilla.com, exit_code: 1
[root@sp-admin01.phx1 ~]# /root/bin/update-socorro-all.sh https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Upgrading socorro to https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Upgrading socorro on sp-collector01.phx1.mozilla.com
Upgrading socorro on sp-collector02.phx1.mozilla.com
Upgrading socorro on sp-collector03.phx1.mozilla.com
Upgrading socorro on sp-collector04.phx1.mozilla.com
Upgrading socorro on sp-collector05.phx1.mozilla.com
Upgrading socorro on sp-collector06.phx1.mozilla.com
Upgrading socorro on sp-mware01.phx1.mozilla.com
Upgrading socorro on sp-mware02.phx1.mozilla.com
Upgrading socorro on sp-mware03.phx1.mozilla.com
Upgrading socorro on sp-processor01.phx1.mozilla.com
Upgrading socorro on sp-processor02.phx1.mozilla.com
Upgrading socorro on sp-processor03.phx1.mozilla.com
Upgrading socorro on sp-processor04.phx1.mozilla.com
Upgrading socorro on sp-processor05.phx1.mozilla.com
Upgrading socorro on sp-processor06.phx1.mozilla.com
Upgrading socorro on sp-processor07.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_04_18
Upgrading socorro on sp-processor08.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_04_33
Upgrading socorro on sp-processor09.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_04_47
Upgrading socorro on sp-processor10.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_04_52
Upgrading socorro on sp-web01.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_04_56
Upgrading socorro on sp-web02.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_04_57
Upgrading socorro on sp-web03.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_04_59
Upgrading socorro on sp-web04.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_05_01
Upgrading socorro on sp-web05.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_05_02
Upgrading socorro on sp-admin01.phx1.mozilla.com
Stopping socorro-monitor: [  OK  ]
Starting socorro-monitor: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_05_04
Upgrading socorro on tp-socorro01-master01.phx1.mozilla.com
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_05_09
Upgrading socorro on tp-socorro01-master02.phx1.mozilla.com
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/209/artifact/socorro.tar.gz
Checksum: b0ee6f688ec3c4f373a4e2fd8e0ab646
Backed up original to /data/socorro.30-07-2012_17_05_11
[sp-web01.phx1.mozilla.com] running: service memcached restart
[sp-web02.phx1.mozilla.com] running: service memcached restart
[sp-web03.phx1.mozilla.com] running: service memcached restart
[sp-web04.phx1.mozilla.com] running: service memcached restart
[sp-web05.phx1.mozilla.com] running: service memcached restart
[sp-web01.phx1.mozilla.com] finished: service memcached restart (0.338s)
[sp-web01.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web01.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web02.phx1.mozilla.com] finished: service memcached restart (0.527s)
[sp-web02.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web02.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web05.phx1.mozilla.com] finished: service memcached restart (0.524s)
[sp-web05.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web05.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web03.phx1.mozilla.com] finished: service memcached restart (0.533s)
[sp-web03.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web03.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web04.phx1.mozilla.com] finished: service memcached restart (0.533s)
[sp-web04.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web04.phx1.mozilla.com] out: Starting memcached: [  OK  ]
Status: ASSIGNED → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Bumping to verified:
- automation passes
- verified Bug 778807 - crash-stats.mozilla.com is showing bug numbers as stricken through, regardless of actual bug status
Status: RESOLVED → VERIFIED
Almost forgot cron addition :(

Added to stage and prod

+
+# generate nightly reports for crash-stats UI
+00 02 * * * socorro /data/socorro/application/scripts/crons/cron_daily_matviews.sh
Component: Server Operations: Web Operations → WebOps: Other
Product: mozilla.org → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.