Closed Bug 711103 Opened 14 years ago Closed 14 years ago

Push Socorro 2.3.4.1

Categories

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Product:
Infrastructure & Operations Graveyard
Component:
WebOps: Other
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: laura, Assigned: jason)

Details

Build https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Revision c920975555e7f0ca5e6aca020037052340f7a36a Awaiting QA signoff, mbrandt please comment in the bug. This release fixes bug 710757.
Assignee: server-ops → jthomas
Pushed to prod. [root@sp-admin01.phx1 ~]# update-socorro-all.sh https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Upgrading socorro to https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Upgrading socorro on sp-collector01.phx1.mozilla.com Stopping socorro-crashmover: [ OK ] Starting socorro-crashmover: [ OK ] Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_26_21 Upgrading socorro on sp-collector02.phx1.mozilla.com Stopping socorro-crashmover: [ OK ] Starting socorro-crashmover: [ OK ] Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_26_30 Upgrading socorro on sp-collector03.phx1.mozilla.com Stopping socorro-crashmover: [ OK ] Starting socorro-crashmover: [ OK ] Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_26_35 Upgrading socorro on sp-collector04.phx1.mozilla.com Stopping socorro-crashmover: [ OK ] Starting socorro-crashmover: [ OK ] Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_26_50 Upgrading socorro on sp-collector05.phx1.mozilla.com Stopping socorro-crashmover: [ OK ] Starting socorro-crashmover: [ OK ] Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_27_09 Upgrading socorro on sp-collector06.phx1.mozilla.com Stopping socorro-crashmover: [ OK ] Starting socorro-crashmover: [ OK ] Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_27_15 Upgrading socorro on sp-mware01.phx1.mozilla.com Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_27_33 Upgrading socorro on sp-mware02.phx1.mozilla.com Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_27_35 Upgrading socorro on sp-mware03.phx1.mozilla.com Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_27_38 Upgrading socorro on sp-processor01.phx1.mozilla.com Stopping socorro-processor: [ OK ] Starting socorro-processor: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_27_40 Upgrading socorro on sp-processor02.phx1.mozilla.com Stopping socorro-processor: [ OK ] Starting socorro-processor: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_27_44 Upgrading socorro on sp-processor03.phx1.mozilla.com Stopping socorro-processor: [ OK ] Starting socorro-processor: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_27_49 Upgrading socorro on sp-processor04.phx1.mozilla.com Stopping socorro-processor: [ OK ] Starting socorro-processor: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_27_55 Upgrading socorro on sp-processor05.phx1.mozilla.com Stopping socorro-processor: [ OK ] Starting socorro-processor: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_28_02 Upgrading socorro on sp-processor06.phx1.mozilla.com Stopping socorro-processor: [ OK ] Starting socorro-processor: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_28_09 Upgrading socorro on sp-processor07.phx1.mozilla.com Stopping socorro-processor: [ OK ] Starting socorro-processor: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_28_14 Upgrading socorro on sp-processor08.phx1.mozilla.com Stopping socorro-processor: [ OK ] Starting socorro-processor: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_28_17 Upgrading socorro on sp-processor09.phx1.mozilla.com Stopping socorro-processor: [ OK ] Starting socorro-processor: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_28_24 Upgrading socorro on sp-processor10.phx1.mozilla.com Stopping socorro-processor: [ OK ] Starting socorro-processor: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_28_29 Upgrading socorro on sp-web01.phx1.mozilla.com Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_28_36 Upgrading socorro on sp-web02.phx1.mozilla.com Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_28_54 Upgrading socorro on sp-web03.phx1.mozilla.com Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_29_01 Upgrading socorro on sp-web04.phx1.mozilla.com Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_29_06 Upgrading socorro on sp-web05.phx1.mozilla.com Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_29_24 update-socorro-all.sh https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Upgrading socorro to https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Upgrading socorro on tp-socorro01-master01.phx1.mozilla.com Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_56_28 Upgrading socorro on tp-socorro01-master02.phx1.mozilla.com Socorro build installed successfully! Downloaded from https://ci.mozilla.org/job/socorro-release/58/artifact/socorro.tar.gz Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_56_30 [root@sp-admin01.phx1 ~]# /data/bin/update-socorro.sh Stopping socorro-monitor: [FAILED] Starting socorro-monitor: [ OK ] Stopping httpd: [ OK ] Starting httpd: [ OK ] Socorro build installed successfully! Downloaded from Checksum: c04ea6030793957744cb292c2a935813 Backed up original to /data/socorro.15-12-2011_09_59_47 [sp-web01.phx1.mozilla.com] running: service memcached restart [sp-web02.phx1.mozilla.com] running: service memcached restart [sp-web03.phx1.mozilla.com] running: service memcached restart [sp-web04.phx1.mozilla.com] running: service memcached restart [sp-web05.phx1.mozilla.com] running: service memcached restart [sp-web05.phx1.mozilla.com] finished: service memcached restart (0.884s) [sp-web05.phx1.mozilla.com] out: Stopping memcached: [ OK ] [sp-web05.phx1.mozilla.com] out: Starting memcached: [ OK ] [sp-web04.phx1.mozilla.com] finished: service memcached restart (1.200s) [sp-web04.phx1.mozilla.com] out: Stopping memcached: [ OK ] [sp-web04.phx1.mozilla.com] out: Starting memcached: [ OK ] [sp-web02.phx1.mozilla.com] finished: service memcached restart (1.202s) [sp-web02.phx1.mozilla.com] out: Stopping memcached: [ OK ] [sp-web02.phx1.mozilla.com] out: Starting memcached: [ OK ] [sp-web01.phx1.mozilla.com] finished: service memcached restart (1.203s) [sp-web01.phx1.mozilla.com] out: Stopping memcached: [ OK ] [sp-web01.phx1.mozilla.com] out: Starting memcached: [ OK ] [sp-web03.phx1.mozilla.com] finished: service memcached restart (1.202s) [sp-web03.phx1.mozilla.com] out: Stopping memcached: [ OK ] [sp-web03.phx1.mozilla.com] out: Starting memcached: [ OK ]
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
QA verified. Automation passed expect for a test that looks for crashstats data for 9.0b6 which is expected. The cronjob won't run until 2am tomorrow to populate that data. ffts pass. The app does return intermittent 500's but this is due to an outage at PHX: [10:30:29.324] GET https://crash-stats.mozilla.com/query?advanced=1 [HTTP/1.1 500 Internal Server Error 2012ms] 2011-12-15 10:30:30 -08:00 --- error: [5xx Error] File: application/libraries/drivers/Database/Pgsql.php; Line: 48; Message: pg_connect(): Unable to connect to PostgreSQL server: could not connect to server: Connection refused nagios-phx1: [161] sp-admin01.phx1:File Age - /var/log/socorro/cron_submitdumpstostaging.log is CRITICAL: Connection refused or timed out nagios-phx1: [176] sp-admin01.phx1:hplog is CRITICAL: CRITICAL 0002: Uncorrectable Machine Check Exception (Board 0, Processor 1, APIC ID 0x00000011, Bank 0x00000005, Status 0xBE00000000800400, Address 0x00003FFFA02872E0, Misc 0x0000000000007FFF)
Status: RESOLVED → VERIFIED
Component: Server Operations: Web Operations → WebOps: Other
Product: mozilla.org → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.