All users were logged out of Bugzilla on October 13th, 2018

Status

RESOLVED FIXED
6 years ago
5 years ago

People

(Reporter: rhelmer, Assigned: bburton)

Tracking

Details

(URL)

Attachments

(1 attachment)

(Reporter)

Description

6 years ago
This release is to swap out a component due to a regression. We'll need to add the following to the crontab when the release is out:


# run duplicate detection
02 * * * * socorro /data/socorro/application/scripts/crons/cron_duplicates.sh
# create "clean" version of incoming reports, only containing valid crashes
# reportsclean must run after duplicates, see bug 697772
45 * * * * socorro /data/socorro/application/scripts/crons/cron_reportsclean.sh
(Assignee)

Updated

6 years ago
Assignee: server-ops-webops → bburton
(Reporter)

Comment 1

6 years ago
Created attachment 647665 [details]
17.4 crontab

Actually need a few more crontab changes, disable crontabber and also add the bugzilla cron back.
(Assignee)

Comment 2

6 years ago
push done

[root@sp-admin01.phx1 ~]# /root/bin/update-socorro-all.sh  https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Upgrading socorro to https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Upgrading socorro on sp-collector01.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_43_37
Upgrading socorro on sp-collector02.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_43_42
Upgrading socorro on sp-collector03.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_43_46
Upgrading socorro on sp-collector04.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_43_52
Upgrading socorro on sp-collector05.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_43_56
Upgrading socorro on sp-collector06.phx1.mozilla.com
Stopping socorro-crashmover: [  OK  ]
Starting socorro-crashmover: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_44_01
Upgrading socorro on sp-mware01.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_44_10
Upgrading socorro on sp-mware02.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_44_27
Upgrading socorro on sp-mware03.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_44_40
Upgrading socorro on sp-processor01.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_44_44
Upgrading socorro on sp-processor02.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_44_48
Upgrading socorro on sp-processor03.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_44_52
Upgrading socorro on sp-processor04.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_45_06
Upgrading socorro on sp-processor05.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_45_09
Upgrading socorro on sp-processor06.phx1.mozilla.com
Stopping socorro-processor: [FAILED]
Starting socorro-processor: Can't lock the lock file "/var/lock/socorro-processor.lock". Is another instance running?
[FAILED]
ERROR: could not start socorro-processor
Could not upgrade socorro on sp-processor06.phx1.mozilla.com, exit_code: 1
[root@sp-admin01.phx1 ~]# /root/bin/update-socorro-all.sh  https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Upgrading socorro to https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Upgrading socorro on sp-collector01.phx1.mozilla.com
Upgrading socorro on sp-collector02.phx1.mozilla.com
Upgrading socorro on sp-collector03.phx1.mozilla.com
Upgrading socorro on sp-collector04.phx1.mozilla.com
Upgrading socorro on sp-collector05.phx1.mozilla.com
Upgrading socorro on sp-collector06.phx1.mozilla.com
Upgrading socorro on sp-mware01.phx1.mozilla.com
Upgrading socorro on sp-mware02.phx1.mozilla.com
Upgrading socorro on sp-mware03.phx1.mozilla.com
Upgrading socorro on sp-processor01.phx1.mozilla.com
Upgrading socorro on sp-processor02.phx1.mozilla.com
Upgrading socorro on sp-processor03.phx1.mozilla.com
Upgrading socorro on sp-processor04.phx1.mozilla.com
Upgrading socorro on sp-processor05.phx1.mozilla.com
Upgrading socorro on sp-processor06.phx1.mozilla.com
Upgrading socorro on sp-processor07.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_46_51
Upgrading socorro on sp-processor08.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_46_55
Upgrading socorro on sp-processor09.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_46_59
Upgrading socorro on sp-processor10.phx1.mozilla.com
Stopping socorro-processor: [  OK  ]
Starting socorro-processor: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_47_03
Upgrading socorro on sp-web01.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_47_17
Upgrading socorro on sp-web02.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_47_18
Upgrading socorro on sp-web03.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_47_21
Upgrading socorro on sp-web04.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_47_22
Upgrading socorro on sp-web05.phx1.mozilla.com
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_47_24
Upgrading socorro on sp-admin01.phx1.mozilla.com
Stopping socorro-monitor: [  OK  ]
Starting socorro-monitor: [  OK  ]
Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_47_26
Upgrading socorro on tp-socorro01-master01.phx1.mozilla.com
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_47_40
Upgrading socorro on tp-socorro01-master02.phx1.mozilla.com
Socorro build installed successfully!
Downloaded from https://ci.mozilla.org/job/socorro-release/210/artifact/socorro.tar.gz
Checksum: 57e7e18e0a01b3e203356abda7137029
Backed up original to /data/socorro.31-07-2012_15_47_42
[sp-web01.phx1.mozilla.com] running: service memcached restart
[sp-web02.phx1.mozilla.com] running: service memcached restart
[sp-web03.phx1.mozilla.com] running: service memcached restart
[sp-web04.phx1.mozilla.com] running: service memcached restart
[sp-web05.phx1.mozilla.com] running: service memcached restart
[sp-web01.phx1.mozilla.com] finished: service memcached restart (0.364s)
[sp-web01.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web01.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web02.phx1.mozilla.com] finished: service memcached restart (0.555s)
[sp-web02.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web02.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web05.phx1.mozilla.com] finished: service memcached restart (0.552s)
[sp-web05.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web05.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web03.phx1.mozilla.com] finished: service memcached restart (0.594s)
[sp-web03.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web03.phx1.mozilla.com] out: Starting memcached: [  OK  ]
[sp-web04.phx1.mozilla.com] finished: service memcached restart (0.594s)
[sp-web04.phx1.mozilla.com] out: Stopping memcached: [  OK  ]
[sp-web04.phx1.mozilla.com] out: Starting memcached: [  OK  ]
(Assignee)

Comment 3

6 years ago
Prod cron updated

[bburton@sp-admin01.phx1 ~]$ cat /etc/cron.d/socorro 
MAILTO="cron-socorro@mozilla.com"

#*/5 * * * * socorro /data/socorro/application/scripts/crons/crontabber.sh

*/5 * * * * socorro /data/socorro/application/scripts/crons/cron_status.sh
02 * * * * socorro /data/socorro/application/scripts/crons/cron_aggregates.sh

05 * * * * socorro /data/socorro/application/scripts/crons/cron_ftpscraper.sh
00 17 * * * socorro /data/socorro/application/scripts/crons/cron_modulelist.sh
05 21 * * 2 socorro /data/socorro/application/scripts/crons/cron_create_partitions.sh

*/3 * * * * socorro /data/socorro/application/scripts/crons/cron_submitter.sh crash-reports.allizom.org 1000 > /dev/null 2>&1
05 00 * * * socorro /data/socorro/application/scripts/crons/cron_libraries.sh > /var/log/socorro/cron_libraries.log 2>&1
55 03 * * * socorro /data/bin/cron_daily_reports.sh
05 * * * * socorro /data/socorro/application/scripts/crons/cron_update_adus.sh

# generate nightly reports for crash-stats UI
00 02 * * * socorro /data/socorro/application/scripts/crons/cron_daily_matviews.sh
# annotate crashes with info from bugzilla
11 * * * * socorro /data/socorro/application/scripts/crons/cron_bugzilla.sh
# run duplicate detection
02 * * * * socorro /data/socorro/application/scripts/crons/cron_duplicates.sh
# create "clean" version of incoming reports, only containing valid crashes
# reportsclean must run after duplicates, see bug 697772
45 * * * * socorro /data/socorro/application/scripts/crons/cron_reportsclean.sh
(Assignee)

Comment 4

6 years ago
Stage cron updated

[bburton@socorroadm.stage.private.phx1 ~]$ cat /etc/cron.d/socorro 
MAILTO="cron-socorro@mozilla.com"

#*/5 * * * * socorro /data/socorro/application/scripts/crons/crontabber.sh

*/5 * * * * socorro /data/socorro/application/scripts/crons/cron_status.sh
02 * * * * socorro /data/socorro/application/scripts/crons/cron_aggregates.sh

05 * * * * socorro /data/socorro/application/scripts/crons/cron_ftpscraper.sh
00 17 * * * socorro /data/socorro/application/scripts/crons/cron_modulelist.sh
05 21 * * 2 socorro /data/socorro/application/scripts/crons/cron_create_partitions.sh

*/3 * * * * socorro /data/socorro/application/scripts/crons/cron_submitter.sh crash-reports.allizom.org 1000 > /dev/null 2>&1
05 00 * * * socorro /data/socorro/application/scripts/crons/cron_libraries.sh > /var/log/socorro/cron_libraries.log 2>&1
55 03 * * * socorro /data/bin/cron_daily_reports.sh
05 * * * * socorro /data/socorro/application/scripts/crons/cron_update_adus.sh

# generate nightly reports for crash-stats UI
00 02 * * * socorro /data/socorro/application/scripts/crons/cron_daily_matviews.sh
# annotate crashes with info from bugzilla
11 * * * * socorro /data/socorro/application/scripts/crons/cron_bugzilla.sh
# run duplicate detection
02 * * * * socorro /data/socorro/application/scripts/crons/cron_duplicates.sh
# create "clean" version of incoming reports, only containing valid crashes
# reportsclean must run after duplicates, see bug 697772
45 * * * * socorro /data/socorro/application/scripts/crons/cron_reportsclean.sh
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Component: Server Operations: Web Operations → WebOps: Other
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.