822828 - [socorro stage] enable cron_fixbrokendumps.sh

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task, P1)

Description

[Robert Helmer [:rhelmer]]

Attachment: minidump elfhack fixer for b2g

This needs a set of changes on the staging admin box, please add ASAP as we need to ship tomorrow:

Please add this to /etc/cron.d/socorro running hourly:

/data/socorro/application/scripts/crons/cron_fixbrokendumps.sh

Remove the old state file:
~socorro/persistent/fixbrokendumps.pickle

Add this to /etc/socorro/common.conf:

export brokenBoot2GeckoFixer=/data/bin/minidump_hack-b2g

Finally, put the attached binary in /data/bin/ and make sure it's executable.

Comment 1

[Brandon Burton [:solarce]]

Binary installed

[[e-mail]1 bin]# ls -lah /data/bin/minidump_hack-b2g 
-rwxr-xr-x 1 root root 34K Dec 18 14:31 /data/bin/minidump_hack-b2g

Comment 2

[Brandon Burton [:solarce]]

Pushing changes to svn

bburton@voltaire [02:33:36] [~/code/mozilla/sysadmins/puppet/trunk/modules/webapp/files/socorro-stage] 
-> % svn diff
Index: etc-socorro/common.conf
===================================================================
--- etc-socorro/common.conf	(revision 54642)
+++ etc-socorro/common.conf	(working copy)
@@ -49,3 +49,5 @@
 # bug 790097
 export statsdHost=graphite1.dmz.phx1.mozilla.com
 export statsdPrefix=socorro-stage
+# bug 822828
+export brokenBoot2GeckoFixer=/data/bin/minidump_hack-b2g
Index: admin/etc-cron.d/socorro
===================================================================
--- admin/etc-cron.d/socorro	(revision 54642)
+++ admin/etc-cron.d/socorro	(working copy)
@@ -17,3 +17,5 @@
 
 # STAGE ONLY - submit crashes to dev
 */3 * * * * socorro /data/socorro/application/scripts/crons/cron_submitter.sh crash-reports-dev.allizom.org 1000 > /dev/null 2>&1
+
+0 * * * * * socorro /data/socorro/application/scripts/crons/cron_fixbrokendumps.sh

bburton@voltaire [02:33:54] [~/code/mozilla/sysadmins/puppet/trunk/modules/webapp/files/socorro-stage] 
-> % svn ci -m "adding b2g fix to stage, bug 822828"
Sending        socorro-stage/admin/etc-cron.d/socorro
Sending        socorro-stage/etc-socorro/common.conf
Transmitting file data ..
Committed revision 54718.

Comment 3

[Brandon Burton [:solarce]]

Puppet changes applied successfully

notice: /File[/etc/cron.d/socorro]/content: 
--- /etc/cron.d/socorro	2012-12-14 17:25:47.000000000 -0800
+++ /tmp/puppet-file20121218-17416-9swj6w-0	2012-12-18 14:45:23.000000000 -0800
@@ -17,3 +17,5 @@
 
 # STAGE ONLY - submit crashes to dev
 */3 * * * * socorro /data/socorro/application/scripts/crons/cron_submitter.sh crash-reports-dev.allizom.org 1000 > /dev/null 2>&1
+
+0 * * * * * socorro /data/socorro/application/scripts/crons/cron_fixbrokendumps.sh

info: FileBucket adding {md5}296ec120eca760412d1aaba19ebc58f5
info: /File[/etc/cron.d/socorro]: Filebucketed /etc/cron.d/socorro to main with sum 296ec120eca760412d1aaba19ebc58f5
notice: /File[/etc/cron.d/socorro]/content: content changed '{md5}296ec120eca760412d1aaba19ebc58f5' to '{md5}2107a21cc07760bb219d8e5a8897881c'
notice: /Stage[main]/Webapp::Socorro/File[/etc/socorro/common.conf]/content: 
--- /etc/socorro/common.conf	2012-11-07 07:37:22.000000000 -0800
+++ /tmp/puppet-file20121218-17416-7b9fxn-0	2012-12-18 14:45:25.000000000 -0800
@@ -49,3 +49,5 @@
 # bug 790097
 export statsdHost=graphite1.dmz.phx1.mozilla.com
 export statsdPrefix=socorro-stage
+# bug 822828
+export brokenBoot2GeckoFixer=/data/bin/minidump_hack-b2g

info: FileBucket adding {md5}2f472f468faa6ed8f095b609a4619fa6
info: /Stage[main]/Webapp::Socorro/File[/etc/socorro/common.conf]: Filebucketed /etc/socorro/common.conf to main with sum 2f472f468faa6ed8f095b609a4619fa6
notice: /Stage[main]/Webapp::Socorro/File[/etc/socorro/common.conf]/content: content changed '{md5}2f472f468faa6ed8f095b609a4619fa6' to '{md5}defb965cd90f8874085de0b0e1b3d9c8'
info: /Stage[main]/Webapp::Socorro/File[/etc/socorro/common.conf]: Scheduling refresh of Service[socorro-monitor]
notice: /Stage[main]/Hadoop::Hadoop_client/Exec[install -m 775 -o hdfs -g hadoop -d /var/log/hadoop]/returns: executed successfully
notice: /Stage[main]/Webapp::Socorro::Monitor/Service[socorro-monitor]: Triggered 'refresh' from 1 events
notice: Finished catalog run in 86.15 seconds

Script ran manually and with no errors

[[e-mail]1 bin]# su - socorro
[[e-mail]1 ~]$ /data/socorro/application/scripts/crons/cron_fixbrokendumps.sh
[[e-mail]1 ~]$ echo $?
0