Closed Bug 819403 Opened 12 years ago Closed 12 years ago

on sp-admin01: Please kill off hung cron jobs and remove lock files

Categories

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Product:
Infrastructure & Operations Graveyard
Component:
WebOps: Other
Platform:
x86_64
Linux
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: selenamarie, Unassigned)

References

Details

These processes are hung and need to be killed, and any related lockfiles removed from /tmp and /var/tmp: 

socorro  10707 10702  0 Dec06 ?        00:00:00 /bin/bash /data/socorro/application/scripts/crons/cron_update_adus.sh
socorro  11454 11453  0 Dec06 ?        00:00:00 /bin/sh -c /data/socorro/application/scripts/crons/cron_submitter.sh crash-reports.allizom.org 1000 > /dev/null 2>&1
socorro  11455 11454  0 Dec06 ?        00:00:00 /bin/sh -c /data/socorro/application/scripts/crons/cron_submitter.sh crash-reports.allizom.org 1000 > /dev/null 2>&1
socorro  11456 11455  0 Dec06 ?        00:00:01 /bin/sh -c /data/socorro/application/scripts/crons/cron_submitter.sh crash-reports.allizom.org 1000 > /dev/null 2>&1
socorro  30226 30222  0 00:05 ?        00:00:00 /bin/sh -c /data/socorro/application/scripts/crons/cron_libraries.sh > /var/log/socorro/cron_libraries.log 2>&1
socorro  30228 30226  0 00:05 ?        00:00:00 /bin/sh /data/socorro/application/scripts/crons/cron_libraries.sh
Depends on: 817718
[root@sp-admin01.phx1 ~]# kill -9 10707
[root@sp-admin01.phx1 ~]# kill -9 11454
[root@sp-admin01.phx1 ~]# kill -9 11455
[root@sp-admin01.phx1 ~]# kill -9 11456
[root@sp-admin01.phx1 ~]# kill -9 30226
[root@sp-admin01.phx1 ~]# kill -9 30228

[root@sp-admin01.phx1 tmp]# ls -lah *.lock
-rw-r--r-- 1 socorro     socorro     6 Dec  6 17:05 cron_update_adus.lock
-rw-r----- 1 sdeckelmann sdeckelmann 0 Dec  3 16:27 selena-crash-reports.allizom.org.lock


[root@sp-admin01.phx1 tmp]# rm -rf cron_update_adus.lock
[root@sp-admin01.phx1 tmp]# cd /var/tmp
[root@sp-admin01.phx1 tmp]# ls *.lock
crash-reports.allizom.org.lock
[root@sp-admin01.phx1 tmp]# rm -rf crash-reports.allizom.org.lock 
[root@sp-admin01.phx1 tmp]# 

Killed and cleaned up, Dan will review all Zeus settings for prod socorro VIPs for any possible regression and will look at the cron scripts to see what better error output we could add to get more info on what is killing these jobs
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Component: Server Operations: Web Operations → WebOps: Other
Product: mozilla.org → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.