If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

ensure that cron jobs email on error

RESOLVED FIXED in 1.7.7

Status

Socorro
General
RESOLVED FIXED
7 years ago
6 years ago

People

(Reporter: rhelmer, Assigned: rhelmer)

Tracking

Trunk
1.7.7
x86_64
All

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Assignee)

Description

7 years ago
Created attachment 527058 [details] [diff] [review]
if pyjob exits non-zero, print info to stdout so crond will email

It's come to my attention that cron jobs aren't always sending us email on error. We are setting the exit code non-zero, but in many cases there is no output.

We are currently redirecting both stderr and stdout to a log file; what we really want is to ensure that the python cron jobs are using stderr appropriately and only log stdout (although this makes reading the logs somewhat painful, we might want to look into putting it in both places).

This will take a little while to test, so I propose for 1.7.7 we print a generic "Cron job (scriptname) exited non-zero: (exit code)"
(Assignee)

Comment 1

7 years ago
Got jabba to take a look, and tested using my account on stage:

Committed revision 3069.
(Assignee)

Updated

7 years ago
Assignee: nobody → rhelmer
(Assignee)

Comment 2

7 years ago
Just got an email for a cron job known to be failing (bug 629054):

Subject: Cron <socorro@admin1> /data/socorro/application/scripts/crons/cron_signatures.sh
Cron job startSignatures exited non-zero: 1
Status: NEW → ASSIGNED
(Assignee)

Updated

7 years ago
Status: ASSIGNED → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED
(Assignee)

Comment 3

7 years ago
(In reply to comment #0)
> We are currently redirecting both stderr and stdout to a log file; what we
> really want is to ensure that the python cron jobs are using stderr
> appropriately and only log stdout (although this makes reading the logs
> somewhat painful, we might want to look into putting it in both places).
> 
> This will take a little while to test, so I propose for 1.7.7 we print a
> generic "Cron job (scriptname) exited non-zero: (exit code)"

Filed bug 651201 to follow up on the above.
(Assignee)

Comment 4

7 years ago
(In reply to comment #3)
> (In reply to comment #0)
> > We are currently redirecting both stderr and stdout to a log file; what we
> > really want is to ensure that the python cron jobs are using stderr
> > appropriately and only log stdout (although this makes reading the logs
> > somewhat painful, we might want to look into putting it in both places).
> > 
> > This will take a little while to test, so I propose for 1.7.7 we print a
> > generic "Cron job (scriptname) exited non-zero: (exit code)"
> 
> Filed bug 651201 to follow up on the above.

Sorry meant bug 651652 here.
Component: Socorro → General
Product: Webtools → Socorro
You need to log in before you can comment on or make changes to this bug.