745731 - Daily crash data CSVs missing for April 14th and 15th

Status: VERIFIED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Robert Kaiser]

https://crash-analysis.mozilla.com/crash_analysis/20120414/ doesn't contain a daily crash data CSV (*-pub-crashdata.csv.gz), and https://crash-analysis.mozilla.com/crash_analysis/20120415/ doesn't even exist.

I guess the Socorro outage this weekend affected the cron jobs and those CSVs weren't created correctly.

Comment 1

[Robert Kaiser]

Marcia, Sheila: FYI, this bug is why my reports (and probably chofmann's) don't exist for those days, we'll probably need to re-run our scripts for those days when this bug is fixed.

Comment 2

[Robert Helmer [:rhelmer]]

This script needs to be copied and modified, and run as "socorro" user on sp-admin01:

/data/bin/cron_daily_reports.sh

There are 4 spots in this script where this is used:
date -d "1 days ago"

This needs to be run for the 14th and the 15th, probably easiest to just copy and modify the script to run "2 days ago" then "3 days ago".

I'll file a bug to pull this script into socorro and clean it up, make date overrideable as an arg etc.

Comment 3

[Robert Kaiser]

(In reply to Robert Helmer [:rhelmer] from comment #2)
> This needs to be run for the 14th and the 15th, probably easiest to just
> copy and modify the script to run "2 days ago" then "3 days ago".

It's just "2 days ago" in addition to a re-run for "1 days ago", right? The 15th is yesterday. :)

Comment 4

[Robert Helmer [:rhelmer]]

(In reply to Robert Kaiser (:[e-mail]) from comment #3)
> (In reply to Robert Helmer [:rhelmer] from comment #2)
> > This needs to be run for the 14th and the 15th, probably easiest to just
> > copy and modify the script to run "2 days ago" then "3 days ago".
> 
> It's just "2 days ago" in addition to a re-run for "1 days ago", right? The
> 15th is yesterday. :)

Yes that's right, thanks :)

Comment 5

[Robert Helmer [:rhelmer]]

Filed bug 745830 to make this simpler in the future.

Comment 6

[Dumitru Gherman [:dumitru]]

https://crash-analysis.mozilla.com/crash_analysis/20120415/20120415-pub-crashdata.csv.gz
https://crash-analysis.mozilla.com/crash_analysis/20120414/20120414-pub-crashdata.csv.gz

Comment 7

[Robert Kaiser]

It looks like permissions for those files are off:

gzip: /mnt/crashanalysis/crash_analysis/20120414/20120414-pub-crashdata.csv.gz: Permission denied
gzip: /mnt/crashanalysis/crash_analysis/20120415/20120415-pub-crashdata.csv.gz: Permission denied

That's accessing them as rkaiser. The file from the 16th seems to be accessible though.

Comment 8

[Dumitru Gherman [:dumitru]]

[[e-mail]1 crash_analysis]# chmod 644 20120414/20120414-pub-crashdata.csv.gz
[[e-mail]1 crash_analysis]# chmod 644 20120415/20120415-pub-crashdata.csv.gz

Comment 9

[Robert Kaiser]

Verified, my report scripts can work with the data now! :)