Run FTP-scraper several times a day

VERIFIED FIXED in 2.2.2

Status

Socorro
General
VERIFIED FIXED
7 years ago
7 years ago

People

(Reporter: jberkus, Assigned: rhelmer)

Tracking

2.2.2
x86
Mac OS X

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

7 years ago
Currently the FTP scraper only runs once a day, at 7am.  This leaves open the possibility that a version might be released after that and not included in the nightly roll-ups.

Please run the FTP-scraper at least once more every day, just before running all of the daily rollups.
(Assignee)

Comment 1

7 years ago
OK so should we leave the current cron entry in place (7:05) and add a new one, maybe 30 minutes before the scheduled run?

This script normally takes well under 1 minute to run so it should not be a problem (as we move to the cronjobs table we can be a little more serious about dependency if we want).
Status: NEW → ASSIGNED
(Reporter)

Comment 2

7 years ago
Robert,

Yes.

Whether you want to run it *more* than twice a day is up to you.  I just need it the two times.
(Assignee)

Updated

7 years ago
Target Milestone: --- → 2.2.2
(Assignee)

Comment 3

7 years ago
Just checked this into staging, here's the new schedule:
00 02 * * * socorro /data/socorro/application/scripts/crons/cron_newtcbs.sh
30 01 * * * socorro /data/socorro/application/scripts/crons/cron_builds.sh
05 07 * * * socorro /data/socorro/application/scripts/crons/cron_builds.sh

This could go out ahead of 2.2.2 FWIW, not on prod yet though.
Status: ASSIGNED → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED
Hiya rhelmer ... we've been playing a bit of phone tag on irc today. Perchance can you post the log results showing this is running and I'll bump to verified.
(Assignee)

Comment 5

7 years ago
Here's a quick grep that shows i'ts running at both 7:05 and 1:30:

[rhelmer@admin1.socorro.stage.private.sjc1 socorro]$ grep 'current configuration' cron_builds.log
2011-08-28 07:05:01,223 INFO - current configuration:
2011-08-29 01:30:01,907 INFO - current configuration:
2011-08-29 07:05:01,520 INFO - current configuration:

I also inspected the log and looks normal (still getting spurious errors in the old/nightly case, but that's a totally different matter. I'll file a bug on that).
(In reply to Robert Helmer [:rhelmer] from comment #5)
> Here's a quick grep that shows i'ts running at both 7:05 and 1:30:
> 
> [rhelmer@admin1.socorro.stage.private.sjc1 socorro]$ grep 'current
> configuration' cron_builds.log
> 2011-08-28 07:05:01,223 INFO - current configuration:
> 2011-08-29 01:30:01,907 INFO - current configuration:
> 2011-08-29 07:05:01,520 INFO - current configuration:

QA verified. The crons running look right.
Status: RESOLVED → VERIFIED
Component: Socorro → General
Product: Webtools → Socorro
You need to log in before you can comment on or make changes to this bug.