Closed Bug 1478080 Opened 4 years ago Closed 4 years ago

vendor and fork crontabber

Categories

(Socorro :: General, task, P2)

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: willkg, Assigned: willkg)

References

Details

Attachments

(1 file)

Socorro engineering wrote a library called crontabber which manages coordinating and running scheduled and interdependent tasks. It has a variety of features some of which we still use.

This library is (probably) used by other people:

https://libraries.io/pypi/crontabber

We're in the process of simplifying our postgres situation, upgrading to Python 3, and simplifying Socorro in general. It will greatly minimize the work I need to do if we take this moment to vendor crontabber effectively forking it and then rewriting it in place in our repo.

This bug covers that.
Relatedly, if we can swing it, it'd be super to rewrite it such that all the bookkeeping is done in Django-land.

Making this a P2.
Priority: -- → P2
I want to do this in two stages:

Stage 1: vendor the existing crontabber and its tests and make sure that all works

Stage 2: (in a different bug) "rewrite" crontabber using Django models for bookkeeping, a new Job base class, and a Django management command to run things


This bug covers stage 1. Grabbing this to do soon because it'll make the switch to Python 3 easier and the switch from sqlalchemy to Django easier, too.
Assignee: nobody → willkg
Status: NEW → ASSIGNED
PR 4605 implements stage 1--vendoring the necessary bits of crontabber. It updates the code a smidge, removes some bits we don't use, reworks some of the code to use Socorro versions of things (TransactionExecutor, dbapi2_util, App, etc), and fixes some Python 3 issues.

It's not super great, but it's a step in the right direction and reduces future work. So there's that.

After this lands, I'll write up a bug for rewriting crontabber using Django models.
Commits pushed to master at https://github.com/mozilla-services/socorro

https://github.com/mozilla-services/socorro/commit/f4b26d0a485f7e6ec0306f82cd27441e7ecafea4
fix bug 1478080: vendor crontabber

This brings the necessary bits of crontabber back into Socorro and updates
it to pass linting and work with Python 3.

This doesn't bring in the tests since those are written for nose and there's
a lot of it and we really just want to test Socorro crontabber jobs--not an
entire library. If it turns out that was a bad decision, we can add (and
overhaul) them later.

https://github.com/mozilla-services/socorro/commit/1ef13d1a3d7b3a99feb7cd505d414bce194a23d0
Merge pull request #4605 from willkg/1478080-crontabber

fix bug 1478080: vendor crontabber
Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.