Closed
Bug 1478080
Opened 6 years ago
Closed 6 years ago
vendor and fork crontabber
Categories
(Socorro :: General, task, P2)
Socorro
General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: willkg, Assigned: willkg)
References
Details
Attachments
(1 file)
Socorro engineering wrote a library called crontabber which manages coordinating and running scheduled and interdependent tasks. It has a variety of features some of which we still use.
This library is (probably) used by other people:
https://libraries.io/pypi/crontabber
We're in the process of simplifying our postgres situation, upgrading to Python 3, and simplifying Socorro in general. It will greatly minimize the work I need to do if we take this moment to vendor crontabber effectively forking it and then rewriting it in place in our repo.
This bug covers that.
Assignee | ||
Comment 1•6 years ago
|
||
Relatedly, if we can swing it, it'd be super to rewrite it such that all the bookkeeping is done in Django-land.
Making this a P2.
Priority: -- → P2
Assignee | ||
Comment 2•6 years ago
|
||
I want to do this in two stages:
Stage 1: vendor the existing crontabber and its tests and make sure that all works
Stage 2: (in a different bug) "rewrite" crontabber using Django models for bookkeeping, a new Job base class, and a Django management command to run things
This bug covers stage 1. Grabbing this to do soon because it'll make the switch to Python 3 easier and the switch from sqlalchemy to Django easier, too.
Assignee: nobody → willkg
Status: NEW → ASSIGNED
Assignee | ||
Comment 3•6 years ago
|
||
Assignee | ||
Comment 4•6 years ago
|
||
PR 4605 implements stage 1--vendoring the necessary bits of crontabber. It updates the code a smidge, removes some bits we don't use, reworks some of the code to use Socorro versions of things (TransactionExecutor, dbapi2_util, App, etc), and fixes some Python 3 issues.
It's not super great, but it's a step in the right direction and reduces future work. So there's that.
After this lands, I'll write up a bug for rewriting crontabber using Django models.
Comment 5•6 years ago
|
||
Commits pushed to master at https://github.com/mozilla-services/socorro
https://github.com/mozilla-services/socorro/commit/f4b26d0a485f7e6ec0306f82cd27441e7ecafea4
fix bug 1478080: vendor crontabber
This brings the necessary bits of crontabber back into Socorro and updates
it to pass linting and work with Python 3.
This doesn't bring in the tests since those are written for nose and there's
a lot of it and we really just want to test Socorro crontabber jobs--not an
entire library. If it turns out that was a bad decision, we can add (and
overhaul) them later.
https://github.com/mozilla-services/socorro/commit/1ef13d1a3d7b3a99feb7cd505d414bce194a23d0
Merge pull request #4605 from willkg/1478080-crontabber
fix bug 1478080: vendor crontabber
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•