1306674 - Store jobs and push data inside central database

Status: RESOLVED FIXED

(Tree Management :: Treeherder, defect)

Description

[William Lachance (:wlach)]

The last step of the django orm migration is to move the job and result set data into the central db.

Tentative plan:

1. Add/extend database models to cover result sets (let's rename to "pushes") and job metadata (we already have a table for jobs, we just need to extend it).
2. Refactor jobs model to ingest jobs and result sets serially
3. Start ingesting job and result set metadata in central db side-by-side with old data.
4. Switch API endpoints to return data from the central db instead of datasource
5. Turn off datasource ingestion

My expectation would be to deploy 3, 4, and 5 seperately though probably in fairly rapid succession (just need to be sure to give myself enough time to verify that things are working after each step)

Comment 1

[Armen [:armenzg]]

What is the datasource ingestion? Is that not the logic to receive new BB and TC jobs?

Comment 2

[William Lachance (:wlach)]

(In reply to BACK ON TUESDAY - Armen Zambrano [:armenzg] (EDT/UTC-4) from comment #1)
> What is the datasource ingestion? Is that not the logic to receive new BB
> and TC jobs?

I was referring to the logic to put TC and BB job data into the per-project (as opposed to global) databases.

Comment 3

[William Lachance (:wlach)]

This became a meta-bug, which we can now close.