Closed Bug 1265037 Opened 6 years ago Closed 6 years ago

Add an intermediate "job" table/model to the master db


(Tree Management :: Treeherder, defect)

Not set


(Not tracked)



(Reporter: wlach, Assigned: wlach)




(2 files)

To facilitate moving more project-specific artifacts to the master database, it would be useful to create a table/model corresponding to the job. This way we could track guid, id, in one place, and look it up easily. Eventually, this table could be fleshed out into a full-fledged replacement for the project-specific jobs table.
Comment on attachment 8741901 [details] [review]
[treeherder] wlach:1265037 > mozilla:master

Hi James, I think you'd be a good person to take a look at this. This isn't very useful by itself, but I think it should make the eventual migration away from datasource easier, while we incrementally migrate away from it with things like bug 1218537.
Attachment #8741901 - Flags: review?(james)
Blocks: 1265503
Blocks: 1266062
Attachment #8741901 - Flags: review?(james) → review+
Commits pushed to master at
Bug 1265037 - Create an intermediate jobs model/table

Not really useful in-and-of-itself, but should be helpful by acting
as a centralized place to relate and map job information from what
is stored in the main treeherder database to what's in the per-project
databases. Eventually, we can extend this into a full-fledged jobs model.
Bug 1265037 - Temporarily disable creation of intermediate job entries

We will wait to turn this on until the initial migration is complete to
ensure jobs are more-or-less in chronological order
Merge pull request #1411 from wlach/1265037

Bug 1265037 - Create an intermediate jobs model/table
Commits pushed to master at
Revert "Bug 1265037 - Temporarily disable creation of intermediate job entries"

This reverts commit 081d86bd7b699af7128841c266939464cdd795c9.
Bug 1265037 - Adapt migration script to assume most jobs migrated
Merge pull request #1425 from wlach/1265037-2

Bug 1265037 - Enable dynamic intermediate job ingestion
Commit pushed to master at
Bug 1265037 - Fix edge case in migration script

If ingestion was ongoing during the migration, it was possible for things
to error out since operations were batched inside a transaction. Since
it should be rare to need to insert new items when this script is being
run, just skip the transaction.
The migration was horrible and required hand-hacking, but I believe this is now done.
Closed: 6 years ago
Resolution: --- → FIXED
Depends on: 1272532
Depends on: 1274191
You need to log in before you can comment on or make changes to this bug.