1265503 - Reference new intermediate jobs table from performance datums

Status: RESOLVED FIXED

(Tree Management :: Perfherder, defect)

Description

[William Lachance (:wlach)]

We'll be keeping the job metadata in one centralized place once bug 1265037 lands, we should use that for performance datum instead of an arbitrary reference to the per-project id. This is a prerequisite to migrating away from the per-project jobs tables.

Comment 1

[GitHub Autolander Bot]

Attachment: [treeherder] mozilla:1265503 > mozilla:master

Comment 2

[William Lachance (:wlach)]

Comment on attachment 8809526 [details] [review]
[treeherder] mozilla:1265503 > mozilla:master

Hey Rob, I know you're not all that familiar with this part of the code, but I figured this would be a good time to learn. I can sit down and explain this change to you when we both have a spare moment.

Comment 3

[Robert Wood [:rwood]]

Comment on attachment 8809526 [details] [review]
[treeherder] mozilla:1265503 > mozilla:master

LGTM!

Comment 4

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/ce4903cd63e60d397a3329b91a8919c984825dca
Bug 1265503 - Rename column for datasource job id

https://github.com/mozilla/treeherder/commit/8b94f6a0ff4322c35c71dd403f65a6fa931d673c
Bug 1265503 - Rename datasource job attribute for perf datum to ds_job

This will let us access the id of the to-be-added ORM job via a
`.job_id` parameter.

Comment 5

[GitHub Autolander Bot]

Attachment: [treeherder] wlach:1265503-2 > mozilla:master

Comment 6

[William Lachance (:wlach)]

Comment on attachment 8819911 [details] [review]
[treeherder] wlach:1265503-2 > mozilla:master

Hey Rob, can you take a look at this for me? It's a followup from the previous work in this bug. We can now access the main job metadata from a performance datum, which will facilitate us dropping the reference to the datasource tables.

Comment 7

[Robert Wood [:rwood]]

Comment on attachment 8819911 [details] [review]
[treeherder] wlach:1265503-2 > mozilla:master

LGTM, and applied the patch locally and ran the tests, they passed fine.

Comment 8

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/931865befe4ce3e5352fa5d9dcfd4a117e0dc8c6
Bug 1265503 - Move data cycling tests into their own file

https://github.com/mozilla/treeherder/commit/ddd9b13eef464805574ed61c03e36d961de42f93
Bug 1265503 - Add a reference to the main db job objects from performance datum

Comment 9

[William Lachance (:wlach)]

Attachment: SQL to perform migration

Once again we need to modify the database with manual sql. In this case, we can use the table swapping method so we don't have to pause job ingestion. The above
sql needs to be modified at run time to use the maximum id we get in the 3rd last query.

Comment 10

[William Lachance (:wlach)]

SQL migrations have been applied and code is landed. We're done here.