1126937 - We don't expire data from the performance_series and series_signature tables

Status: RESOLVED WONTFIX

(Tree Management :: Perfherder, defect, P4)

Description

[Ed Morley [:emorley]]

Not handled here:
https://github.com/mozilla/treeherder-service/blob/93ae4541feb7013bd7181f5a714644a51521e55b/treeherder/model/derived/jobs.py#L157

Schema:

performance_series:
https://github.com/mozilla/treeherder-service/blob/c1713cab92e353c12a0288be6b16dc2de141bebe/treeherder/model/sql/template_schema/project_jobs_1.sql.tmpl#L319

series_signature: 
https://github.com/mozilla/treeherder-service/blob/c1713cab92e353c12a0288be6b16dc2de141bebe/treeherder/model/sql/template_schema/project_jobs_1.sql.tmpl#L287

Total size of job_eta on prod, across all projects (bug 1078523 comment 20):
- performance_series: 5.3 GB
- series_signature: 0.2 GB
(Though we haven't been ingesting for the full 4 months, so the win from this bug will be bigger in the future)

Comment 1

[William Lachance (:wlach)]

Note that the `performance_series` table already auto-expires old data (each row represents a json graph of data within a certain recency range, when data goes outside the range it gets expired).

The only win possible with this is if when we retire a test, which doesn't happen all that often.

Comment 2

[moijes]

Please can you elaborate more on this so that it can be clear as to what is required of someone who is to work on this. I'd like to work on this but I am having difficulty understanding the problem.

Comment 3

[William Lachance (:wlach)]

(In reply to moijes from comment #2)
> Please can you elaborate more on this so that it can be clear as to what is
> required of someone who is to work on this. I'd like to work on this but I
> am having difficulty understanding the problem.

I'd rather wait on implementing this, as it would introduce a basically unused codepath (currently nothing needs to be expired) that will almost certainly need to change later when we refactor treeherder's model API.

Comment 4

[William Lachance (:wlach)]

It's not clear that we'll ever want to expire performance data at this point, with the new database structure there's very little problem in the area of scalability. Let's mark this as won't fix. We can always open a new bug in the future.