1149642 - Perfherder should store one year's worth of data for each series

Status: RESOLVED FIXED

(Tree Management :: Perfherder, defect)

Description

[William Lachance (:wlach)]

For doing historical analysis, we want at least 365 days worth of series data (I don't think we need to keep the raw performance artifacts around). I think the current perfherder model should scale to this amount -- for the 163.com tp5 set e.g. 90 days is in the range of 22k worth of JSON, so one year's worth shouldn't be too much worse. We can always gzip the information to reduce the disk consumption if that's an issue.

Comment 1

[William Lachance (:wlach)]

Attachment: PR

This adds the mechanism to store *new* data in a one year series. To get off to a running start, we'll want to copy the currently-stored data for the 90-day interval *before* applying this. I wrote up a small python script to do this, will attach that to this bug.

Comment 2

[William Lachance (:wlach)]

Attachment: Conversion script

This script should be run as follows from the virtual environment before merging and deploying the pull request:

python /path/to/copy-performance-series.py 7776000 31536000

Comment 3

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/7763eeae185264f5312d53707d9eb90368f1cae0
Bug 1149642 - Store one year's worth of performance data

Comment 4

[William Lachance (:wlach)]

Merged and deployed. Will take a while to accumulate this much data, ofc!