1198786 - Stop storing "performance artifacts"

Status: RESOLVED FIXED

(Tree Management :: Perfherder, defect)

Description

[William Lachance (:wlach)]

We were originally going to store "performance artifacts" for each talos run for 3rd party alert generation, but this turned out to not be ever used. These are taking up quite a bit of space and don't really fit into the new data model I'm working on in bug 1192976. The fact that the API to retrieve them has been broken for months without anyone noticing is a good sign that they're probably safe to remove. :)

I think the only potentially useful thing in them are the raw replicates from each talos run, but there are better ways of storing that which we can always add later (when we have a feature that uses them).

I'd like to do this before bug 1192976, as it will make migrating the database a simpler process (since there will be lots more free space to do it).

Comment 1

[Ed Morley [:emorley]]

Depending on how much this relies on data expiring naturally, I may ask them to hold off on bug 1190361 so that defrag lets us see the wins here sooner. (But I'll wait for the PR so I understand what we're no longer storing and whether it can be deleted/dropped sooner).

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

+314

Comment 3

[William Lachance (:wlach)]

(In reply to Ed Morley [:emorley] from comment #1)
> Depending on how much this relies on data expiring naturally, I may ask them
> to hold off on bug 1190361 so that defrag lets us see the wins here sooner.
> (But I'll wait for the PR so I understand what we're no longer storing and
> whether it can be deleted/dropped sooner).

Basically this bug means dropping the per-project "performance_artifact" tables, which are rather big.

Comment 4

[Ed Morley [:emorley]]

Ah awesome! :-)

Comment 5

[William Lachance (:wlach)]

Attachment: Stop storing performance artifacts

Hi Ed, could you take a look at this? I know you're less familiar with the perf subsystem, but most of this patch (except for the unit test) is just removing code in areas you are probably at least somewhat familiar with.

:jmaher: If you could sanity check my updated unit test, that would be good. :)

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8653001 [details] [review]
Stop storing performance artifacts

this code is foreign to me, but I did look over the unittest and it was nice to see.  I assume we are still storing the raw replicates somewhere?  Other than that, the more code we can remove the better.

Comment 7

[Ed Morley [:emorley]]

Comment on attachment 8653001 [details] [review]
Stop storing performance artifacts

Forgot to set the r+ when I commented on the PR earlier.
r=me with the comments fixed :-)

Comment 8

[Ed Morley [:emorley]]

I've not seen any errors whilst the test branch was deployed to stage (and there were also no complaints from people using the API), so think we're good to go here. (Stage was reset back to master ~30 mins ago btw).

Comment 9

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/7440089508eb136a2f620d0cd145773c6b05ee99
Bug 1198786 - Stop storing performance artifacts

They were never used for anything and take up a lot of space. They
also don't fit into the new performance model we're working on. We
can always bring back the useful bits later.

Comment 10

[William Lachance (:wlach)]

(In reply to Ed Morley [:emorley] from comment #8)
> I've not seen any errors whilst the test branch was deployed to stage (and
> there were also no complaints from people using the API), so think we're
> good to go here. (Stage was reset back to master ~30 mins ago btw).

Yup, merged. I will drop the performance_artifact tables from stage later today.