1142631 - We're storing large amounts of per-suite auxillary data per test in perfherder unnecessarily

Status: RESOLVED FIXED

(Tree Management :: Perfherder, defect)

Description

[William Lachance (:wlach)]

In investigating bug 1142488, I found out that we're storing a large amount of auxillary data (mostly metrics on memory consumption, etc) that is generated by talos during a suite run per-test. We're not currently using this for anything, and it's absolutely massive (probably accounting for 80% or so of the disk usage for performance artifacts).

If we were going to store this (I'm not sure we should, at least in this form  -- according to :jmaher this aggregated data has never been very useful), it should be stored as part of a suite-level series summary.

For now, let's just stop storing it altogether, since it definitely doesn't belong in the test artifacts. We can consider bringing it back in some form when we have a real user story for it.

Comment 1

[William Lachance (:wlach)]

Attachment: Bug 1142631 - Don't store talos auxillary data per-test

Comment 2

[Ed Morley [:emorley]]

Comment on attachment 8576788 [details] [review]
Bug 1142631 - Don't store talos auxillary data per-test

I suspect flake8 might complain about the "#for" but that can be fixed on commit.
(And happy to tweak the pep8 warnings ignore list too)

Comment 3

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder-service

https://github.com/mozilla/treeherder-service/commit/cd50b4cb1f91c3a372464b7c932afe5aeb86ce9d
Bug 1142631 - Don't store talos auxillary data per-test

Comment 4

[William Lachance (:wlach)]

(In reply to Ed Morley [:edmorley] from comment #2)
> Comment on attachment 8576788 [details] [review]
> Bug 1142631 - Don't store talos auxillary data per-test
> 
> I suspect flake8 might complain about the "#for" but that can be fixed on
> commit.
> (And happy to tweak the pep8 warnings ignore list too)

You are right, I fixed that before merging. I probably should have waited for travis to finish before doing so, but I'm pretty sure that it should all be fine (since I ran unit tests locally and they worked)