Closed Bug 1294694 Opened 8 years ago Closed 8 years ago

Prune telemetry aggregates

Categories

(Cloud Services Graveyard :: Metrics: Pipeline, defect, P2)

Product:
Cloud Services Graveyard
Component:
Metrics: Pipeline
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: rvitillo, Assigned: robotblake)

References

Details

User Story

The data that powers the main telemetry.mozilla.org (tmo) dashboard has been growing without bounds for a while now. As we will soon limit storage limits, the time has come to do some pruning. 

We don't need to keep around data about every single nightly build of last year, for example. We might want to keep say one data point per week, which will still allow us to plot trends while reducing drastically our storage requirements.

The first part of this project is to talk with various stakeholders and come up with a pruning strategy. Then, implement that strategy in python_mozaggregator, which is the job that generates the data that powers tmo.
No description provided.
Assignee: nobody → fbertsch
Points: --- → 2
Priority: -- → P2
Blocks: 1297060
Summary: Prune tmo dataset → Prune telemetry aggregates
Blocks: 1297063
After discussing with the group and robotblake, we've decided to just increase the instance size to 2TB. Handing this off to robotblake.
Status: NEW → ASSIGNED
Assignee: fbertsch → bimsland
Blake, please make sure this bugs lands before we run out of space. Backfilling this particular job is very expensive and slow.
The cluster size has been increase to 2TB.
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Product: Cloud Services → Cloud Services Graveyard
You need to log in before you can comment on or make changes to this bug.