Daily Aggregate Compression in Redshift

RESOLVED INVALID

Status

RESOLVED INVALID
4 years ago
4 years ago

People

(Reporter: Mardak, Assigned: tspurway)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: .008)

(Reporter)

Description

4 years ago
Copied from https://github.com/tspurway/infernyx/issues/10

Right now we write data to Redshift quite often. This gives us great latency (freshness), but also increases storage requirements due to low aggregation. This increases the price of the Redshift instance as well as reduce query performance.

We can introduce a daily job in Infernyx that 'compresses' the data by writing a single row for each unique key per day, and cleaning out the existing day's data.
(Reporter)

Updated

4 years ago
Depends on: 1105555
(Reporter)

Updated

4 years ago
Iteration: 37.1 → 37.2
(Assignee)

Updated

4 years ago
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → INVALID
(Reporter)

Updated

4 years ago
Iteration: 37.2 → 36.3
Points: --- → 3
Whiteboard: .? → .008
(Reporter)

Updated

4 years ago
Assignee: nobody → tspurway
You need to log in before you can comment on or make changes to this bug.