Closed Bug 1225140 Opened 7 years ago Closed 7 years ago
Further parallelize executive report
Currently the executive report needs to ingest data in order, one day at a time. This makes it very time consuming to run the report on a large date range, since it can't easily be parallelized across analysis machines (beyond parallelizing by reporting period). I would like to be able to run each day's report to produce a cuckoo filter containing that day's summary, then combine these summaries into a report. This would mean that all the days could be run at the same time in parallel, hopefully speeding things up a lot.
There is a custom one-off cuckoo filter version I provided to mreid. However, custom one-offs will not scale well so mreid is testing out longitudinal analysis using the Redshift databases.
Assignee: mtrinkala → nobody
The change to use Redshift / SQL to power the exec report renders this obsolete.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.