CURRENT: 33 m4.xlarge - .299$ an hour per instance * 33 = 9.867$ - 528 GiB - 132 vCPU PROPOSE: 1 X1 - $13.338 On Demand - $7.671 Hourly 1-Year Term - 1952 GiB - 128 vCPU With 4x the memory and no network overhead, not only would queries by lightning fast, but we would reduce the need for resources. More people would be able to use that machine than use the similarly priced cluster. Whole datasets could be cached in memory. For example, a single version of longitudinal takes up ~275 GiB. We could cache the most recent version, and probably a few other datasets (client_count, xSec, etc.), and have lightning fast queries for the majority of our users. The downside is this does not run on EMR, so it would probably be more work on the Ops side.
Blake mentioned that he's looked into this, and it may not be possible. There's very little information online about running Presto on an X1 instance. I'm pinging robotblake to see what he's found.
The JVM doesn't scale particularly well past 32GB so we're going to skip this, also with Athena being a viable option for the heavy queries.
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.