Proposal: Presto DB Running on a single X1 Instance

RESOLVED WONTFIX

Status

P3
normal
RESOLVED WONTFIX
2 years ago
2 months ago

People

(Reporter: frank, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

2 years ago
CURRENT:
33 m4.xlarge
- .299$ an hour per instance * 33 = 9.867$ 
- 528 GiB
- 132 vCPU

PROPOSE:
1 X1
- $13.338 On Demand
- $7.671 Hourly 1-Year Term
- 1952 GiB
- 128 vCPU

With 4x the memory and no network overhead, not only would queries by lightning fast, but we would reduce the need for resources. More people would be able to use that machine than use the similarly priced cluster. Whole datasets could be cached in memory.

For example, a single version of longitudinal takes up ~275 GiB. We could cache the most recent version, and probably a few other datasets (client_count, xSec, etc.), and have lightning fast queries for the majority of our users.

The downside is this does not run on EMR, so it would probably be more work on the Ops side.
(Reporter)

Updated

2 years ago
Blocks: 1255751

Updated

2 years ago
Points: --- → 3
Priority: -- → P3
(Reporter)

Comment 1

2 years ago
Blake mentioned that he's looked into this, and it may not be possible. There's very little information online about running Presto on an X1 instance. I'm pinging robotblake to see what he's found.
Flags: needinfo?(bimsland)
The JVM doesn't scale particularly well past 32GB so we're going to skip this, also with Athena being a viable option for the heavy queries.
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Flags: needinfo?(bimsland)
Resolution: --- → WONTFIX

Updated

2 months ago
Product: Cloud Services → Cloud Services Graveyard
You need to log in before you can comment on or make changes to this bug.