Closed Bug 1490041 Opened 7 years ago Closed 7 years ago

Increase i3.8xlarge on demand limit

Categories

(Data Platform and Tools Graveyard :: Operations, enhancement, P1)

Product:
Data Platform and Tools Graveyard
Component:
Operations
Type:
enhancement

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: frank, Assigned: jason)

References

Details

(Whiteboard: [DataOps])

This will enable longitudinal to run on databricks.
Whiteboard: [DataOps]
Melissa, I was hoping to get this moving ASAP. Do you have any issue with me opening a ticket on AWS? The job requires 16 nodes, so a limit of 25 or 50 should suffice. Currently we are at 2.
Flags: needinfo?(moconnor)
Databricks account already has: Running On-Demand i3.8xlarge instances 50. Which account do you need the increase in?
Assignee: nobody → jthomas
Flags: needinfo?(moconnor)
Priority: -- → P1
Great, that's what I needed! I was looking at the wrong account. Thanks Jason!
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → INVALID
Jason, would it be possible to bump this 100? That way we can manually let Airflow rerun the historical longitudinals without it going over the instance limit. The job takes 17 nodes, and iirc Airflow will only launch 4 simultaneous DAG runs.
Status: RESOLVED → REOPENED
Flags: needinfo?(jthomas)
Resolution: INVALID → ---
I've filed a ticket with AWS support and cc'd you on it.
Flags: needinfo?(jthomas)
Okay I filed two tickets. The first ticket was to increase it to 200 instances which caused a 'internal review' and may take time to process. The second ticket was for 100 instances which was auto approved and will be available within the next 30 mins. [US West (Oregon)]: EC2 Instances / Instance Limit (i3.8xlarge), New Limit = 100
Status: REOPENED → RESOLVED
Closed: 7 years ago → 7 years ago
Resolution: --- → FIXED
Product: Data Platform and Tools → Data Platform and Tools Graveyard
You need to log in before you can comment on or make changes to this bug.