Closed Bug 1803359 Opened 3 years ago Closed 3 years ago

review Eliot instance size and production configuration

Categories

(Eliot :: General, task, P2)

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: willkg, Assigned: willkg)

Details

Since the increase in sym file sizes, Eliot has been falling over in production (bug #1793984). It's probably time for us to re-examine the instance size and other configuration.

Grabbing this to do now.

Assignee: nobody → willkg
Status: NEW → ASSIGNED

I started writing up a big bugzilla comment, but then realized I want to add a bunch of graphs, so I switched it to a Google doc.

https://docs.google.com/document/d/1ZKCUKG34k1qkGO3drPbuf6UsghAmgBnCXtds1wKx3nA/edit#

It comes with a grafana dashboard, too!

https://earthangel-b40313e5.influxcloud.net/d/wtG190NVk/tecken-eliot-memory-usage-analysis-2022-10-27?orgId=1&refresh=1m

My recommendation at this time is to leave the instance sizing as is and number of gunicorn workers capped at 5. I think we should keep an eye on it, so I improved the dashboard I was using to look at memory issues. Having said that, I predict we'll keep things as they are until we move to GCP.

This analysis will probably help with the migration.

Marking as FIXED. I'll reopen if there are comments in the doc that cause us to revisit the recommendation.

Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Summary: re-up eliot prod configuration → review Eliot instance size and production configuration

Moving to Eliot product.

Component: Symbolication → General
Product: Tecken → Eliot
You need to log in before you can comment on or make changes to this bug.