1340552 - Create Elasticsearch instances on the stage and prod Heroku apps

Status: RESOLVED INCOMPLETE

(Tree Management :: Treeherder: Infrastructure, defect, P3)

Description

[Ed Morley [:emorley]]

For bug 1321798, we'll need to add the Elastic Cloud Elasticsearch addon to the stage/prod Heroku apps, so they match treeherder-prototype.

As part of doing this, we'll need to:
1) Check the ES control panel config/versions/security settings used on treeherder-prototype to make sure we're happy with it
2) Create the new addons (renaming to match the naming scheme used elsewhere)
3) Generate new credentials for the various users across each environment (and share those around the Treeherder team for eg Kibana, or figure out a better way to store them, eg in environment variables)
4) Sync the config/prefs from #1 to the new addons
5) Define the `ELASTICSEARCH_URL` environment variable on stage/prod

Comment 1

[Ed Morley [:emorley]]

The addon on the prototype instance was removed in bug 1382227 due to bug 1378361 comment 8 and also bug 1382229.

Before we start using this addon again we should also confirm that we're happy adding a dependency on a service that is so flaky at times, and takes so long to reply to support emails.

Comment 2

[Ed Morley [:emorley]]

The instances was recreated on the prototype instance at some point by George.

Comment 3

[Ed Morley [:emorley]]

Marking incomplete since the auto-classify project is on hold.

Comment 4

[Ed Morley [:emorley]]

(In reply to Ed Morley [:emorley] from comment #2)

The instances was recreated on the prototype instance at some point by George.

I've had to delete the treeherder-prototype instance, since it was running out of CPU credits so timing out when requests were being made to it, causing eg 16,546 exceptions in New Relic in the last 24 hours.