1182616 - [Air Mozilla] ElasticSearch cluster for airmozilla dev, stage and prod

Status: RESOLVED FIXED

(Infrastructure & Operations :: IT-Managed Tools, task)

Description

[Peter Bengtsson [:peterbe]]

In Q3 we're going to need to use ElasticSearch on Air Mozilla. 
I think all I need is a URL to be configured for dev, stage and prod in some secure fashion. 

My intern is actively working on a new feature to do ES queries. Her work will be possible to merge to master and being preffed off but it would be nice if she can see her work for realz before she finishes.

Comment 1

[C. Liang [:cyliang]]

@peterbe:  Do you have some time this upcoming week to have a quick chat about this?  Mostly, what we're looking for is some sense of what your storage / performance / backup needs will be for this data, so we can get things set up properly.

Comment 2

[Peter Bengtsson [:peterbe]]

(In reply to C. Liang [:cyliang] from comment #1)
> @peterbe:  Do you have some time this upcoming week to have a quick chat
> about this?  Mostly, what we're looking for is some sense of what your
> storage / performance / backup needs will be for this data, so we can get
> things set up properly.

Yes please! My calendar is pretty up to date if you want to book me into something.

Comment 3

[C. Liang [:cyliang]]

Based on today's discussion, it makes the most sense to spin up a small ES cluster just for AirMo.  I've filed a bug requesting VMs for this cluster and have modified the title of this bug.

Comment 4

[C. Liang [:cyliang]]

I've set up an Elasticsearch cluster at http://airmo-elasticsearch-zlb.webapp.scl3.mozilla.com:9200.  Access to that cluster is currently limited to the generic cluster.  If, during the proto-typing phase, we need to allow access from a different server, please let me know.  (We generally have tight restrictions on Elasticsearch access due to the history of Elasticsearch vulnerabilities. >_< )

Comment 5

[Peter Bengtsson [:peterbe]]

(In reply to C. Liang [:cyliang] from comment #4)
> I've set up an Elasticsearch cluster at
> http://airmo-elasticsearch-zlb.webapp.scl3.mozilla.com:9200.  Access to that
> cluster is currently limited to the generic cluster.  If, during the
> proto-typing phase, we need to allow access from a different server, please
> let me know.  (We generally have tight restrictions on Elasticsearch access
> due to the history of Elasticsearch vulnerabilities. >_< )

Cool. I'll have to make sure the dev, stage and prod servers have different prefixes on the index name. 

I gather that :ericz is working on trying to move the airmozilla dev server to the new python 2.7 based cluster. So I guess we ought to give access from that one too.

Comment 6

[C. Liang [:cyliang]]

I see that data has landed in the Air Mozilla ES cluster. =)