922477 - Create new ElasticSearch cluster for non-public bugs

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task, P4)

Description

[Kyle Lahnakoski [:ekyle]]

Similar to 872363, but this cluster can only be accessible to people with LDAP access

Comment 1

[Daniel Maher [:phrawzty]]

From email:

On 2013-09-30 10:57 PM, Mark Côté wrote:
> On 2013-09-26 4:40 AM, Daniel Maher wrote:
>> So to summarise, you want :
>> 1. Another ES cluster with the same config as the one I set up for you
>> previously.
>> 2. For the existing ES cluster to have its role changed.
>> 3. To ensure VPN access to the existing cluster (and, eventually, the
>> new one).
>>
>> That about right ?
>>
> Correct!
>
> Mark

Comment 2

[Mark Côté [:mcote]]

Depending on how quickly the second cluster can be set up, we may not need to bother with step 2.  We're just looking to start testing the full, private-and-public ETL first, since it is simpler.  If it's quick to set up another ES cluster, to keep things simple we can keep the original in its role as public.  If it will take a while, then we can reopen bug 872363 and put in the new cluster info.  Regardless, in both cases we'll need VPN access (at least for myself and :ekyle) and access to the appropriate machine set up for bug 922877.

Comment 3

[Daniel Maher [:phrawzty]]

Hello,

elasticsearch[4-6].bugs.scl3.mozilla.com are now set up in an ES cluster.  As per bug 930503, you now have VPN access to those machines (as well was [1-3]).

It may be worth noting that access to ES on both clusters is currently "internal-only", meaning that if you want to access these machines from public-facing web nodes, we'll need to set up additional infrastructure.  Please re-open and comment on this bug if that is the case.

Have a great day !

Comment 4

[Kyle Lahnakoski [:ekyle]]

Just to confirm:  There are two clusters [1-3] and [4-6] which are completely independent of each other.  The [4-6] cluster is intended for the non-public bugs, and the [1-3] for the public bugs (as per https://bugzilla.mozilla.org/show_bug.cgi?id=872363).  Yes?

Comment 5

[Mark Côté [:mcote]]

I think it's up to us to define that.  I'm fine with your proposal.

Comment 6

[Kyle Lahnakoski [:ekyle]]

I also confirmed I can access all 6 servers.  Thank you!

Comment 7

[Kyle Lahnakoski [:ekyle]]

Mark, I mostly what to confirm the clusters are not sharing names, and they can not reach each other.

Comment 8

[Daniel Maher [:phrawzty]]

(In reply to Kyle Lahnakoski [:ekyle] from comment #4)
> Just to confirm:  There are two clusters [1-3] and [4-6] which are
> completely independent of each other.  The [4-6] cluster is intended for the
> non-public bugs, and the [1-3] for the public bugs (as per
> https://bugzilla.mozilla.org/show_bug.cgi?id=872363).  Yes?

In our internal docs 1-3 is for public bugs, and 4-6 is for non-public bugs; both clusters are entirely independent of one another.