1510332 - Create AWS Amazon account for Armen

Status: RESOLVED WONTFIX

(Cloud Services :: Operations: Miscellaneous, task)

Description

[Armen [:armenzg]]

I need to deploy an SPA that we need to keep pretty tight because we will temporarily have org related data until we can fetch this data directly from the people API.

This is a somewhat important account to create since I need it for the all hands since I have directors requesting this project and want to deploy it for them.

RRA:
https://docs.google.com/document/d/1LChbVoYM3cxjDYBDnrZFQI35gTztcaElomesqzVxpaw/edit#heading=h.ufs6i6416jwo

Comment 1

[Chris Kolosiwsky [:ckolos] (ckolos has left the building)]

I think my biggest question surrounding this (other than awaiting :ulfr's opinion) is why activedata (via bugzillaETL) isn't a fit for this use case? 

Activedata already has all the access required as well as a sizeable ES cluster backing it up. It also has the additional benefit of being a production app with monitoring etc. and proper access controls including SSO.

Comment 2

[Kyle Lahnakoski [:ekyle]]

I have been waiting for a bug like this to open up.  :)

There is a piece of code [1] that pulled the phonebook and put it into the private cluster. It is not part of the Bugzilla-ETL docker image, but as you can see by the size, we could add it with relative ease. 

[1] phonebook ETL - https://github.com/klahnakoski/MoDevETL/blob/dev/MoDevETL/org_chart.py

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

:ekyle, can you help :armenzg determine if all his data needs can be pulled from the bugillaETL project?  If so maybe the app he is working on could switch data sources and be stood up faster?  If not, it would be good to document that in this bug what is missing.

Comment 4

[Habibullah Pagarkar (use Slack)]

Since this app is related to Bugzilla data pulled for Firefox, it is to be supported by the CloudOps team. To be supported by us, the needs are a little more complex. Additionally, this app will request sensitive data it needs to be behind SSO, which complicates matters more. :ckolos, can you outline the steps needed to make this app CloudOps supportable if using bz-etl isn't an option.

Comment 5

[Chris Kolosiwsky [:ckolos] (ckolos has left the building)]

In order to deploy any app to production, the following standards must be met:

 - A deployable docker container, containing the app, must be created according to the standards listed here: https://github.com/mozilla-services/dockerflow . This includes CI builds and automated docker hub pushes via CircleCI.
 - Infrastructure deployments via Jenkins and Ansible (this is done by cloudops)
 - Instance configuration via Jenkins + Ansible + Puppet (also cloud-ops)
 - For an app containing/utilizing PII, additional steps are required. At a minimum, SSO/Auth0 must be used.
 - As above, the CloudOps security team will need to evaluate the application and may add additional requirements as needed.
 - Depending on the sensitivity of the data, the application may be required to be run in our Prod account. *Very* sensitive applications must be deployed in the Data account and need to be handled by the data ops team.

Comment 6

[Armen [:armenzg]]

Kyle will be sending me some information and see if we can use it.

Comment 7

[Kyle Lahnakoski [:ekyle]]

I looked into the private cluster: Although accessing the query tool and sending queries looks fine, there seems to be some configuration issues around CORS and re-authentication-after-timeout. Bug 1511085 goes into detail.

Armen, 

Despite these issues, you might be able to access the cluster: 

1. Turn off CORS checking on your browser.
2. I assume there is a ES6 client library that does the authentication navigation, as needed

Comment 8

[Armen [:armenzg]]

I've spoken with Rail and he's going to help me set this up via release-services in the same way that ship-it got set up with SSO. Once we get close I will speak with kang again (since I did the RRA initially with him) to make sure everything makes sense.

I think we can close this bug for now.

If you're digging into bzETL problems for my sake please don't use your time on it.