1399162 - Move aggregates service to ops purview (deploy prod)

Status: RESOLVED FIXED

(Data Platform and Tools :: General, enhancement, P1)

Comment 1

[Harold Woo]

Just some notes for self:

- rerun deployment to stage with master branch of python_mozaggregator to verify everything is still working properly
- verify deployment of prod service wont take CNAME of current running service (app_dns_name: "{{ stack }}.{{ dns_zone }}" in projects/mozaggregator/ansible/envs/default.yml?)
- push deployment to prod
- change rds password of temp db in prod manually to whatever is in hiera
- test service functionality (may need to take db snapshot and use stale data.  try importing snapshot from staging)

- schedule time to do cutover and stop ingestion to old prod service's db
- make backup of prod RDS 
- make changes to cloudops-deployment to reference prod-rds (env vars set in hiera sops?).  This may require a redeploy of prod to pull new secrets
- test elb endpoint and service again
- move cname of current service to new deployment
- test
- start ingestion of data to new prod db (iam perms required?)

old bug -
https://bugzilla.mozilla.org/show_bug.cgi?id=1342947

old prs
https://github.com/mozilla/python_mozaggregator/pull/63
https://github.com/mozilla-services/cloudops-deployment/pull/1564
https://github.com/mozilla/python_mozaggregator/pull/50

open issues
- add newrelic-admin entrypoint?

Comment 2

[Harold Woo]

- front service with openresty/nginx
- investigate whether or not we can use the same prod database as currently in use (this will simplify the migration greatly)

Comment 3

[Harold Woo]

x - rerun mozaggregator-env jenkins job for stage|prod
x - ensure elb listener ports change (for nginx change)
x - manually repoint rds dns cnames to telemetry-aggregates rds endpoints (for prod point this to the existing dev dbs)
x - modify hiera-sops so that mozaggregator points to the production cnames (dbs in dev) with correct password
x - log on and ensure that nginx/openresty is installed
x - continue mozaggregator jenkins job for prod and redo steps
- frank to test functionality of prod endpiont
- ask ckolos or it to modify cname aggregates.telemetry.mozilla.org -> mozaggregator.prod.mozaws.net...or the ELB endpoint directly
---> ask ckolos what his opinion is on which is better. If we use ELB directly we need to modify the SSL cert on the elb to use tmo (da1ea4a6-bcac-43aa-b3b9-84b4912f63de)

projects/mozaggregator/ansible/envs/prod.yml:
elb_cert_arn: arn:aws:acm:us-west-2:361527076523:certificate/da1ea4a6-bcac-43aa-b3b9-84b4912f63de