1548302 - Apply CI-private Terraform config to remaining AWS regions

Status: RESOLVED FIXED

(Developer Services :: Mercurial: hg.mozilla.org, task, P1)

Description

[Connor Sheehan [:sheehan]]

Now that all the required VPN/VPC connections have been created for the remaining AWS regions, we need to import the created resources into Terraform and apply the aws-vpc module to them. There will likely be a few Terraform changes required to get this working.

Comment 1

[Connor Sheehan [:sheehan]]

Attachment: terraform: add "to-mdc" search term in `aws_vpn_gateway` data source (Bug 1548302) r?bstack

Previously we searched for "to-mdc's". The new gateway only
links back to one of the datacentres, so keeping the name while
selecting the correct connection seems like the correct way to
handle this.

Comment 2

[Connor Sheehan [:sheehan]]

Attachment: terraform: remove subnets in availability zone C (Bug 1548302) r?bstack

Turns out, us-west-1 doesn't have an availability zone C. This
was mentioned to me during review, and I thought I checked each
region to ensure there was a third AZ for each, but apparently
the way I checked was wrong. Oops!

Considering we aren't using the subnet in that AZ at the moment
any way, this isn't a huge deal. So this commit removes the C
availability zone subnets (public and private).

Comment 3

[Connor Sheehan [:sheehan]]

Attachment: terraform: remove security group rule allowing Telegraf traffic from MDC1 (Bug 1548302) r?bstack

This rule was previously in place to allow our on-premises hosts
to send data to the test InfluxDB instance. Now that we have the
production service online, we can remove it.

Comment 4

[Connor Sheehan [:sheehan]]

Attachment: terraform: rename Route53 record variable to be region agnostic (Bug 1548302) r?bstack

The Terraform variable name included "uw2", short for
"us-west-2". Now that we are deploying to more regions,
it should be changed.

Comment 5

[Connor Sheehan [:sheehan]]

Attachment: terraform: create unique certificate per region (Bug 1548302) r?bstack

The previous plan was to create a single certificate that would
be shared across AWS regions, but we have since learned that
ACM certificates are per-region only. So this commit moves the
certificate resource definition from the main.tf file for the
entire Terraform config and into the aws-vpc modules for each
specific AWS region. We also remove helper variables from the
module.

Comment 6

[Connor Sheehan [:sheehan]]

Attachment: terraform: stand up AWS VPC environment in us-west-1 (Bug 1548302) r?bstack

Now that our environment is reproducible in different AWS regions,
this commit creates a new instance of the aws-vpc modules for use
in us-west-1.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/f250d8415b45
terraform: add "to-mdc" search term in aws_vpn_gateway data source r=bstack
https://hg.mozilla.org/hgcustom/version-control-tools/rev/4172e809ba0e
terraform: remove subnets in availability zone C r=bstack
https://hg.mozilla.org/hgcustom/version-control-tools/rev/19a3a9e03887
terraform: remove security group rule allowing Telegraf traffic from MDC1 r=bstack
https://hg.mozilla.org/hgcustom/version-control-tools/rev/64e8e9a62933
terraform: rename Route53 record variable to be region agnostic r=bstack
https://hg.mozilla.org/hgcustom/version-control-tools/rev/c6ea6220b09d
terraform: create unique certificate per region r=bstack
https://hg.mozilla.org/hgcustom/version-control-tools/rev/0ac1bb96d120
terraform: stand up AWS VPC environment in us-west-1 r=bstack

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/e63c96151e7d
ansible: add new hosts to bootstrap hostgroup

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/858555769242
ansible: add new hosts in `us-west-1` to `hgweb-mirrors` hostgroup

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/ca6924b0110d
terraform: stand up private hg service in us-east-1 
https://hg.mozilla.org/hgcustom/version-control-tools/rev/c5cdceaf4de3
ansible: add us-east-1 hosts to bootstrap hostgroup

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/e3dce3f0d26a
terraform: stand up private hg service in us-east-2 
https://hg.mozilla.org/hgcustom/version-control-tools/rev/cbfd7f665c0f
ansible: move us-east-1 hosts to hgweb-mirrors hostgroup and us-east-2 hosts to bootstrap group 
https://hg.mozilla.org/hgcustom/version-control-tools/rev/0bf07df77731
ansible: split `hgweb-mirrors` into regional subgroups 
https://hg.mozilla.org/hgcustom/version-control-tools/rev/4c5a2e4076a4
ansible: move newly bootstrapped hosts in `ci-ue2` hostgroup 
https://hg.mozilla.org/hgcustom/version-control-tools/rev/4588772ce1ae
terraform: stand up private hg service in eu-central-1 
https://hg.mozilla.org/hgcustom/version-control-tools/rev/09eac4fb3815
terraform: add AMI ID for CentOS7 in eu-central-1 
https://hg.mozilla.org/hgcustom/version-control-tools/rev/ccc7fbade54f
bootstrap: add us-east-2 hosts to bootstrap hostgroup

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/c72925f6b17c
ansible: fix `bootstrap_hostname` for eu-central-1 mirrors

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/c1c4a668367b
ansible: use correct IPs for eu-central-1 hosts

Comment 14

[Connor Sheehan [:sheehan]]

All regions have the Terraform config applied and hosts have been bootstrapped.