Closed Bug 1627697 Opened 6 years ago Closed 6 years ago

Recreate Firefox CI TC PG instance with correct name and switch to it

Categories

(Cloud Services :: Operations: Taskcluster, defect)

Product:
Cloud Services
Component:
Operations: Taskcluster
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: brian, Assigned: edunham)

Details

https://github.com/mozilla-services/cloudops-infra/pull/1972

edunham tells me she is sorting this out. Her approach is

  1. manually create new db
  2. terraform state rm and import to swap new db for old in its mind
  3. create users and run db:upgrade in new db
  4. change ip address for db in sops
  5. redeploy firefox ci tc
  6. manually delete old db

Chatted with edunham and she plans to finish this Tuesday.

Well, in almost that order:

  • created new DB
  • provisioned users in new db
  • added all users to sops
  • Removed the old DB from the Terraform state and replaced it with the new one:
$ terraform state rm module.postgres.google_sql_database_instance.master 
Removed module.postgres.google_sql_database_instance.master
Successfully removed 1 resource instance(s).
$ terraform import module.postgres.google_sql_database_instance.master taskcluster-prod-firefoxcitc-v1
module.postgres.google_sql_database_instance.master: Importing from ID "taskcluster-prod-firefoxcitc-v1"...
module.postgres.google_sql_database_instance.master: Import prepared!
  Prepared google_sql_database_instance for import
module.postgres.google_sql_database_instance.master: Refreshing state... [id=taskcluster-prod-firefoxcitc-v1]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

Next time we deploy firefoxci (probably tomorrow morning) it will run the db:upgrade step from Jenkins as part of the deployment, since we landed that PR today. After a firefoxci deploy succeeds I will manually delete the misnamed DB.

Firefoxci is now switched over to the correct instance. I've stopped the old one and will delete it in a bit.

Nothing paged and nobody complained when I shut down the old one, so I've deleted it.

Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.