1868861 - Add GCP clonebundles to `northamerica-northeast1`

Status: RESOLVED FIXED

(Developer Services :: Mercurial: hg.mozilla.org, task)

Description

[Andrew Halberstadt [:ahal]]

We had an "outage" because GCP resources in both us-central1 and us-west1 were being maxed out. We limit many pools to these regions because that is where we upload clonebundles for hg.m.o. It would be nice to add a third region to mitigate this occurrence.

After a quick search on:
https://googlecloudplatform.github.io/region-picker/

I propose northamerica-northeast1. It's $0.002/hour more expensive than us-east5 or us-east1, but Google claims it is 100% carbon free vs 60% and 26% for the us-east regions.

Comment 1

[Andrew Halberstadt [:ahal]]

Hey Connor, we've been hitting a spike of resources being maxed out in our current regions again. Could we get this uploaded sometime in the coming days?

Comment 2

[Connor Sheehan [:sheehan]]

Yes, I will prioritize this and try to get it up early next week. It fell off my radar a little bit, apologies for that.

Comment 3

[Connor Sheehan [:sheehan]]

Attachment: terraform: add new GCS bucket in `northamerica-northeast1` (Bug 1868861) r?zeid

Add a new bucket for clonebundle uploads in northamerica-northeast1.
Add an upload policy so the hgbundler user can upload to the bucket,
and allow public access to resources in the bucket.

Comment 4

[Connor Sheehan [:sheehan]]

Attachment: manifest-scraper: add `northamerica-northeast1` to the required regions set (Bug 1868861) r?zeid

Enforce northamerica-northeast1 as a required region in the
manifest IP scraper script.

Comment 5

[Connor Sheehan [:sheehan]]

Attachment: bundles: add the `northamerica-northeast1` bundle bucket to the upload script (Bug 1868861) r?zeid

This will cause the bundle generation script to upload bundles to the
new bucket, and add the new bundle URLs to the clonebundles manifest
files.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/5867c24595ad
terraform: add new GCS bucket in northamerica-northeast1 r=zeid
https://hg.mozilla.org/hgcustom/version-control-tools/rev/489c6b11fc79
manifest-scraper: add northamerica-northeast1 to the required regions set r=zeid
https://hg.mozilla.org/hgcustom/version-control-tools/rev/b80774e2aeb9
bundles: add the northamerica-northeast1 bundle bucket to the upload script r=zeid

Comment 7

[Connor Sheehan [:sheehan]]

This is deployed now. New bundles will be generated overnight and should be available in northamerica-northeast1 tomorrow. I'll update here if something goes wrong with the bundle upload process.

Comment 8

[Andrew Halberstadt [:ahal]]

Awesome, thanks!