Add GCP clonebundles to `northamerica-northeast1`
Categories
(Developer Services :: Mercurial: hg.mozilla.org, task)
Tracking
(Not tracked)
People
(Reporter: ahal, Assigned: sheehan)
References
(Blocks 1 open bug)
Details
Attachments
(3 external files)
We had an "outage" because GCP resources in both us-central1 and us-west1 were being maxed out. We limit many pools to these regions because that is where we upload clonebundles for hg.m.o. It would be nice to add a third region to mitigate this occurrence.
After a quick search on:
https://googlecloudplatform.github.io/region-picker/
I propose northamerica-northeast1. It's $0.002/hour more expensive than us-east5 or us-east1, but Google claims it is 100% carbon free vs 60% and 26% for the us-east regions.
|
Reporter | |
Comment 1•2 years ago
|
||
Hey Connor, we've been hitting a spike of resources being maxed out in our current regions again. Could we get this uploaded sometime in the coming days?
|
Assignee | |
Comment 2•2 years ago
|
||
Yes, I will prioritize this and try to get it up early next week. It fell off my radar a little bit, apologies for that.
|
|
Assignee | |
Comment 3•2 years ago
|
||
Add a new bucket for clonebundle uploads in northamerica-northeast1.
Add an upload policy so the hgbundler user can upload to the bucket,
and allow public access to resources in the bucket.
|
|
Assignee | |
Comment 4•2 years ago
|
||
Enforce northamerica-northeast1 as a required region in the
manifest IP scraper script.
|
|
Assignee | |
Comment 5•2 years ago
|
||
This will cause the bundle generation script to upload bundles to the
new bucket, and add the new bundle URLs to the clonebundles manifest
files.
Pushed by cosheehan@mozilla.com:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/5867c24595ad
terraform: add new GCS bucket in northamerica-northeast1 r=zeid
https://hg.mozilla.org/hgcustom/version-control-tools/rev/489c6b11fc79
manifest-scraper: add northamerica-northeast1 to the required regions set r=zeid
https://hg.mozilla.org/hgcustom/version-control-tools/rev/b80774e2aeb9
bundles: add the northamerica-northeast1 bundle bucket to the upload script r=zeid
|
|
Assignee | |
Comment 7•2 years ago
|
||
This is deployed now. New bundles will be generated overnight and should be available in northamerica-northeast1 tomorrow. I'll update here if something goes wrong with the bundle upload process.
|
|
Reporter | |
Comment 8•2 years ago
|
||
Awesome, thanks!
Description
•