Closed Bug 1285506 Opened 5 years ago Closed 4 years ago

Deploy AMIs for win2012 taskcluster worker type in us-west-1 and eu-central-1

Categories

(Infrastructure & Operations Graveyard :: CIDuty, task)

task
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: pmoore, Assigned: pmoore)

References

Details

Attachments

(3 files)

Before we can do this, we need to make sure s3 cache buckets/clone bundles exist in those regions...
Assignee: nobody → pmoore
Status: NEW → ASSIGNED
Attachment #8773252 - Flags: review?(rthijssen)
policy updated to include Frankfurt (https://console.aws.amazon.com/iam/home?region=us-west-2#policies/arn:aws:iam::692406183521:policy/taskcluster-level-1-sccache):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1467642244000",
            "Effect": "Allow",
            "Action": [
                "s3:DeleteObject",
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": [
                "arn:aws:s3:::taskcluster-level-1-sccache-eu-central-1/*",
                "arn:aws:s3:::taskcluster-level-1-sccache-us-east-1/*",
                "arn:aws:s3:::taskcluster-level-1-sccache-us-west-1/*",
                "arn:aws:s3:::taskcluster-level-1-sccache-us-west-2/*"
            ]
        },
        {
            "Sid": "Stmt1467642244001",
            "Effect": "Allow",
            "Action": [
                "s3:GetBucketLocation",
                "s3:GetBucketTagging",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::taskcluster-level-1-sccache-eu-central-1",
                "arn:aws:s3:::taskcluster-level-1-sccache-us-east-1",
                "arn:aws:s3:::taskcluster-level-1-sccache-us-west-1",
                "arn:aws:s3:::taskcluster-level-1-sccache-us-west-2"
            ]
        }
    ]
}
jhford: do we need to modify cloud-mirror to support building in eu-central-1
Flags: needinfo?(jhford)
Comment on attachment 8774428 [details]
Bug 1285506 - include eu-central-1 in tc sccache bucket regions;

https://reviewboard.mozilla.org/r/66890/#review63932
Attachment #8774428 - Flags: review?(mh+mozilla) → review+
Pushed by cbook@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/784f2a4f0d22
include eu-central-1 in tc sccache bucket regions; r=glandium
Keywords: checkin-needed
Rob, we currently have eu-central-1 enabled in cloud-mirror.

Here's an example image that's cached there.  Let me double check that the queue will rewrite to that location as well.

https://cloud-mirror.taskcluster.net/v1/redirect/s3/eu-central-1/https%3A%2F%2Fwww.mozilla.org%2Fmedia%2Fimg%2Fhome%2Fvoices%2Fmosaic-background.565019b6e6bc.jpg
Flags: needinfo?(jhford)
Rob, we did not have url rewriting on the cache side enabled for cloud-mirror.  I've written a patch for it and asked for review.  cloud-mirror will not be effective until this patch lands.
Attached file queue patch
Attachment #8774718 - Flags: review?(jopsen)
Commits pushed to master at https://github.com/taskcluster/taskcluster-queue

https://github.com/taskcluster/taskcluster-queue/commit/7f3205db23f5f37b3718b2ca03a806f84808bb1d
bug 1285506 -- let queue rewrite urls for eu-central-1

https://github.com/taskcluster/taskcluster-queue/commit/98b90ddfdedb6c96cd8b6d197149cf7576ffdef0
Merge pull request #106 from taskcluster/enable-eu-central-1

bug 1285506 -- let queue rewrite urls for eu-central-1
Comment on attachment 8774718 [details] [review]
queue patch

In the process of landing...
Attachment #8774718 - Flags: review?(jopsen) → review+
gps: do you have any concerns or thoughts regarding hg clonebundles or cloning from builders in eu-central-1? is it needed|difficult|possible to set up bundle caches in eu-central-1 to support builds there?
Flags: needinfo?(gps)
Thank you for notifying me about spinning up automation in a new region so we can configure the Mercurial bundles in that region. If we fail to configure the bundles, it isn't the end of the world: bundles will be downloaded from the CDN instead of S3 and we'll get hit with a CDN bill. But I strongly prefer having the bundles in place from day 0.

I've triggered manual generation of the bundles, so they should start appearing in eu-central-1 within an hour or so.
Flags: needinfo?(gps)
We're actively using eu-central-1 and us-west-1 so closing this now.
Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Removing leave-open keyword from resolved bugs, per :sylvestre.
Keywords: leave-open
Component: Platform Support → Buildduty
Product: Release Engineering → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.