Currently Release Engineering do some smoke testing on the staging environment https://stage.taskcluster.nonprod.cloudops.mozgcp.net when there is a new release of taskcluster. See http://docs.mozilla-releng.net/en/latest/taskcluster/tc_staging.html#run-fxci-to-send-mozilla-central-tasks-to-the-staging-cluster for more details. They test recent fxci pushes against the staging environment, and therefore the staging taskcluster needs access to a subset of fxci production images. Either: * both OCC and the docker-worker AMI generation process should grant access to the worker AMIs they generate in aws account 692406183521, to the aws staging account 710952102342 after creating them, or * the fxci tool that currently copies taskcluster entities between [FirefoxCI](https://firefox-ci-tc.services.mozilla.com) and [Stage-TC](https://stage.taskcluster.nonprod.cloudops.mozgcp.net) should take care of this Perhaps the fxci tool is the better place to do this, as it already syncs state between the two clusters, and would mean a change in one place, rather than in multiple image generating tools. It would also keep the code close together that syncs state between the two clusters, and be under the control of the team that do the testing, rather than a team that doesn't otherwise have much involvement in the process / staging environment. I'm not sure which subset of machine images from scm level 1,2,3 need to be shared.
Bug 1766815 Comment 0 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
Currently Release Engineering do some smoke testing on the Stage-TC environment https://stage.taskcluster.nonprod.cloudops.mozgcp.net when there is a new release of taskcluster. See http://docs.mozilla-releng.net/en/latest/taskcluster/tc_staging.html#run-fxci-to-send-mozilla-central-tasks-to-the-staging-cluster for more details. Recent FirefoxCI pushes are tested against Stage-TC, and therefore Stage-TC Worker Manager's AWS Provider AWS account (710952102342) needs read/launch permission to a subset of FirefoxCI Worker Manager's AWS account (692406183521) production images. Either: 1) both OCC and the docker-worker AMI generation process should grant access to the worker AMIs they generate in aws account 692406183521, to aws account 710952102342 after creating them, or 2) the [fxci tool](https://hg.mozilla.org/ci/ci-configuration/file/tip/src/fxci) that currently copies taskcluster worker pool definitions, and secrets etc between [FirefoxCI](https://firefox-ci-tc.services.mozilla.com) and [Stage-TC](https://stage.taskcluster.nonprod.cloudops.mozgcp.net) should take care of sharing the worker AMIs I would propose option 2). The fxci tool already synchronises state between the two clusters, and this would nicely collocate all of the the synchronisation code to a single location. For example, if at runtime it was discovered that certain images hadn't been shared between the aws accounts, it is a harder problem to solve when a separate process is required to share the images. The fxci tool could simply scan the worker pool definitions that it has copied across to staging, extracts the AMI ids, and then execute the aws commands to ensures they are shared with the staging account. I'm not sure which subset of machine images from scm level 1,2,3 need to be shared, but that should be determinable from the above linked docs, or from the Release Engineering team, if it isn't clear.
Currently Release Engineering perform smoke testing on the Stage-TC environment https://stage.taskcluster.nonprod.cloudops.mozgcp.net when there is a new release of taskcluster. See http://docs.mozilla-releng.net/en/latest/taskcluster/tc_staging.html#run-fxci-to-send-mozilla-central-tasks-to-the-staging-cluster for more details. Recent FirefoxCI pushes are tested against Stage-TC, and therefore Stage-TC Worker Manager's AWS Provider AWS account (710952102342) needs read/launch permission to a subset of FirefoxCI Worker Manager's AWS account (692406183521) production images. Either: 1) both OCC and the docker-worker AMI generation process should grant access to the worker AMIs they generate in aws account 692406183521, to aws account 710952102342 after creating them, or 2) the [fxci tool](https://hg.mozilla.org/ci/ci-configuration/file/tip/src/fxci) that currently copies taskcluster worker pool definitions, and secrets etc between [FirefoxCI](https://firefox-ci-tc.services.mozilla.com) and [Stage-TC](https://stage.taskcluster.nonprod.cloudops.mozgcp.net) should take care of sharing the worker AMIs I would propose option 2). The fxci tool already synchronises state between the two clusters, and this would nicely collocate all of the the synchronisation code to a single location. For example, if at runtime it was discovered that certain images hadn't been shared between the aws accounts, it is a harder problem to solve when a separate process is required to share the images. The fxci tool could simply scan the worker pool definitions that it has copied across to staging, extracts the AMI ids, and then execute the aws commands to ensures they are shared with the staging account. I'm not sure which subset of machine images from scm level 1,2,3 need to be shared, but that should be determinable from the above linked docs, or from the Release Engineering team, if it isn't clear.
Currently Release Engineering perform smoke testing on the Stage-TC environment https://stage.taskcluster.nonprod.cloudops.mozgcp.net when there is a new release of taskcluster. See http://docs.mozilla-releng.net/en/latest/taskcluster/tc_staging.html#run-fxci-to-send-mozilla-central-tasks-to-the-staging-cluster for more details. Recent FirefoxCI pushes are tested against Stage-TC, and therefore Stage-TC Worker Manager's AWS Provider AWS account (710952102342) needs read/launch permission to a subset of FirefoxCI Worker Manager's AWS account (692406183521) production images. Either: 1) both OCC and the docker-worker AMI generation process should grant access to the worker AMIs they generate in aws account 692406183521, to aws account 710952102342 after creating them, or 2) the [fxci tool](https://hg.mozilla.org/ci/ci-configuration/file/tip/src/fxci) that currently copies taskcluster worker pool definitions, and secrets etc between [FirefoxCI](https://firefox-ci-tc.services.mozilla.com) and [Stage-TC](https://stage.taskcluster.nonprod.cloudops.mozgcp.net) should take care of sharing the worker AMIs I would propose option 2). The fxci tool already synchronises state between the two clusters, and this would nicely collocate all of the the synchronisation code to a single location. For example, if at runtime it was discovered that certain images hadn't been shared between the aws accounts, it is a harder problem to solve when a separate process is required to share the images. The fxci tool could simply scan the worker pool definitions that it has copied across to staging, extract the AMI IDs, and then execute aws commands to ensure the aws staging account has appropriate access to launch instances using those AMIs. I'm not sure which subset of machine images from scm level 1,2,3 need to be shared, but that should be determinable from the above linked docs, or from the Release Engineering team, if it isn't clear.
Currently Release Engineering perform smoke testing on the Stage-TC environment https://stage.taskcluster.nonprod.cloudops.mozgcp.net when there is a new release of taskcluster. See http://docs.mozilla-releng.net/en/latest/taskcluster/tc_staging.html#run-fxci-to-send-mozilla-central-tasks-to-the-staging-cluster for more details. Recent FirefoxCI pushes are tested against Stage-TC, and therefore Stage-TC Worker Manager's AWS Provider AWS account (710952102342) needs read/launch permission to a subset of FirefoxCI Worker Manager's AWS account (692406183521) production images. Either: 1) both OCC and the docker-worker AMI generation process should grant access to the worker AMIs they generate in aws account 692406183521, to aws account 710952102342 after creating them, or 2) the [fxci tool](https://hg.mozilla.org/ci/ci-configuration/file/tip/src/fxci) that currently copies taskcluster worker pool definitions, etc[1] between [FirefoxCI](https://firefox-ci-tc.services.mozilla.com) and [Stage-TC](https://stage.taskcluster.nonprod.cloudops.mozgcp.net) should take care of sharing the worker AMIs I would propose option 2). The fxci tool already synchronises state between the two clusters, and this would nicely collocate all of the the synchronisation code to a single location. For example, if at runtime it was discovered that certain images hadn't been shared between the aws accounts, it is a harder problem to solve when a separate process is required to share the images. The fxci tool could simply scan the worker pool definitions that it has copied across to staging, extract the AMI IDs, and then execute aws commands[2] to ensure the aws staging account has appropriate access to launch instances using those AMIs. I'm not sure which subset of machine images from scm level 1,2,3 need to be shared, but that should be determinable from the above linked docs, or from the Release Engineering team, if it isn't clear. --- [1] Based on [this](https://mozilla-hub.atlassian.net/browse/FCP-53?focusedCommentId=520014) it may be that https://hg.mozilla.org/build/braindump/file/8b5a9f4328615aef1635cfe16c86b430e8a40b4b/taskcluster/copy_secrets_to_staging.py is used for copying secrets to staging, rather than fxci, I'm not sure. [2] The command to share an AMI is: ``` aws --region "${REGION}" ec2 modify-image-attribute --image-id "${AMI_ID}" --launch-permission 'Add=[{UserId=710952102342}]' ```
Currently Release Engineering perform smoke testing on the [Stage-TC environment](https://stage.taskcluster.nonprod.cloudops.mozgcp.net) when there is a new release of taskcluster. See http://docs.mozilla-releng.net/en/latest/taskcluster/tc_staging.html#run-fxci-to-send-mozilla-central-tasks-to-the-staging-cluster for more details. Recent FirefoxCI pushes are tested against Stage-TC, and therefore Stage-TC Worker Manager's AWS Provider AWS account (710952102342) needs read/launch permission to a subset of FirefoxCI Worker Manager's AWS account (692406183521) production images. Either: 1) both OCC and the docker-worker AMI generation process should grant access to the worker AMIs they generate in aws account 692406183521, to aws account 710952102342 after creating them, or 2) the [fxci tool](https://hg.mozilla.org/ci/ci-configuration/file/tip/src/fxci) that currently copies taskcluster worker pool definitions, etc[1] between [FirefoxCI](https://firefox-ci-tc.services.mozilla.com) and [Stage-TC](https://stage.taskcluster.nonprod.cloudops.mozgcp.net) should take care of sharing the worker AMIs I would propose option 2). The fxci tool already synchronises state between the two clusters, and this would nicely collocate all of the the synchronisation code to a single location. For example, if at runtime it was discovered that certain images hadn't been shared between the aws accounts, it is a harder problem to solve when a separate process is required to share the images. The fxci tool could simply scan the worker pool definitions that it has copied across to staging, extract the AMI IDs, and then execute aws commands[2] to ensure the aws staging account has appropriate access to launch instances using those AMIs. I'm not sure which subset of machine images from scm level 1,2,3 need to be shared, but that should be determinable from the above linked docs, or from the Release Engineering team, if it isn't clear. --- [1] Based on [this](https://mozilla-hub.atlassian.net/browse/FCP-53?focusedCommentId=520014) it may be that https://hg.mozilla.org/build/braindump/file/8b5a9f4328615aef1635cfe16c86b430e8a40b4b/taskcluster/copy_secrets_to_staging.py is used for copying secrets to staging, rather than fxci, I'm not sure. [2] The command to share an AMI is: ``` aws --region "${REGION}" ec2 modify-image-attribute --image-id "${AMI_ID}" --launch-permission 'Add=[{UserId=710952102342}]' ```
Currently Release Engineering perform smoke testing on the [Stage-TC environment](https://stage.taskcluster.nonprod.cloudops.mozgcp.net) when there is a new release of taskcluster. See http://docs.mozilla-releng.net/en/latest/taskcluster/tc_staging.html#run-fxci-to-send-mozilla-central-tasks-to-the-staging-cluster for more details. Recent FirefoxCI pushes are tested against Stage-TC, and therefore Stage-TC Worker Manager's AWS Provider AWS account (710952102342) needs read/launch permission to a subset of FirefoxCI Worker Manager's AWS account (692406183521) production images. Either: 1) Both OCC and the docker-worker AMI generation process should grant access to the worker AMIs they generate in aws account 692406183521, to aws account 710952102342 after creating them, or 2) The [fxci tool](https://hg.mozilla.org/ci/ci-configuration/file/tip/src/fxci) that currently copies taskcluster worker pool definitions, etc[1] between [FirefoxCI](https://firefox-ci-tc.services.mozilla.com) and [Stage-TC](https://stage.taskcluster.nonprod.cloudops.mozgcp.net) should take care of sharing the worker AMIs I would propose option 2). The fxci tool already synchronises state between the two clusters, and this would nicely collocate all of the the synchronisation code to a single location. For example, if at runtime it was discovered that certain images hadn't been shared between the aws accounts, it is a harder problem to solve when a separate process is required to share the images. The fxci tool could simply scan the worker pool definitions that it has copied across to staging, extract the AMI IDs, and then execute aws commands[2] to ensure the aws staging account has appropriate access to launch instances using those AMIs. I'm not sure which subset of machine images from scm level 1,2,3 need to be shared, but that should be determinable from the above linked docs, or from the Release Engineering team, if it isn't clear. --- [1] Based on [this](https://mozilla-hub.atlassian.net/browse/FCP-53?focusedCommentId=520014) it may be that https://hg.mozilla.org/build/braindump/file/8b5a9f4328615aef1635cfe16c86b430e8a40b4b/taskcluster/copy_secrets_to_staging.py is used for copying secrets to staging, rather than fxci, I'm not sure. [2] The command to share an AMI is: ``` aws --region "${REGION}" ec2 modify-image-attribute --image-id "${AMI_ID}" --launch-permission 'Add=[{UserId=710952102342}]' ```
Currently Release Engineering perform smoke testing on the [Stage-TC environment](https://stage.taskcluster.nonprod.cloudops.mozgcp.net) when there is a new release of taskcluster. See http://docs.mozilla-releng.net/en/latest/taskcluster/tc_staging.html#run-fxci-to-send-mozilla-central-tasks-to-the-staging-cluster for more details. Recent FirefoxCI pushes are tested against Stage-TC, and therefore Stage-TC Worker Manager's AWS Provider AWS account (710952102342) needs read/launch permission to a subset of FirefoxCI Worker Manager's AWS account (692406183521) production images. Either: 1) Both OCC and the docker-worker AMI generation process should grant access to the worker AMIs they generate in aws account 692406183521, to aws account 710952102342 after creating them, or 2) The [fxci tool](https://hg.mozilla.org/ci/ci-configuration/file/tip/src/fxci) that currently copies taskcluster worker pool definitions, etc[1] between [FirefoxCI](https://firefox-ci-tc.services.mozilla.com) and [Stage-TC](https://stage.taskcluster.nonprod.cloudops.mozgcp.net) should take care of sharing the worker AMIs. I would propose option 2). The fxci tool already synchronises state between the two clusters, and this would nicely collocate all of the the synchronisation code to a single location. For example, if at runtime it was discovered that certain images hadn't been shared between the aws accounts, it is a harder problem to solve when a separate process is required to share the images. The fxci tool could simply scan the worker pool definitions that it has copied across to staging, extract the AMI IDs, and then execute aws commands[2] to ensure the aws staging account has appropriate access to launch instances using those AMIs. I'm not sure which subset of machine images from scm level 1,2,3 need to be shared, but that should be determinable from the above linked docs, or from the Release Engineering team, if it isn't clear. --- [1] Based on [this](https://mozilla-hub.atlassian.net/browse/FCP-53?focusedCommentId=520014) it may be that https://hg.mozilla.org/build/braindump/file/8b5a9f4328615aef1635cfe16c86b430e8a40b4b/taskcluster/copy_secrets_to_staging.py is used for copying secrets to staging, rather than fxci, I'm not sure. [2] The command to share an AMI is: ``` aws --region "${REGION}" ec2 modify-image-attribute --image-id "${AMI_ID}" --launch-permission 'Add=[{UserId=710952102342}]' ```