Closed
Bug 1423801
Opened 6 years ago
Closed 6 years ago
Create sccache buckets for comm-central derived builds.
Categories
(Taskcluster :: Operations and Service Requests, task)
Taskcluster
Operations and Service Requests
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: tomprince, Assigned: tomprince)
References
Details
Attachments
(2 files, 1 obsolete file)
I'd like to enable sccache for thunderbird builds, and will need level-1 and level-3 S3 buckets for that.
Flags: needinfo?(jopsen)
Comment 1•6 years ago
|
||
Sorry, for being slow to get on with this... I've started to create some bucket, managed with terraform... Anyways, I filed with AWS to get a bucket increase as we need more buckets :)
Comment 2•6 years ago
|
||
Created buckets: comm-central-level-1-sccache-eu-central-1 comm-central-level-2-sccache-eu-central-1 comm-central-level-3-sccache-eu-central-1 comm-central-level-1-sccache-us-east-2 comm-central-level-2-sccache-us-east-2 comm-central-level-3-sccache-us-east-2 comm-central-level-1-sccache-us-west-1 comm-central-level-2-sccache-us-west-1 comm-central-level-3-sccache-us-west-1 comm-central-level-1-sccache-us-west-2 comm-central-level-2-sccache-us-west-2 comm-central-level-3-sccache-us-west-2 Attached to taskcluster-auth, so credentials can be created using: https://docs.taskcluster.net/reference/platform/taskcluster-auth/references/api#awsS3Credentials Also attached to the same IAM roles as gecko level-1 and level-3. But I'm guessing comm-central won't be running on gecko level-3 builders, so that might not be important. @dustin, we should grant the scope: auth:aws-s3:read-write:<bucket>/* To the right appropriate comm-central roles... I'm guessing you have try-like thing that needs the level-1 scopes and some repo role that needs level-3. These seems to be tc-admin managed, what would you suggest?
Flags: needinfo?(jopsen) → needinfo?(dustin)
Comment 3•6 years ago
|
||
It looks like the way these are managed for gecko is that the auth scopes are defined in roles `project:taskcluster:level-L-sccache-buckets` https://tools.taskcluster.net/auth/roles/project%3Ataskcluster%3Alevel-1-sccache-buckets That role is then assumed in `moz-tree:level:N` https://tools.taskcluster.net/auth/roles/moz-tree%3Alevel%3A1 and that role is, in turn, assumed in `repo:hg.mozilla.org/try:*` https://tools.taskcluster.net/auth/roles/repo%3Ahg.mozilla.org%2Ftry%3A* There is a try-comm-central repo: https://tools.taskcluster.net/auth/roles/repo%3Ahg.mozilla.org%2Ftry-comm-central%3A* and that, too, assumes moz-tree:level:1 If we do not want to blaze a new trail here, we can just assign the scopes to `project:taskcluster:level-L-sccache-buckets`. If, as I suspect, the idea is to separate sccache by trust domain, then maybe we need to split the `moz-tree:level:L` roles by trust domain, too. We can split them but leave their scopes identical except for sccache at first, then continue to chip away a the shared resources until they are completely separated. So: which should we do?
Flags: needinfo?(dustin)
Assignee | ||
Comment 4•6 years ago
|
||
I think we do eventually want to split the roles. As a first step, maybe we want to do the following: - Create moz-tree:level:<N>:<trust-domain> and add that to the tc-admin settings. - Add project:taskcluster:comm:level-<n>-sccache-buckets to moz-tree:level:<N>:comm (under project:taskcluster since they are in the TC AWS account) - Do the same for trust-domain = gecko. (this could also wait for the new year) In the new year, clean things up by removing the role project:taskcluster:level-<n>-sccache-bucket in favor of the new roles.
Assignee | ||
Comment 5•6 years ago
|
||
https://github.com/taskcluster/taskcluster-admin/pull/10
Comment hidden (mozreview-request) |
Comment 7•6 years ago
|
||
OK, agreed. I set up `moz-tree:level:N:*` with the same scopes as `moz-tree:level:N` used to have, then added `moz-tree:level:N:gecko` and `moz-tree:level:N:comm`. The latter two implement the "cascade" where L2 implies L1 and L3 implies L2 and L1. https://tools.taskcluster.net/auth/roles/moz-tree:level:1 https://tools.taskcluster.net/auth/roles/moz-tree:level:1:* https://tools.taskcluster.net/auth/roles/moz-tree:level:1:gecko https://tools.taskcluster.net/auth/roles/moz-tree:level:1:comm https://tools.taskcluster.net/auth/roles/moz-tree:level:2 https://tools.taskcluster.net/auth/roles/moz-tree:level:2:* https://tools.taskcluster.net/auth/roles/moz-tree:level:2:gecko https://tools.taskcluster.net/auth/roles/moz-tree:level:2:comm https://tools.taskcluster.net/auth/roles/moz-tree:level:3 https://tools.taskcluster.net/auth/roles/moz-tree:level:3:* https://tools.taskcluster.net/auth/roles/moz-tree:level:3:gecko https://tools.taskcluster.net/auth/roles/moz-tree:level:3:comm Then I moved the gecko sccache roles: https://tools.taskcluster.net/auth/roles/project%3Ataskcluster%3Agecko%3Alevel-1-sccache-buckets https://tools.taskcluster.net/auth/roles/project%3Ataskcluster%3Agecko%3Alevel-2-sccache-buckets https://tools.taskcluster.net/auth/roles/project%3Ataskcluster%3Agecko%3Alevel-3-sccache-buckets and created new comm sccache roles https://tools.taskcluster.net/auth/roles/project%3Ataskcluster%3Acomm%3Alevel-1-sccache-buckets https://tools.taskcluster.net/auth/roles/project%3Ataskcluster%3Acomm%3Alevel-2-sccache-buckets https://tools.taskcluster.net/auth/roles/project%3Ataskcluster%3Acomm%3Alevel-3-sccache-buckets and finally edited the moz-tree roles to include the appropriate sccache roles.
Comment 8•6 years ago
|
||
Oops, "assume:project:taskcluster:level-N-sccache-buckets" appears directly in the scopes used in-tree. I re-added those roles as aliases to the new :gecko: roles. Tom, I'll leave it to you to make a patch to use the new role names (since that will need to include the trust domain)
Comment hidden (mozreview-request) |
Comment 10•6 years ago
|
||
I had to add those scopes to moz-tree:level:N:gecko too. That will need to be reverted eventually.
Keywords: leave-open
Comment hidden (mozreview-request) |
Comment 12•6 years ago
|
||
mozreview-review |
Comment on attachment 8938529 [details] Bug 1423801: Include trust-domain in the scopes to use for sccache; https://reviewboard.mozilla.org/r/209192/#review215012
Attachment #8938529 -
Flags: review?(dustin) → review+
Comment 13•6 years ago
|
||
Pushed by mozilla@hocat.ca: https://hg.mozilla.org/integration/autoland/rev/a931893981e5 Include trust-domain in the scopes to use for sccache; r=dustin
Comment 14•6 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/a931893981e5
Comment hidden (Intermittent Failures Robot) |
Assignee | ||
Comment 16•6 years ago
|
||
I've got this partly working in a branch. There are a couple of issues: - sccache doesn't use keys when trying to read from the cache, so the buckets need to be world-readable. - I think the list of regions doesn't match the regions that workers are in. I know at least us-east-1 has workers but no buckets.
Comment 17•6 years ago
|
||
@tomprince, okay.. I'll: - add us-east-1, and, - make them public readable
Comment hidden (mozreview-request) |
Assignee | ||
Updated•6 years ago
|
Attachment #8938465 -
Attachment is obsolete: true
Comment 20•6 years ago
|
||
mozreview-review |
Comment on attachment 8939701 [details] Bug 1423801: Use comm-central sccache buckets; https://reviewboard.mozilla.org/r/210004/#review215832
Attachment #8939701 -
Flags: review?(philipp) → review+
Comment 21•6 years ago
|
||
Pushed by mozilla@hocat.ca: https://hg.mozilla.org/comm-central/rev/678d7a3b822a Use comm-central sccache buckets; r?Fallen
Assignee | ||
Updated•6 years ago
|
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Comment 22•6 years ago
|
||
Removing leave-open keyword from resolved bugs, per :sylvestre.
Keywords: leave-open
Updated•6 years ago
|
Assignee: nobody → mozilla
Updated•5 years ago
|
Component: Service Request → Operations and Service Requests
You need to log in
before you can comment on or make changes to this bug.
Description
•