1324094 - Investigate adding new scopes to action tasks for caches above level-1

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Brian Stack [:bstack]]

This is needed for backfilling on repos other than try. More discussion in bug 1289823 comment 23.

Comment 1

[Brian Stack [:bstack]]

Attachment: Bug 1324094 - Add correct cache scopes to action tasks

Review commit: https://reviewboard.mozilla.org/r/99198/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/99198/

Comment 2

[Brian Stack [:bstack]]

Landing this should probably be blocked on bug 1284989.

Comment 3

[Brian Stack [:bstack]]

After running this from the task creator with the appropriate scopes, it successfully backfilled bc5 tests as seen in https://treeherder.mozilla.org/#/jobs?repo=autoland&filter-searchStr=linux%2064%20pgo%20tc%20mochitest&tochange=3d4f10649284edd6397e57052411b213cb24ca62

Comment 4

[Brian Stack [:bstack]]

Comment on attachment 8819404 [details]
Bug 1324094 - Add correct cache scopes to action tasks

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/99198/diff/1-2/

Comment 5

[Armen [:armenzg]]

Comment on attachment 8819404 [details]
Bug 1324094 - Add correct cache scopes to action tasks

https://reviewboard.mozilla.org/r/99198/#review99592

As long as dustin is happy with it :)

Comment 6

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8819404 [details]
Bug 1324094 - Add correct cache scopes to action tasks

https://reviewboard.mozilla.org/r/99198/#review99980

Aside from the comments below, there are security implications to supporting this on more than the try repo.  I think you already have those in mind, based on other bug traffic I've seen go by, but it's worth mentioning again :)

::: taskcluster/taskgraph/action.yml:21
(Diff revision 2)
> +  - "docker-worker:cache:level-2-*"
> +  - "docker-worker:cache:level-3-*"

These are more than a level-1 action task would need.

Also, these scopes shouldn't be needed anymore now that bug 1269443 is landed.  They are included in the `repo:hg.mozilla.org/try:*` role.

I think the right fix here is for the decision task to dynamically calculate the `repo:..` role for the action.yml file that it uploads as an artifact.

::: taskcluster/taskgraph/action.yml:26
(Diff revision 2)
> +  - "queue:route:tc-treeherder.v2.*"
> +  - "queue:route:tc-treeherder-stage.v2.*"

I'm not sure why these were added..

Comment 7

[Brian Stack [:bstack]]

Comment on attachment 8819404 [details]
Bug 1324094 - Add correct cache scopes to action tasks

https://reviewboard.mozilla.org/r/99198/#review100338

::: taskcluster/taskgraph/action.yml:26
(Diff revision 2)
> +  - "queue:route:tc-treeherder.v2.*"
> +  - "queue:route:tc-treeherder-stage.v2.*"

These were added because of the failure I saw in https://tools.taskcluster.net/task-inspector/#H0GH4bWrTCOmjPLyQ_oBcA/0

Perhaps they will not be needing if we do the assume:repo:hg.mozilla... change you mentioned in the other issue. I'll try it out and see.

Comment 8

[Brian Stack [:bstack]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=e86da23a8fc0

Comment 9

[Brian Stack [:bstack]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=4e6d225a4aed

Comment 10

[Brian Stack [:bstack]]

Comment on attachment 8819404 [details]
Bug 1324094 - Add correct cache scopes to action tasks

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/99198/diff/2-3/

Comment 11

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8819404 [details]
Bug 1324094 - Add correct cache scopes to action tasks

https://reviewboard.mozilla.org/r/99198/#review100652

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/02cbb6db8bd2
Add correct cache scopes to action tasks r=armenzg,dustin

Comment 13

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/02cbb6db8bd2

Comment 14

[Brian Stack [:bstack]]

Now that this is merged, we should be able to backfill so long as we assign scopes to https://tools.taskcluster.net/auth/clients/#project%252fateam%252fpulse_actions

I believe we've decided that as a temporary measure until treeherder supports tc credentials fully, we will give that client "assume:repo:hg.mozilla.org/*". I believe that is an extremely powerful set of scopes, but I also understand that backfilling for these jobs is something we want to have sooner rather than later. If you're ok with it, I'll go ahead and do so.

Comment 16

[Dustin J. Mitchell [:dustin] (he/him)]

Aww, TIL don't use unicode in Bugzilla.  Ship it :)

Comment 17

[Armen [:armenzg]]

Could we file a bug for adding that ability to TH and another bug for reverting giving these privileges?

Is this part of the outcome of the RRA? Is that bug ready to be fixed?

Comment 18

[Brian Stack [:bstack]]

I've created bug 1325657 for the removal of the scopes. Bug 1273096 is for doing things with tc creds correctly in th and is underway now. Bug 1278986 will also need to be fixed before we can move forward with getting rid of these scopes (and eventually all tc interactions from pulse_actions).

Comment 19

[Ed Morley [:emorley]]

(Moving to a Taskcluster component, since the changed files were taskcluster config files in mozilla-central rather than in the Treeherder repo)