1406209 - Try to make actions/registry.py action task template use .taskcluster.yml

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Brian Stack [:bstack]]

This should be easy now that .taskcluster.yml is json-e with a tasks_for. We just need to set tasks_for to action tasks and we're good. I want this because the one in registry.py keeps getting out of date with the decision task one.

Comment 1

[Brian Stack [:bstack]]

Attachment: Bug 1406209 - Use .taskcluster.yml for action tasks templates

Review commit: https://reviewboard.mozilla.org/r/187912/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/187912/

Comment 2

[Brian Stack [:bstack]]

Comment on attachment 8916845 [details]
Bug 1406209 - Use .taskcluster.yml for action tasks templates

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/187912/diff/1-2/

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8916845 [details]
Bug 1406209 - Use .taskcluster.yml for action tasks templates

https://reviewboard.mozilla.org/r/187912/#review193180

Looks good to me, but I am sure I am failing to imagine potential issues.

::: .taskcluster.yml:22
(Diff revision 2)
> +        '${action.taskGroupId}'
> +      else:
> +        '${as_slugid("decision")}' # same as taskId; this is how automation identifies a decision tsak
>      schedulerId: 'gecko-level-${repository.level}'
>  
> -    created: {$fromNow: ''}
> +    created: {$if: 'tasks_for == "action"', then: {$$fromNow: ''}, else: {$fromNow: ''}}

This might be worth a comment :)

::: .taskcluster.yml:29
(Diff revision 2)
> -    expires: {$fromNow: '1 year 1 second'} # 1 second so artifacts expire first, despite rounding errors
> +    expires: {$if: 'tasks_for == "action"', then: {$$fromNow: '1 year 1 second'}, else: {$fromNow: '1 year 1 second'}} # 1 second so artifacts expire first, despite rounding errors
>      metadata:
>        $merge:
>          - owner: "${ownerEmail}"
>            source: "${repoUrl}/raw-file/${push.revision}/.taskcluster.yml"
>          - $if: 'tasks_for == "hg-push"'

a good use for $switch when it's finished :)

Comment 4

[Jonas Finnemann Jensen (:jonasfj)]

Comment on attachment 8916845 [details]
Bug 1406209 - Use .taskcluster.yml for action tasks templates

https://reviewboard.mozilla.org/r/187910/#review193204

::: taskcluster/taskgraph/actions/registry.py:217
(Diff revision 2)
> -                        },
> -                    },
> -                },
> +                    },
> -            }
> +                }
> +
> +                return jsone.render(taskcluster_yml, context)['tasks'][0]

Instead I propose:

```python
return {
  "$let": context,
  "in": "taskcluster_yml",
}
```

that way you won't need the `$$fromNow` construct.

Comment 5

[Brian Stack [:bstack]]

Comment on attachment 8916845 [details]
Bug 1406209 - Use .taskcluster.yml for action tasks templates

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/187912/diff/2-3/

Comment 6

[Jonas Finnemann Jensen (:jonasfj)]

Comment on attachment 8916845 [details]
Bug 1406209 - Use .taskcluster.yml for action tasks templates

https://reviewboard.mozilla.org/r/187912/#review194256

::: .taskcluster.yml:18
(Diff revision 3)
> -  - taskId: '${as_slugid("decision")}'
> -    taskGroupId: '${as_slugid("decision")}' # same as tsakId; this is how automation identifies a decision tsak
> +      taskId: {$if: 'tasks_for != "action"', then: '${as_slugid("decision")}'}
> +      taskGroupId:
> +        $if: 'tasks_for == "action"'
> +        then:
> +          '${action.taskGroupId}'
> +        else:

Nit, just formatting but using `tasks_for != "action"` both places might be nice, like:

```yml
      taskId: {$if: 'tasks_for != "action"', then: '${as_slugid("decision")}'}
      taskGroupId:
        $if: 'tasks_for != "action"'
        then: '${as_slugid("decision")}'
        else: '${action.taskGroupId}'
```

::: taskcluster/taskgraph/actions/registry.py:197
(Diff revision 3)
> +                if not isinstance(taskcluster_yml['tasks'], list):
> +                    raise Exception('.taskcluster.yml "tasks" must be a list for action tasks')
> +
> -            return {
> +                return {
> -                'created': {'$fromNow': ''},
> -                'deadline': {'$fromNow': '12 hours'},
> +                    '$let': {
> +                        'tasks_for': 'action',

If you are annoyed with the size of duplicating all this for each action... You can do:
```yml
$let:
  $merge:
    - $eval: actionVariables
    - action:
        name: ...
        ...
        cb_name: ...
```

And then in `render_actions_json()` define `variables` to contain `actionVariables` with all the values you need.

This is just a note, there is absolutely no need for doing anything like this at the moment. Since we don't have that many actions :)

Comment 7

[Mozilla Autoland]

We're sorry, Autoland could not rebase your commits for you automatically. Please manually rebase your commits and try again.

hg error in cmd: hg rebase -s e25d58d4fe58 -d d72d02a5f8ac: rebasing 426890:e25d58d4fe58 "Bug 1406209 - Use .taskcluster.yml for action tasks templates r=dustin,jonasfj" (tip)
merging build/sparse-profiles/taskgraph
merging taskcluster/taskgraph/actions/registry.py
warning: conflicts while merging build/sparse-profiles/taskgraph! (edit, then use 'hg resolve --mark')
warning: conflicts while merging taskcluster/taskgraph/actions/registry.py! (edit, then use 'hg resolve --mark')
unresolved conflicts (see hg resolve, then hg rebase --continue)

Comment 8

[Brian Stack [:bstack]]

Comment on attachment 8916845 [details]
Bug 1406209 - Use .taskcluster.yml for action tasks templates

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/187912/diff/3-4/

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/215bc8a3310c
Use .taskcluster.yml for action tasks templates r=dustin,jonasfj

Comment 10

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/215bc8a3310c