1481203 - Cannot mark Thunderbird 60.0 as shipped: No valid scope found

Status: RESOLVED FIXED

(Thunderbird :: Build Config, defect)

Description

[Johan Lorenzo [:jlorenzo]]

https://tools.taskcluster.net/groups/B5SChkanQf2ahu0Kr1POgQ/tasks/a16X_sdwSxKSnErZgCb6oQ/runs/1/logs/public%2Flogs%2Flive_backing.log:

> 2018-08-06 12:38:58,287 - scriptworker.client - ERROR - Failed to run async_main
> Traceback (most recent call last):
>   File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/client.py", line 203, in _handle_asyncio_loop
>     await async_main(context)
>   File "/builds/scriptworker/lib/python3.6/site-packages/shipitscript/script.py", line 20, in async_main
>     validate_task_schema(context)
>   File "/builds/scriptworker/lib/python3.6/site-packages/shipitscript/task.py", line 42, in validate_task_schema
>     action = get_task_action(context)
>   File "/builds/scriptworker/lib/python3.6/site-packages/shipitscript/task.py", line 49, in get_task_action
>     scope = _get_scope(context, "action")
>   File "/builds/scriptworker/lib/python3.6/site-packages/shipitscript/task.py", line 26, in _get_scope
>     too_many_item_error_message='More than one valid scope given',
>   File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/utils.py", line 700, in get_single_item_from_sequence
>     raise ErrorClass(error_message)
> scriptworker.exceptions.TaskVerificationError: No valid scope found. Task must have a scope that starts with "project:comm:thunderbird:releng:ship-it:action". Given: ['project:comm:thunderbird:releng:ship-it:production']

We're missing the action scope.

Comment 1

[Johan Lorenzo [:jlorenzo]]

In the meantime, I manually marked the release as shipped.

Comment 2

[Mihai Tabara [:mtabara]⌚️GMT]

(In reply to Johan Lorenzo [:jlorenzo] from comment #1)
> In the meantime, I manually marked the release as shipped.

Yeah, sounds like in-tree payload misconfig. Like Johan said, we need "project:comm:thunderbird:releng:ship-it:action:mark-as-shipped" to be added both in-tree in the task payload and also in the relpro TC client[1].

[1]: https://tools.taskcluster.net/auth/clients/project%2Freleng%2Freleasepromotion%2Fproduction

Comment 3

[Tom Prince [:tomprince]]

This broke again for Thunderbird 60.0b11. (I've marked it as shipped manually, again)

Comment 4

[Rob Lemley [:rjl]]

This file will need adjusting, look at the Mozilla version to see what needs to be done.
https://hg.mozilla.org/releases/comm-esr60/file/tip/taskcluster/ci/release-mark-as-shipped/kind.yml

Comment 5

[Rob Lemley [:rjl]]

Attachment: Bug 1481203 - Add mark-as-shipped scope to release-mark-as-shipped kind

This will fix the problem of not being able to mark a release as shipped.

Comment 6

[Jorg K (CEST = GMT+2)]

Comment on attachment 9013137 [details]
Bug 1481203 - Add mark-as-shipped scope to release-mark-as-shipped kind

Jorg K (GMT+2) has approved the revision.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/5488135778be
Add mark-as-shipped scope to release-mark-as-shipped kind. rs=jorgk

Comment 8

[Jorg K (CEST = GMT+2)]

TB 60.2.1 ESR:
https://hg.mozilla.org/releases/comm-esr60/rev/92e42fc308f00260956a3650ebe95067a20e46f5

Comment 9

[Jorg K (CEST = GMT+2)]

Beta uplift here?

Comment 10

[Rob Lemley [:rjl]]

Yes we will need these in beta.

Comment 11

[Jorg K (CEST = GMT+2)]

Beta (TB 63):
https://hg.mozilla.org/releases/comm-beta/rev/d183c526eb855957e755841fba24748a1d390881

Comment 12

[Rob Lemley [:rjl]]

Task failed again on 63.0b1.

Need to change
'project:comm:thunderbird:releng:ship-it:production'
to
'project:comm:thunderbird:releng:ship-it:server:production'

2018-10-11 20:09:48,745 - scriptworker.client - ERROR - Failed to run async_main
Traceback (most recent call last):
  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/client.py", line 203, in _handle_asyncio_loop
    await async_main(context)
  File "/builds/scriptworker/lib/python3.6/site-packages/shipitscript/script.py", line 22, in async_main
    context.ship_it_instance_config = get_ship_it_instance_config_from_scope(context)
  File "/builds/scriptworker/lib/python3.6/site-packages/shipitscript/task.py", line 31, in get_ship_it_instance_config_from_scope
    scope = _get_scope(context, "server")
  File "/builds/scriptworker/lib/python3.6/site-packages/shipitscript/task.py", line 26, in _get_scope
    too_many_item_error_message='More than one valid scope given',
  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/utils.py", line 700, in get_single_item_from_sequence
    raise ErrorClass(error_message)
scriptworker.exceptions.TaskVerificationError: No valid scope found. Task must have a scope that starts with "project:comm:thunderbird:releng:ship-it:server". Given: ['project:comm:thunderbird:releng:ship-it:production', 'project:comm:thunderbird:releng:ship-it:action:mark-as-shipped']

Comment 13

[Rob Lemley [:rjl]]

Attachment: Bug 1481203 - Fix server scopes on release-mark-as-shipped task. r?tomprince

Error message asks for a scope
"project:comm:thunderbird:releng:ship-it:server" in additon to
"project:comm:thunderbird:releng:ship-it:action:mark-as-shipped"

Comment 14

[Rob Lemley [:rjl]]

Attachment: ci-configuration-tb-grants.patch

This patch needs to be applied to the ci-configuration repository and adds the addition scope grants necessary for the latest changes to Taskcluster.

Comment 15

[Tom Prince [:tomprince]]

Can you upload the ci-config patch to phabricator?

Comment 16

[Rob Lemley [:rjl]]

Attachment: Bug 1481203 - Add new release-mark-as-shipped scopes for Thunderbird r?dustin

The Thunderbird release-mark-as-shipped task fails due to a scope problem.
Task must have a scope that starts with
"project:comm:thunderbird:releng:ship-it:server",
yet there is no such scope.

{scope_prefix}:server:staging and {scope_prefix}:server:production are not
present at
Per https://github.com/mozilla-releng/shipitscript/blob/master/SCOPES.md,

The scopes for :releng:ship-it are not listed at:
https://tools.taskcluster.net/auth/clients/project%2Freleng%2Freleasepromotion%2Fproduction

This patch adds them.

Comment 17

[Rob Lemley [:rjl]]

Attachment: Fix release-mark-as-shipped server scopes - BETA/ESR60 version

comm-beta is using an older revision of release-mark-as-shipped/kind.yml as it's base and the patch for c-c will not apply. As it is not feasible to update kind.yml due to "by-release-level" not being available in taskgraph yet, here is a version of the patch that applies to comm-beta.

Comment 18

[Rob Lemley [:rjl]]

Comment on attachment 9016824 [details] [diff] [review]
Fix release-mark-as-shipped server scopes - BETA/ESR60 version

This will also work for comm-esr60.

Comment 19

[Rob Lemley [:rjl]]

https://phabricator.services.mozilla.com/D8481 is ready to land.

Comment 20

[Jorg K (CEST = GMT+2)]

Sorry, I can't get "hg phabread" set up right now, see bug 1499305, so I can't land this at the moment.

Comment 21

[Jorg K (CEST = GMT+2)]

I finally got "hg phabread" going again. Sadly the patch doesn't apply, just look at it:

diff --git a/grants.yml b/grants.yml
--- a/grants.yml
+++ b/grants.yml

As far as I can tell, "grants.yml" doesn't exist anywhere in the entire codebase, neither M-C nor C-C. The beta version applies to taskcluster/ci/release-mark-as-shipped/kind.yml

Comment 22

[Rob Lemley [:rjl]]

The grants.yml patch was for the ci-configuration repository, and that's already landed separately.

Attachment 9016492 [details] applies to C-C (https://phabricator.services.mozilla.com/D8481).

Attachment 9016824 [details] [diff] should apply to both beta and esr60.

I don't know of a way to put patches that differ for beta and esr in phabricator, that's why that one is here.

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/229c5a7bba53
Fix server scopes on release-mark-as-shipped task. r=mkmelin

Comment 24

[Jorg K (CEST = GMT+2)]

TB 63 beta 2:
https://hg.mozilla.org/releases/comm-beta/rev/1b7d33c8b83204292e9b6f6b57e4a769c8065c9a

Comment 25

[Jorg K (CEST = GMT+2)]

TB 60.3:
https://hg.mozilla.org/releases/comm-esr60/rev/783d2480b609c753984c2a9e3a3d3af1376109c0

Comment 26

[Rob Lemley [:rjl]]

New ship-it error:
Traceback (most recent call last):
  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/client.py", line 203, in _handle_asyncio_loop
    await async_main(context)
  File "/builds/scriptworker/lib/python3.6/site-packages/shipitscript/script.py", line 22, in async_main
    context.ship_it_instance_config = get_ship_it_instance_config_from_scope(context)
  File "/builds/scriptworker/lib/python3.6/site-packages/shipitscript/task.py", line 37, in get_ship_it_instance_config_from_scope
    raise TaskVerificationError('This worker is not configured to handle scope "{}"'.format(scope))
scriptworker.exceptions.TaskVerificationError: This worker is not configured to handle scope "project:comm:thunderbird:releng:ship-it:server:production"

Per :tomprince via IRC, the Taskcluster scriptworkers need to be updated via puppet to recognize the new scopes. I've submitted a pull request at: https://github.com/mozilla-releng/build-puppet/pull/278

Comment 27

[Rob Lemley [:rjl]]

This ran successfully on the 60.3 release