1829648 - Add scopes the application-services cron task

Status: RESOLVED FIXED

(Release Engineering :: Firefox-CI Administration, task)

Description

[Ben Dean-Kawamura [:bdk]]

After merging our PR to implement nightly builds, our nightly cron tasks have been failing with a permissions errors. It looks like we need to grant it the secrets:get:project/application-services/symbols-token scope.

Comment 1

[Julien Cristau [:jcristau]]

Attachment: Bug 1829648 - add scope for application-services nightly cron. r?#releng

The nightly cron job now runs tasks that require
secrets:get:project/application-services/symbols-token.

Comment 2

[Julien Cristau [:jcristau]]

Attachment: Bug 1829648 - add hook for application-services nightly cron job. r?#releng

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/ci/ci-configuration/rev/1f0490d79528
add scope for application-services nightly cron. r=releng-reviewers,bhearsum
https://hg.mozilla.org/ci/ci-configuration/rev/7d6da6530dd2
add hook for application-services nightly cron job. r=releng-reviewers,bhearsum

Comment 4

[Julien Cristau [:jcristau]]

Attachment: Bug 1829648 - cleanup application-services release scopes. r?#releng

The appservices-{level}-beetmover worker type was used before
https://github.com/mozilla/application-services/pull/3168; since then
it's changed to app-services-{level}-beetmover. It's high time we
removed the corresponding grant.

Comment 5

[Julien Cristau [:jcristau]]

Attachment: Bug 1829648 - clean up old secrets for application-services. r?#releng!,bdk!

I haven't been able to find where
project/application-services/gradle-plugin-publish was used;
project/application-services/publish has been unused since the switch to
beetmover in https://github.com/mozilla/application-services/pull/744.

Both pre-date ci-config and were migrated from the previous manually
maintained config.

Comment 6

[Julien Cristau [:jcristau]]

Attachment: Bug 1829648 - grant release-signing and beetmover scopes to application-services' nightly cron. r?#releng

application-services is now publishing daily artifacts from cron so
needs access to maven.mozilla.org from non-github-release graphs.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/ci/ci-configuration/rev/0f2616cb9673
grant release-signing and beetmover scopes to application-services' nightly cron. r=releng-reviewers,ahal,gbrown

Comment 8

[Julien Cristau [:jcristau]]

I'm abusing this bug to clean up some obsolete grants from A-S release jobs while we're looking at this.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/ci/ci-configuration/rev/8c7f1f7adef0
cleanup application-services release scopes. r=releng-reviewers,gbrown
https://hg.mozilla.org/ci/ci-configuration/rev/d2b8ae6b553e
clean up old secrets for application-services. r=bdk,releng-reviewers,gbrown

Comment 10

[Julien Cristau [:jcristau]]

Cron job triggered manually, https://firefox-ci-tc.services.mozilla.com/tasks/NLb8d4JJTLOx06xz7HIcEQ is green; closing.

Comment 11

[Ben Dean-Kawamura [:bdk]]

The decision task succeeded, however the signing tasks seem to have failed with the MALFORMED_PAYLOAD exception. Any idea why those tasks are failing?

https://firefox-ci-tc.services.mozilla.com/tasks/groups/NLb8d4JJTLOx06xz7HIcEQ

Comment 12

[Ben Dean-Kawamura [:bdk]]

FWIW, this task was working when running from a PR 4 days ago: https://firefox-ci-tc.services.mozilla.com/tasks/FuQ3IbAwQCWFRHgkCr-ySQ

Comment 13

[Andrew Halberstadt [:ahal]]

That's a CoT error, see the chain_of_trust.log to see the actual error message.

It's happening because Chain of Trust is hitting an exception while trying to rebuild and verify the Decision task's definition. Looks like it's expecting a task["extra"]["cron"] value in the definition. Looks like app-services is missing this bit of config in the .tc.yml:
https://github.com/mozilla-mobile/firefox-android/blob/main/.taskcluster.yml#L347

Comment 14

[Julien Cristau [:jcristau]]

2023-04-24T16:03:00    ERROR - Error while rebuilding scriptworker:parent NLb8d4JJTLOx06xz7HIcEQ task definition!
Traceback (most recent call last):
  File "/app/lib/python3.9/site-packages/scriptworker/cot/verify.py", line 1569, in verify_parent_task_definition
    jsone_context, tmpl = await get_jsone_context_and_template(chain, parent_link, decision_link, tasks_for)
  File "/app/lib/python3.9/site-packages/scriptworker/cot/verify.py", line 1539, in get_jsone_context_and_template
    jsone_context = await populate_jsone_context(chain, parent_link, decision_link, tasks_for)
  File "/app/lib/python3.9/site-packages/scriptworker/cot/verify.py", line 1312, in populate_jsone_context
    jsone_context.update(await _get_additional_git_cron_jsone_context(decision_link))
  File "/app/lib/python3.9/site-packages/scriptworker/cot/verify.py", line 1139, in _get_additional_git_cron_jsone_context
    "cron": load_json_or_yaml(decision_link.task["extra"]["cron"]),
KeyError: 'cron'
2023-04-24T16:03:00 CRITICAL - Chain of Trust verification error!

Chain-of-trust verification expects something like https://github.com/mozilla-mobile/firefox-android/blob/a4a2449c2037b4b84d7ecee3cf07366c4986b767/.taskcluster.yml#L349 for cron decision tasks, which is missing from https://github.com/mozilla/application-services/blob/main/.taskcluster.yml.

Comment 15

[Ben Dean-Kawamura [:bdk]]

After fixing the COT errors, the build works. Thanks again!

https://firefox-ci-tc.services.mozilla.com/tasks/groups/LXzVhwIKRSG3voC3YDD3nQ