2044114 - Use Git notes to transfer Try metadata

Status: ASSIGNED

(Developer Infrastructure :: Try, task)

Description

[Andrew Halberstadt [:ahal]]

When pushing to Try, we need to pass some metadata from the user's local machine to the Decision task in CI. This contains things like a list of tasks to run, or an environment variable to set. Currently we commit this metadata into a file at the root of the repo called try_task_config.json

This works well, but it has the downside of needing to make a new commit for each try push. This has some drawbacks, like the revision is different each time which complicates sha based caching. But there's an even bigger drawback in Github:

It means what you push to try is not what ends up landing. It is fundamentally incompatible with PR-based workflows.

Instead of committing the metadata to a file, we can instead add it to a Git note. Then, we push the Git note ref to the repo, and the Decision task fetches that ref. This keeps the Try metadata out of version control and avoids us needing to change the commit sha. The downside to this, is that Github does not emit webhook events when pushing note refs to the remote repo. So we won't be able to rely on the Taskcluster Github app to parse the Decision task. Luckily we already have our own infrastructure in-place to handle this in build-decision.

Comment 1

[Andrew Halberstadt [:ahal]]

Attachment: Bug 2044114 - Support overriding parameters via Git note, r?#taskgraph-reviewers!

This ports the following change from upstream Taskgraph:
https://github.com/taskcluster/taskgraph/commit/2afefff7b52b9a8fecf4c50db6276732428dbbca

Comment 2

[Andrew Halberstadt [:ahal]]

Attachment: WIP: Bug 2044114 - Support `git-push` build-decision events in Decision tasks, r?#releng-reviewers!

Comment 3

[Andrew Halberstadt [:ahal]]

Attachment: WIP: Bug 2044114 - Use Git notes to pass Try metadata to Decision tasks, r?#releng-reviewers!

Comment 4

[Andrew Halberstadt [:ahal]]

Attachment: WIP: Bug 2044114 - Allow overriding Decision parameters on firefox-dev and staging-firefox, r?#releng-reviewers!

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/7f5762b2885c
https://hg.mozilla.org/integration/autoland/rev/e9e4a4c64b9e
Support overriding parameters via Git note, r=taskgraph-reviewers,releng-reviewers,bhearsum

Comment 6

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/e9e4a4c64b9e

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/5f6d282f6c9b
https://hg.mozilla.org/mozilla-central/rev/470c1ac19c4c
Revert "Bug 2044114 - Support overriding parameters via Git note, r=taskgraph-reviewers,releng-reviewers,bhearsum" for for not scheduling expected tasks for Try pushes, e.g. for release simulations

Comment 8

[Silaghi Andreea]

Backed out on Aryx request for not scheduling expected tasks for Try pushes, e.g. for release simulations.
https://hg.mozilla.org/mozilla-central/rev/470c1ac19c4c35d268348b5c9aba9f5edc1b4430