Taskgraph - supply schema for "Parameters"

RESOLVED FIXED

Status

enhancement
RESOLVED FIXED
4 months ago
2 months ago

People

(Reporter: mhentges, Assigned: mhentges)

Tracking

unspecified
Dependency tree / graph

Firefox Tracking Flags

(firefox69 fixed)

Details

Attachments

(1 attachment)

As part of adding git support for taskgraph, we can no longer uniformly define all taskgraph "parameters" as required (for example, pushlog_id isn't relevant for git clones).

This will leverage taskgraph's schema tooling to validate parameters and allow conditionally required values.

Blocks: 1531936

Tom, would you recommend making this change to the extracted taskgraph, then porting it to in-tree, or the other way around?

Flags: needinfo?(mozilla)

My general workflow has to get the version targeted at mozilla-central reviewed and landed first, and then land on ci/taskgraph. At the moment, the mozilla-central copy is critical code, and ci/taskgraph less so, so the review there is more critical, and there are more stakeholders.

For development, I've sometime worked on ci/taskgraph first, but ported to mozilla-central for review, before porting back to ci/taskgraph to land.

Flags: needinfo?(mozilla)

I've tested this with ./mach python-test taskcluster, as well as playing with ./mach taskgraph tasks.
I'm very open to recommendations for further testing or workflow :)

Attachment #9053018 - Attachment description: Specifies schema for taskgraph parameters → Specify schema for taskgraph parameters
Attachment #9053018 - Attachment description: Specify schema for taskgraph parameters → Validate taskgraph parameters using a schema
Pushed by mhentges@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f5eea3706636
Validate taskgraph parameters using a schema r=tomprince
Backout by shindli@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/4b50c89ff7cf
Backed out 2 changesets (bug 1535056, bug 1513854) for linux build bustages and failed Gecko Decision Task CLOSED TREE

I had left my patch sitting and it got stale - I should've re-tested it before landing.
I'll re-test and re-land this ticket later (next week, hopefully)

Assignee: nobody → mhentges

The failure was caused by pushlog_id being a basestring, not unicode

Flags: needinfo?(catlee)
Pushed by mhentges@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/48687a76aec5
Validate taskgraph parameters using a schema r=tomprince
Status: NEW → RESOLVED
Closed: 2 months ago
Resolution: --- → FIXED
Flags: needinfo?(mhentges)

Ahh, I tested with taskgraph decision, but I was using an old head-ref that didn't contain recent payload changes.
I'll update, learn how to run against try, and land again

Associated try has passed the decision task

Pushed by mhentges@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/50beca34e939
Validate taskgraph parameters using a schema r=tomprince
Status: REOPENED → RESOLVED
Closed: 2 months ago2 months ago
Resolution: --- → FIXED
Flags: needinfo?(mhentges)

I've resolved the specific error (--optimize-target-tasks was being parsed to an int, not a bool).
I'll land again (after checking with Tom that my solution is OK), and I'll continue to iterate if it fails

Attachment #9053018 - Attachment description: Validate taskgraph parameters using a schema → Bug 1535056 - Specifies schema for taskgraph parameters
Pushed by mhentges@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f0f79e047a47
Specifies schema for taskgraph parameters r=tomprince
Status: REOPENED → RESOLVED
Closed: 2 months ago2 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.