1290602 - Abstract out transform-based kinds a little

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

For transform-based task kinds, there are really only three features of the kind implementation:

 * create the inputs to the transforms
   - for tests, this is the list of tests per platform
   - for builds, it's the list of platforms and how to build them
   - for spidermonkey, it's the list of SM variants
   ..
 * run the transforms
 * handle optimization

The middle one (run the transforms) is generic, and in many cases the first is just reading some .yml file, and the last is a no-op.

So I've factored out a TransformTask that performs transforms, and is easy to subclass.  And I've subclassed it for the test kind and will use it in bug 1286075.

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: Bug 1290602: add TransformTask to task-graph machinery;

This abstracts out the process of running transforms as specified in kind.yml,
with some useful default behavior that can be overridden by subclasses.

Review commit: https://reviewboard.mozilla.org/r/68070/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/68070/

Comment 2

[Gregory Szorc [:gps]]

Comment on attachment 8776174 [details]
Bug 1290602: add TransformTask to task-graph machinery;

https://reviewboard.mozilla.org/r/68070/#review65200

::: taskcluster/taskgraph/task/transform.py:12
(Diff revision 1)
> +from taskgraph.util.python_path import find_object
> +from taskgraph.transforms.base import TransformSequence, TransformConfig

You could use `from ..util.python_path import X`. Feels weird to mix relative and absolute from the same package.

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

https://reviewboard.mozilla.org/r/68070/#review65200

> You could use `from ..util.python_path import X`. Feels weird to mix relative and absolute from the same package.

Fair enough.  My defense is that "from . import" is very clearly "from this directory", while "from ..foo import" requires some thinking about how many levels it goes up.

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8776174 [details]
Bug 1290602: add TransformTask to task-graph machinery;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/68070/diff/1-2/

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d645b6c90e9d
add TransformTask to task-graph machinery; r=gps

Comment 6

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/d645b6c90e9d