1351014 - Implement symbol upload tasks as a graph morph

Status: RESOLVED INVALID

(Firefox Build System :: Task Configuration, task)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

we have a symbol-upload task kind, with tasks created to depend on each build we perform -- for certain kinds of builds.  These symbol-upload tasks basically just take an artifact from the build and upload it to the symbol server.

Hoewver, this gets a little complicated since we need to make sure those symbol-upload tasks are created whenever needed, but that we don't include them when the builds they depend on are not needed (for example, if they are optimized away).

We recently added support for "graph morphs" which occur after optimization is copmlete, and transform the whole taskgraph.

The idea here would be to implement a morph which will look for tasks containing `task.extra.symbol-upload` and create a dependent task that will actually perform the symbol upload, then delete the task.extra property from the parent task.

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

Ted, given the recent changes to the symbol server, and the work we talked about today to add a symbol upload kind, does this make sense?

Comment 2

[(not currently active) Ted Mielczarek]

I'm not entirely sure! For non-try branches, symbol upload tasks should always be scheduled for nightly builds. That's the extent of what is implemented right now. If you ask for nightly builds on try, you can also get symbol uploads there.

What I'd like is to preserve that existing behavior, but also be able to schedule a symbol upload task for *any* build on a try push. Ideally this could be done at push time (by selecting the right tasks with `mach try fuzzy`) or after-the-fact by using Treeherder's "Add Jobs" UI.

When we discussed this on IRC this morning you suggested that we filter the tasks out in target task set selection. It looks like there's already code to do that:
https://dxr.mozilla.org/mozilla-central/rev/994a8d6eccbcdc6106794705bd77e3ac5f031be2/taskcluster/taskgraph/target_tasks.py#37

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

OK -- if we do that, then it can't be in a morph.  So tihs bug is invalid :)