1660077 - Deprecate 'soft-dependencies' in favour of a morph

Status: RESOLVED INVALID

(Firefox Build System :: Task Configuration, task, P3)

Description

[Andrew Halberstadt [:ahal]]

In bug 1659187 I'm implementing a feature that's similar to the soft-dependencies one, only it will actually control whether or not the task itself is actually run.

I believe this new if-dependencies feature will be able to handle the current soft-dependencies case just fine.

Comment 1

[Andrew Halberstadt [:ahal]]

According to Tom, my assumption here is false and we want to run the code-review task even if no dependencies were run. He mentioned that the out-of-tree taskgraph has already moved soft-dependencies to a morph however. So we can either do that, or resolve this bug I suppose.

Comment 2

[Tom Prince [:tomprince]]

we want to run the code-review task even if no dependencies were run.

More that it wants to run if some dependencies have run, and depend only on those tasks that actually do run. It is used for the code-review-issues tasks, which depends on all the tasks that have the code-review attribute set, and is used by the code-review bot to see when all tasks in a push by the bot are completed.

Comment 3

[Tom Prince [:tomprince]]

The specific thing that soft-dependencies or a morph allows, is dropping (or in the case of a morph never addding) dependencies on tasks that have been optimized away. For example, if only python files are touched, the python lints should run, and the (e.g.) js lints should not, and code-review-issues should run once all the python lints are done. if-dependents doesn't provide a means of running code-review-issues without also requiring the (e.g.) js lints to run.

Comment 4

[Andrew Halberstadt [:ahal]]

Oh, I pretty much re-did that if-dependents patch from scratch*, and now it accepts a list of dependencies and will run if at least one of those dependencies is scheduled. So it sounds like it would be able to solve the code-review-issues case after all.

* turns out there were a ton of edge cases around triangle graphs and such, and it ended up being way more complex than I thought.. I should be posting it to review soon.

Comment 5

[Andrew Halberstadt [:ahal]]

Oh, nvm.. I see what you're saying now.

Comment 6

[Andrew Halberstadt [:ahal]]

if-dependents doesn't provide a means of running code-review-issues without also requiring the (e.g.) js lints to run.

It occurred to me that we could get around the above problem if we only remove tasks from the prune_candidates if the dependency task was kept due the optimization (or do not optimize) but not if it was kept for dependent tasks. We'd need to adjust the order of operations here so that we test the optimization before we test for dependent tasks (which could have a perf impact as we'd need to actually run the optimizations of more tasks).. but maybe we could do this only for tasks that have prune_deps so it wouldn't be too bad..

This might be an avenue I explore, as it would be a big help with |mach try auto| as well (though in a follow up bug since bug 1659187 is already complex enough).

Comment 7

[Andrew Halberstadt [:ahal]]

I think there's a distinct use case for soft-dependencies here.