1609987 - Port bug 1568006 - Allow pulling taskgraph code from Thunderbird tree

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, enhancement)

Description

[Rob Lemley [:rjl]]

Extend taskgraph.util.verify.verify_docs functionality to check multiple documentation base paths.

For example, when checking that a Kind name is documented, look for it in taskcluster/docs/kinds.rst as well as comm/taskcluster/docs/kinds.rst.

The end goal is to facilitate creating new task kinds that are specific to Thunderbird that are implemented in Thunderbird's repo rather than Firefox's.

Comment 1

[Rob Lemley [:rjl]]

Attachment: Bug 1609987 - Allow checking additional paths in verify_docs. r?tomprince

There's no way for Thunderbird to define a new task kind at the moment because
the verify_docs function only checks one base path.

This sets up a DocsVerifier class which is initiated early in the task graph
generation process with a list of documentation base paths that are set in
config.yml.

verify_docs then checks all of the provided paths. As it iterates through the
list of identifiers to look for in a file, it records each one it finds, then
compares the matched list to the original.

Comment 2

[Rob Lemley [:rjl]]

Attachment: Bug 1609987 - unittests for taskgraph.util.verify. r?tomprince

Some basic tests for new verify docs functionality.

Comment 3

[Rob Lemley [:rjl]]

Attachment: Bug 1609987 - Port bug 1568006: Add project's taskgraph directory to the python path.

For mozilla-central, all the code related to taskgraph lives in
taskcluster/taskgraph. Thunderbird's build requirements are evolving, and
we want to be able to have repository-specific code. The natural place for it
to live is an a package beside taskcluster/ci. Add that to python path,
and provide some hooks for adding to the various registries in taskgraph.

Comment 4

[Rob Lemley [:rjl]]

Attachment: taskcluster_extend_1.patch

This is the start of being able to extend taskcluster functionality
within c-c easier.
D60540 is landing in M-C soonish and will cause bustage without this.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/756e3187d18e
Port bug 1568006: Add project's taskgraph directory to the python path. r=tomprince

Comment 6

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/756e3187d18e

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/bfcc947899e5
Initial support for extending taskgraph with hooks. r=darktrojan

Comment 8

[Rob Lemley [:rjl]]

Attachment: Bug 1609987 - Port bug 1568006: Allow projects to modify decision parameters. r?tomprince

More ports from external taskgraph project. Short term, this will be used to get partial updates
working for Thunderbird.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3a5bd821afe2
Port bug 1568006: Allow projects to modify decision parameters. r=tomprince

Comment 10

[Daniel Varga [:dvarga]]

https://hg.mozilla.org/mozilla-central/rev/3a5bd821afe2

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/533f15e7cc13
Allow checking additional paths in verify_docs. r=tomprince
https://hg.mozilla.org/integration/autoland/rev/30f4c91398ed
unittests for taskgraph.util.verify. r=tomprince

Comment 12

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/533f15e7cc13
https://hg.mozilla.org/mozilla-central/rev/30f4c91398ed