1592162 - Split comm_taskgraph loaders and transforms into separate packages

Status: RESOLVED FIXED

(Thunderbird :: Build Config, enhancement)

Description

[Rob Lemley [:rjl]]

This is to facilitate work going on in a few different bugs.

comm/taskcluster/comm_taskgraph/init.py is a collection of mostly transforms functions. Thunderbird's build needs are changing and this file has grown some as a result. Splitting transforms and loaders out into separate packages will mirror the Firefox structure and keep various WIP patches from colliding.

Comment 1

[Rob Lemley [:rjl]]

Attachment: comm_taskgraph_splitup.patch

Semi-WIP.

Comment 2

[Geoff Lankow (:darktrojan)]

Did you want this reviewed? You said it was waiting for review but it isn't and comment 1 suggests it's unfinished.

Comment 3

[Rob Lemley [:rjl]]

Attachment: comm_taskgraph_splitup.patch

This is ready.

Comment 4

[Rob Lemley [:rjl]]

Attachment: merge_loader.patch

This is semi-beta, but because it's only been verified to work on fetch
tasks.
Toolchains in particular don't really work, but I those issues are likely
to be solved elsewhere.
source-tests come to mind as a possible consumer of this loader.

Comment 5

[Geoff Lankow (:darktrojan)]

Comment on attachment 9106346 [details] [diff] [review]
comm_taskgraph_splitup.patch

Review of attachment 9106346 [details] [diff] [review]:
-----------------------------------------------------------------

::: taskcluster/comm_taskgraph/transforms/addon.py
@@ +54,5 @@
> +            if "ja-JP-mac" in chunked_locales:
> +                label = "unsigned-repack-mac"
> +                job["dependencies"][
> +                    label] = "nightly-l10n-macosx64-shippable-%d/opt" % \
> +                             this_chunk

This is some, uhh, "pretty" wrapping. Let's go back to 99 characters.

Comment 6

[Rob Lemley [:rjl]]

Attachment: comm_taskgraph_splitup.patch

Addresses the formatiing issue in comment 5.

Comment 7

[Rob Lemley [:rjl]]

Attachment: comm_taskgraph_splitup.patch

Using the correct repo... addresses the formatting in comment 5.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/e1879be2884c
Add merge loader. r=darktrojan
https://hg.mozilla.org/comm-central/rev/fee24a698db4
Split comm_taskgraph functions into loader and transform packages. r=darktrojan