1805842 - Job transformation relies a lot on expensive deepcopy

Status: RESOLVED FIXED

(Taskcluster :: General, task)

Description

[[:sergesanspaille]]

copy.deepcopy is a generic way of performing a "deep" copy of an object, and it's used a lot across taskcluster code.

We could make it faster by taking advantage of the tree structure of the jobs (no cycles) to have a more efficient deepcopying routine.

Comment 1

[[:sergesanspaille]]

Attachment: Bug 1805842 - Introduce gecko_taskgraph.util.copy_task r=ahal!

This module defines a single, efficient function to deepcopy a job. It
is faster than deepcopy because it doesn't need to track cycles and
duplicate references that don't make sense for tree (and not graph)
structures.

I measure a speedup > 10% on mach taskgraph tasks --fast >/dev/null.

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fa2cdac6989a
Introduce gecko_taskgraph.util.copy_task r=taskgraph-reviewers,jlorenzo

Comment 3

[Marian-Vasile Laza]

Backed out 4 changesets (Bug 1805842, Bug 1806472, Bug 1806473, Bug 1803537) for causing build bustage on Gecko Decition Task.
Backout link
Push with failures <--> D
Failure Log

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5aae8c28c366
Introduce gecko_taskgraph.util.copy_task r=jlorenzo

Comment 5

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/5aae8c28c366