Hook up toolchain tasks to the build and test task graph

RESOLVED FIXED

Status

defect
RESOLVED FIXED
3 years ago
Last year

People

(Reporter: ted, Assigned: glandium)

Tracking

(Depends on 1 bug, Blocks 1 bug)

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 1 obsolete attachment)

We have "toolchain" tasks defined in-tree currently:
https://dxr.mozilla.org/mozilla-central/source/taskcluster/ci/toolchain/kind.yml

These can be used to build some of our toolchains, like clang/gcc/binutils, so that developers don't have to build them locally. This is good for making repeatable builds and removing single points of failure on people knowing how to build these things.

We should hook these up to the existing build and test task graph--the task definitions should specify how to build a tool (a script plus an external repository and revision to build), and just like the tasks that build docker images toolchain tasks should be scheduled when those inputs change. We should be able to specify that certain types of tasks have a dependency on certain tools, so that the taskgraph will be generated appropriately. Finally, tasks will need to be provided with the artifacts from the toolchain tasks as inputs somehow. We could bake this into a task definition somewhere, specifying that a certain tool should be unpacked into a certain directory in the worker. Dustin says this last part has been discussed as "pre-filled caches" in the past, but not yet implemented.

This would make the testing cycle for testing new toolchains and other tools much simpler--you'd just change the revision in the task definition or the build script and push to try and you'd get a toolchain job and then build jobs using that toolchain.

I'd like this to support use cases that aren't just the toolchains we have defined now. Specifically I've been testing sccache2 on try (bug 1286934) and it's a hassle to build it for all 3 Tier-1 platforms, put it in tooltool, update the in-tree manifests and then push to try. It'd be much easier if it was defined as a toolchain task and I could just tweak the rev and push to try.
Assignee

Comment 1

3 years ago
Sadly, until buildbot jobs are replaced by taskcluster jobs, we'll still have to deal with tooltool :(
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #0)
> Dustin says this last part has been discussed as
> "pre-filled caches" in the past, but not yet implemented.

For windows tasks, this is already implemented in generic worker (the worker we currently run on windows worker types). See the "mounts" section of the generic worker payload here:

https://docs.taskcluster.net/manual/execution/workers/generic-worker
(In reply to Mike Hommey [:glandium] from comment #1)
> Sadly, until buildbot jobs are replaced by taskcluster jobs, we'll still
> have to deal with tooltool :(

Yeah. I don't know what the timeline looks like on that, or if we could figure out something clever to work around this in the meantime.
Random thought: having the toolchain in the task graph makes it considerably easier for local builds to download and use toolchain artifacts from automation in local builds. One could imagine a configure mode that looks at the VCS commit (like artifact builds), finds toolchains built in automation, and automatically uses them. If you are bisecting locally, the toolchain would "just work." That could be really convenient.
Assignee

Comment 5

3 years ago
(In reply to Gregory Szorc [:gps] from comment #4)
> Random thought: having the toolchain in the task graph makes it considerably
> easier for local builds to download and use toolchain artifacts from
> automation in local builds. One could imagine a configure mode that looks at
> the VCS commit (like artifact builds), finds toolchains built in automation,
> and automatically uses them. If you are bisecting locally, the toolchain
> would "just work." That could be really convenient.

You can already do that with tooltool manifests (except for non-public files, but those wouldn't be built on taskcluster anyways).
Assignee

Updated

2 years ago
Depends on: 1335651
Assignee

Comment 6

2 years ago
Assigning this to myself, since I'm currently actively working on most parts of this.
Assignee: nobody → mh+mozilla
Assignee

Updated

2 years ago
Depends on: 1343713
Assignee

Updated

2 years ago
Depends on: 1343716
Assignee

Updated

2 years ago
Depends on: 1343718
Assignee

Updated

2 years ago
Depends on: 1355731
Assignee

Updated

2 years ago
Depends on: 1356529
Assignee

Updated

2 years ago
Depends on: 1356926
Assignee

Updated

2 years ago
Depends on: 1356927
Assignee

Updated

2 years ago
Depends on: 1356929
Assignee

Updated

2 years ago
Depends on: 1356931
Assignee

Updated

2 years ago
Depends on: 1356932
Assignee

Updated

2 years ago
Depends on: 1341934
Assignee

Updated

2 years ago
Depends on: 1356951
Assignee

Updated

2 years ago
Depends on: 1356955
Assignee

Updated

2 years ago
Depends on: 1356974
Assignee

Updated

2 years ago
Blocks: 1356989
Assignee

Updated

2 years ago
Depends on: 1356994
Assignee

Updated

2 years ago
Depends on: 1360609
Assignee

Updated

2 years ago
Depends on: 1359838
Assignee

Updated

2 years ago
No longer depends on: 1359838
Assignee

Updated

2 years ago
Depends on: 1374940
Assignee

Updated

2 years ago
Depends on: 1382564
Assignee

Updated

2 years ago
Depends on: 1382571
Assignee

Updated

2 years ago
Depends on: 1383993
Assignee

Updated

2 years ago
Blocks: 1383996
Assignee

Updated

2 years ago
Depends on: 1384417
Assignee

Updated

2 years ago
Depends on: 1384418
Assignee

Updated

2 years ago
Depends on: 1384419
Assignee

Updated

2 years ago
Depends on: 1384420
Assignee

Updated

2 years ago
Depends on: 1384421
Assignee

Updated

2 years ago
Depends on: 1384422
Assignee

Comment 7

2 years ago
Posted patch Migration helper (obsolete) — Splinter Review
I just opened a bunch of bugs for individual toolchains. This is a helper patch for the migration, not meant to land.

The way it works is:
- Apply the patch
- Add an entry to KNOWN_TOOLCHAINS, for example `'linux64-sccache': '7bbe53cda3f6'`
- Run the command line from a decision task and add the missing toolchains until it doesn't complain anymore.
- Remove the corresponding tooltool manifest entries.
Assignee

Comment 8

2 years ago
This is a revised version that takes into account that some toolchain jobs produce an artifact with the same name. That prevented either win32-clang-cl or win64-clang-cl to be found missing.
Attachment #8890195 - Attachment is obsolete: true
Assignee

Updated

2 years ago
Depends on: 1386028
Assignee

Updated

2 years ago
Depends on: 1386149
Assignee

Updated

2 years ago
Depends on: 1386519
Assignee

Updated

2 years ago
Depends on: 1386539
Assignee

Updated

2 years ago
Depends on: 1386588
Assignee

Updated

2 years ago
Depends on: 1386589
Despite bug 1356529 still being open this seems pretty fixed. Should we punt additional work to followup bugs?
Blocks: 1391408
Assignee

Comment 10

2 years ago
I'm keeping this as a tracking bug. The android toolchains and rust are the main remaining things.
Component: General → Build Config
Product: Taskcluster → Core
glandium: is this bug serving any useful role these days?
Flags: needinfo?(mh+mozilla)
Assignee

Comment 12

Last year
I guess not, since most things are now hooked up. The remainder is things like msvc or old js shell.
Status: NEW → RESOLVED
Closed: Last year
Flags: needinfo?(mh+mozilla)
Resolution: --- → FIXED

Updated

Last year
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.