1209722 - Allow Mozilla CI tools to extend a graph + append builders to a task id

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[Armen [:armenzg]]

This will allow appending buildbot builders a TC build.

For instance, adding Mac buildbot builders to this Mac TC build:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f232e5526f34

Comment 1

[Armen [:armenzg]]

Current progress:
https://github.com/armenzg/mozilla_ci_tools/compare/mozci_bbb...extend_graph

python ~/repos/ci_tools/mozci/scripts/misc/buildbot_to_taskcluster.py --append-to-task Ec22h2a_SG-hHrUHZpUb1w --builders "['Rev5 MacOSX Yosemite 10.10 try opt test mochitest-1']" --repo-name try --revision f232e5526f34 --debug

I'm waiting for some credential auth caching to clear up. I will try again tomorrow.

Comment 2

[Armen [:armenzg]]

My apologies for the delay.
When building this I got to do a lot of refactoring to do it properly.
I also found some other bug that I need to fix to make this work.

As a side effect I also generated the first 3-tier scheduled graph (not that it will work but there):
https://tools.taskcluster.net/task-graph-inspector/#bWbW5aCDSBi416XACrvMOA/

The syntax used is:
python ~/repos/ci_tools/mozci/scripts/misc/buildbot_to_taskcluster.py --repo-name try --revision 88ba41c28dce -g "{'Linux x86-64 try build': {'Ubuntu VM 12.04 x64 try opt test mochitest-1': {}, 'Ubuntu VM 12.04 x64 try opt test mochitest-2': {}}}"

Assuming I don't get distracted I should have the feature you need done tomorrow.

FYI this is simply to make your life not need to a .yml file into your try pushes.

Comment 3

[Armen [:armenzg]]

I will be pushing to code to mozci today with what I'm going to describe in here.

The code can append builders to a given task.
If the graph is still running, we extend the graph and create depenencies to it (We only run the dep tasks once the main task completes).
If the graph has completed, we submit a new graph with tasks that don't have dependencies to any other task but they know from which task to grab the artifacts from.

The commands I've used to add Buildbot builders to a Mac and Linux builds which are running:
> python ~/repos/ci_tools/mozci/scripts/misc/buildbot_to_taskcluster.py --repo-name try \
> --revision f232e5526f34 --trigger-from-task-id XnFH40hgRxis7tk24ATITg \
> --builders "['Ubuntu VM 12.04 try opt test crashtest']" --debug

Extended graph:
https://tools.taskcluster.net/task-graph-inspector/#cTolwPRXTbKF0ESVA_KIzw
The Mac job will run when the backlog is over.
https://treeherder.mozilla.org/#/jobs?repo=try&revision=b57c2ae1eae1
> python ~/repos/ci_tools/mozci/scripts/misc/buildbot_to_taskcluster.py --repo-name try \
> --revision f232e5526f34 --trigger-from-task-id EgPbgnuUQiWxG6DwCqVRSQ \
> --builders "['Rev5 MacOSX Yosemite 10.10 try opt test cppunit']" --debug

This command tests that a new graph is created for other than "running" graphs:
https://tools.taskcluster.net/task-graph-inspector/#U3Y4pJF7S-OIQZm53qmulQ
Untouched original graph:
https://tools.taskcluster.net/task-graph-inspector/#DP-P4TDjSui83vmnWAC2BQ
> python ~/repos/ci_tools/mozci/scripts/misc/buildbot_to_taskcluster.py --repo-name try \
> --revision f232e5526f34 --trigger-from-task-id eIS26AoeQYe9rNClxkWf1g \
> --builders "['Ubuntu VM 12.04 try opt test crashtest']" --debug

Comment 4

[Armen [:armenzg]]

Landed:
https://github.com/armenzg/mozilla_ci_tools/commit/668d1b2f38a298b36924d95524f5dc148c6edb0f

This depends on a piece of Mozharness code (bug 1210945).

Comment 5

[(not currently active) Ted Mielczarek]

I mentioned this on IRC, but just so you don't lose track of it, if you're running test jobs on a Taskcluster Mac build you should use this try push:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=348098145706

Comment 6

[Armen [:armenzg]]

The mozharness changes have landed on inbound:
https://hg.mozilla.org/integration/mozilla-inbound/rev/b7e0c90d75ba

Based on that, I've imported your changes and pushed to try:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=4e195f44cd83

Now, off-band, I've run the following:
> python ~/repos/ci_tools/mozci/scripts/misc/buildbot_to_taskcluster.py 
> --repo-name try --revision 4e195f44cd83 --trigger-from-task-id J05EWiFLTcmCqBYxSPbJ6g 
> --builders "['Rev5 MacOSX Yosemite 10.10 try debug test crashtest']" --debug

You can see the builder being added in here:
https://tools.taskcluster.net/task-graph-inspector/#ILBkwLTER56cufIXEzzQZg/

If that works, I will be closing the bug.

If you want to have a set of credentials that allows you to do the same please let me know and I will set it up for you.

If you can create your own credentials this is what you need:
    queue:create-task:*
    queue:define-task:*
    docker-worker:cache:*
    docker-worker:capability:*
    docker-worker:image:*
    queue:route:*
    scheduler:create-task-graph
    scheduler:extend-task-graph:*

Comment 7

[Armen [:armenzg]]

The scheduled job worked.
I've also simplified the syntax to not need to pass the children as a list but instead we can ask for all the children of a given parent build (--children-of "OS X 10.7 64-bit try leak test build").

You can see the graph and push in here:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=4e195f44cd83
https://tools.taskcluster.net/task-graph-inspector/#AEWg3H0STpO465qf4XeWwg

> python ~/repos/ci_tools/mozci/scripts/buildbot_to_taskcluster.py --repo-name try \
> --revision 4e195f44cd83 --trigger-from-task-id J05EWiFLTcmCqBYxSPbJ6g \
> --children-of "OS X 10.7 64-bit try leak test build" --debug