1490082 - Run ci-admin/ci-config/taskgraphs's unit tests and lint as part of the phabricator review process

Status: RESOLVED FIXED

(Developer Infrastructure :: Source Code Analysis, enhancement)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

These are basic Python things, nothing fancy.

python setup.py test
python setup.py flake8

Comment 1

[Marco Castelluccio [:marco]]

This is a good idea. We currently only support mozilla-central, but we can figure out how to support other projects too (not only ci-admin, but for example NSS too). I've filed https://github.com/mozilla/release-services/issues/1511.

Comment 2

[Pete Moore [:pmoore][:pete]]

Hi Kim,

Is this something your team would like to pick up?

Thanks,
Pete

Comment 3

[Kim Moir [:kmoir] ET]

I talked to Steven about this and he said that Sylvestre's team has a bot that polls lando to run linting tests. This is not within the scope of our tools.

Comment 4

[Sylvestre Ledru [:Sylvestre]]

Pete, we have been discussing about extending our tools to other project.
I think we should start a discussion on how to implement that in a scalable way.

Comment 5

[Tom Prince [:tomprince]]

Attachment: Bug 1490082: Add v2 try_task_config support to taskgraph; r?Callek

Reviewbot uses try_task_config to pass information about what phabricator diff
is being checked via this file, so add support for getting info from it.

Comment 6

[Tom Prince [:tomprince]]

Attachment: Bug 1490082: Add code-review support to taskgraph; r?Callek

This different from the one in mozilla-central, as it uses a morph to generate
the code-review task. This makes it easier for downstream projects, as they
will not need to add boilerplate kind to generate the task.

Comment 7

[Tom Prince [:tomprince]]

Attachment: Bug 1490082: Enable code-review for ci-admin; r?Callek

Comment 8

[Tom Prince [:tomprince]]

Attachment: Bug 1490082: Make --ci-configuration-directory absolute at parsing time; r?Callek

ci-admin check changes the working directory, to make pytest work. If the
--ci-configuration-directory is not absolute, it won't be found after
changing directory. Ask click to make it absolute for us, when parsing the
command-line.

Comment 9

[Tom Prince [:tomprince]]

Attachment: Bug 1490082: Enable taskgraph's multi-repo support for ci-admin; r?Callek

This enables using the same taskgraph config for both ci-admin and
ci-configuration.

Comment 10

[Tom Prince [:tomprince]]

Attachment: Bug 1490082: Switch code-review to use phabricator-build-target; r?bastien

Comment 11

[Tom Prince [:tomprince]]

Attachment: Bug 1490082: Include revision and repository in code-review task extra; r?bastien

Comment 12

[Tom Prince [:tomprince]]

Attachment: Bug 1490082: Use ci-admin's taskgraph for ci-configuration; r?Callek

This means that configuration will be kept in sync, and it will also pick up
the code-review configuration.

Comment 13

[Tom Prince [:tomprince]]

Attachment: Bug 1490082: Include repo paths in checkout cache; r?aki

When checking out multiple repos, robustcheckout won't cleanup
sub-repositories, so include the layout in the cache name.

Comment 14

[Tom Prince [:tomprince]]

Attachment: Bug 1490082: Stop using decision-user image in ci-admin; r?aki

When I first setup the non-gecko taskgraph, I imagined that we would
use a docker image with taskgraph pre-installed. However, rather than
dealing with the implications of chain-of-trust for those images, we
ended up installing taskgraph at runtime. ci-admin pre-dates that change. In
the interest of keeping things aligned, this switches ci-admin to install
taskgraph at runtime.

Comment 15

[Tom Prince [:tomprince]]

Attachment: Bug 1490082: Add support to hg-push to get the .taskcluster.yml from an alternative repo; r?aki

ci-admin and ci-config share a taskgraph configuration. Since ci-config only
contains the taskcluster configuration, there is not a need for it to have it's
own .taskcluster.yml. This adds support to ci-admin to specify that a project uses
a .taskcluster.yml from another repository.

Comment 16

[Tom Prince [:tomprince]]

Attachment: Bug 1490082: Adjust ci-admin's `.taskcluster.yml` to support being used by ci-config; r?aki

Comment 17

[Tom Prince [:tomprince]]

Attachment: Bug 1490082: Remove unused decision-user docker image; r?aki

When I first setup the non-gecko taskgraph, I imagined that we would use a
docker image with taskgraph pre-installed. However, rather than dealing with
the implications of chain-of-trust for those images, we ended up installing
taskgraph at runtime. This removes the now unused image.

Comment 18

[Tom Prince [:tomprince]]

Attachment: Bug 1490082: Switch ci-configuraiton to use a `.taskcluster.yml` from ci-admin; r?aki

This also adjust the hg-push task to pass the required environment variables through.

Comment 19

[Tom Prince [:tomprince]]

Attachment: Bug 1490082: Remove unused .taskcluster.yml; r?aki

Now that ci-configuration is using a .taskcluster.yml from ci-admin, we can
remove the .taskcluster.yml from ci-configuration.