1334484 - [taskcluster-worker] Set up integration tests as part of CI

Status: RESOLVED FIXED

(Taskcluster :: Workers, defect)

Description

[Pete Moore [:pmoore][:pete]]

As much as possible, we want this to run as close to production-like as possible. We should have a top level integrationtest package in taskcluster-worker that contains subpackages of all the tests. These should test against the production queue.

The integration tests should run in taskcluster, on workers that run tasks as an Administrator on Mac (and later Windows), so that the user creation process, and running tasks under a newly spawned user, is flexed.

We should look to parallelise testing the subpackages as much as possible, to keep CI turnaround times short. Note, parallelising tests inside a `go test` execution may also be to some extent possible, although I believe on Mac we are not planning on having capacity > 1 in production, so maybe this is not helpful/feasible.

Comment 1

[Pete Moore [:pmoore][:pete]]

We have taskcluster-worker unit tests now running in TaskCluster in this PR:
  * https://github.com/taskcluster/taskcluster-worker/pull/155

See for example:
  * https://tools.taskcluster.net/task-group-inspector/#/M6ePoqkYStebD4-E-HYYFA

The next step was extending this to include end-to-end integration tests, as a separate "integrationtest" subpackage. However, this required enabling the worker to start up, run a fixed number of tasks, and then shutdown, which has been done in this PR:
  * https://github.com/taskcluster/taskcluster-worker/pull/164

After that I made a small fix here:
  * https://github.com/taskcluster/taskcluster-worker/pull/166

And I'm currently creating the integration tests (and porting them over from generic-worker), which will also be pushed to PR 155 when running successfully locally.

Comment 2

[Pete Moore [:pmoore][:pete]]

Are you happy with the integration testing we have in taskcluster-worker now?

Comment 3

[Jonas Finnemann Jensen (:jonasfj)]

Yeah, they are quite solid :)