1091196 - Create hooks for automatic failure/success

Status: RESOLVED WONTFIX

(Taskcluster :: General, defect)

Description

[Kevin Grandon :kgrandon]

For integration testing, it would be nice to have some taskgraph hooks for automatically failing or passing a taskgraph. Ideally this would allow us to hook into this with other services, and save Mozilla some money by not having to spin up a machine.

Perhaps also consider support for where we might want to have a hook for firing the taskgraph extended event, or delaying the event for some arbitrary amount of time.

Comment 1

[Jonas Finnemann Jensen (:jonasfj)]

When you say hook I assume you mean API end-point. Or maybe I'm getting something completely wrong.

> For integration testing, it would be nice to have some taskgraph hooks for automatically failing
> or passing a taskgraph. Ideally this would allow us to hook into this with other services, and
> save Mozilla some money by not having to spin up a machine.

There is a plan to add a cancel task API end-point to the queue (bug 1080286).
Which could be used to cancel all tasks in a task-graph.
I assume this is interesting if a newer revision of a patch shows up while we're testing an old, if a
pull-request is updated with `git push --force` overwriting old commits.

I'm not sure I understand your use-case for passing tasks?
It could be done, but side effects might not be nice.

> Perhaps also consider support for where we might want to have a hook for firing the taskgraph extended event,
??? An API end-point for publishing the task-graph extended messages without extending a task-graph? 

> or delaying the event for some arbitrary amount of time.
Why would we delay the task-graph extended messages? this can be done in AMQP, but why?

Comment 2

[Kevin Grandon :kgrandon]

This is purely for integration tests, so it would only for testing code which utilizes taskcluster in some way. The main use-cases are mocking success and failure cases. I suppose the alternative could be to spin up a local instance, but it seemed easier to do from the TC side of things. I think James has more info here.

Comment 3

[Jonas Finnemann Jensen (:jonasfj)]

Ahh... For integrations test of components that use taskcluster APIs :)

I'm certainly not happy about sending fake events... etc.
For tasks and taskGraphs, the way to do quick integration tests is claim and resolve your own tasks.
And obviously submit them to a fake provisionerId: 
  queue.claimTask(myTaskId, 0, {
    workerGroup:  'dummy-test-workergroup',
    workerId:     'dummy-test-worker-id'
  })
  queue.reportCompleted(myTaskId, 0, {success: true || false});

And in your task-graph use:
  provisionerId:   "dummy-test-provisioner",
  workerType:      "dummy-test-worker-type"

For more details see:
https://github.com/taskcluster/task-graph-scheduler/blob/cdb1c2281e0f35162a4f2810a8b6ca6002d2dddd/test/scheduler_test.js#L188-L280

You can do the same for extend task-graph, if you don't want to wait for workers to do things, just fake the worker.
We should probably come with a naming scheme for dummy-test- worker-Types, -Groups, -Ids and provisionerIds.
For now just use something consistent, and we'll solve that when we become more restrictive about scopes.

---
Does this solve your problem?

We still can't programmatically fail tasks, we can complete them unsuccessfully, which probably what you want.
To cause task failure we need the cancel API end-point.
Note, we have the following tasks resolutions:
 - completed {success: true} (ran and exited 0)
 - completed {success: true} (ran and non-zero)
 - failed (because of hardware issue, disappearing spot nodes)
(I keep wishing I had come with a better naming scheme for these cases)

Comment 4

[Greg Arndt [:garndt]]

So it appears that this bug has sat opened for awhile, and I'm not sure if the last comment was enough to give details of what to fix in this bug.  Shall we still keep this open?

Comment 5

[Kevin Grandon :kgrandon]

It would still be useful to have, but very low priority right now.

Comment 6

[Jonas Finnemann Jensen (:jonasfj)]

We've added a queue.cancelTask end-point and queue.listTaskGroup end-point... So closing all tasks in a group is pretty easy.

--
This bug is closed as part of the Great Bugzilla Cleaning Sprint of 2016 by bstack and jonasfj, please reopen if you disagree.