Closed Bug 1468155 Opened 6 years ago Closed 6 years ago

Enforce that tasks depend on the tasks whose content they mount

Categories

(Taskcluster :: Workers, enhancement)

enhancement
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: pmoore, Assigned: pmoore)

References

Details

Attachments

(1 file)

In generic-worker, tasks can mount artifacts from other tasks. In almost all cases[1], the task that produces the mounted artifact(s) should have completed before the task that depends on them is scheduled. I'd therefore like to introduce the requirement that if task A mounts artifact(s) from task B, that task A must explicitly depend on task B, otherwise the worker should resolve task A with exception/malformed-payload. ---- [1] An exception to this general principle could be that a task explicitly publishes artifact(s) during its task execution, which should be mounted by another task, yet the task is only resolved some considerable time later. With this new requirement in place, the task that wishes to mount the artifacts would have to wait for the task to complete. In reality this is already the case, since there is no trivial way to schedule the task when the artifacts are available, so in practice the task dependency would already be required. As such, this exceptional case which is probably not worth optimising, however, it could be done as follows: We could expand the schema for an individual task dependency to be either * a taskId (as now) _or_ * an object: { "task": <TASK_ID>, "artifacts": [ <ARTIFACT>... ] } In this case, task A could be scheduled as soon as all of the listed artifacts of task B have been published, which may be prior to task B completing.
(In reply to Pete Moore [:pmoore][:pete] from comment #0) > In this case, task A could be scheduled as soon as all of the listed > artifacts of task B have been published, which may be prior to task B > completing. In other words, a task could depend on the artifacts of other tasks (currently a task can only depend on other tasks).
Attachment #8984989 - Flags: review?(jopsen)
Commits pushed to master at https://github.com/taskcluster/generic-worker https://github.com/taskcluster/generic-worker/commit/af896563274a9ea46ebd89822632c062002a8cca Bug 1468155 - check mounted artifact content's tasks are task dependencies https://github.com/taskcluster/generic-worker/commit/5643391bf3b4c82a86f208d6142de1719bd10cf4 Merge pull request #100 from taskcluster/bug1468155 Bug 1468155 - check mounted artifact content's tasks are task dependencies
Released in generic-worker 10.8.5.
Depends on: 1468633
Comment on attachment 8984989 [details] [review] Github Pull Request for generic-worker as set in github
Attachment #8984989 - Flags: review?(jopsen) → review+
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Component: Generic-Worker → Workers
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: