coalescer should ensure that the calling task is in the task list

RESOLVED FIXED

Status

Release Engineering
General Automation
RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: dustin, Assigned: dividehex)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

2 years ago
Given coalescing key K, imagine we have tasks A, B, C, and D pending.

A worker begins task D, which changes state to running and is thus removed from the coalescer's list.

The worker calls the coalescer, which returns [A, B, C]

The worker then claims A, B, and C, and executes C, thereby superseding A, B, and D.  That's not what we want -- C does not supersede D.

Two things would fix this:

 * Don't remove tasks from the coalescer list until they are *resolved*.  For the most part this will just make a lot more tasks that the worker will fail to claim, but critically it would mean that task D is still in the list, in its proper place (after C).

 * The worker should refuse to supersede anything when it gets a list that does not contain the already-claimed task, as it doesn't know how that task relates to the others.
(Assignee)

Updated

2 years ago
Depends on: 1213039
(Assignee)

Updated

2 years ago
Blocks: 1213039
No longer depends on: 1213039
(Reporter)

Comment 1

2 years ago
https://github.com/taskcluster/docker-worker/pull/209 should address the docker-worker side of this ("refuse to supersede..")
(Assignee)

Comment 2

2 years ago
https://github.com/mozilla/tc-coalesce/pull/1 takes care of the service side.  Taskids will remain ordered in the coalesced list until a task reaches a final state (completed|failed|exception)

This has been merged and pushed to production.
=== releng-tc-coalesce Releases
v26  Deploy 61032b3                jwatkins@mozilla.com            2016/01/29 10:58:08 -0800 (~ 51s ago)
(Assignee)

Updated

2 years ago
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.