1204910 - fix buildbot bridge corner case of deadline exceeded while build is running

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[[e-mail] (:bhearsum)]

Caught this today while testing bug 1196407. If a Task exceeds its deadline while the Buildbot Build is running, the TCListener kills the Buildbot Build, but doesn't delete the Task from the BBB DB. This is because it wants to give the BBListener to process the build finished message. The BBListener fails to attach artifacts (task is already completed - that's expected), but then also attempts to cancel the task, which is very surprising, because that code is protected behind:
                if status.get("state") == "running":
                    self.tc_queue.cancelTask(taskid)

This ends up failing and raising an exception, which means it never gets to the code that deletes the task from the bbb db.

It seems like Taskcluster was returning bad state information for this Task if state == running, but canceling failed. This might be related to bug 1204898.

Comment 1

[Chris Cooper [:coop] (he/him)]

Ben: are we waiting/blocked on investigations in bug 1204898? We should mark it as a dependency if so.

Comment 2

[[e-mail] (:bhearsum)]

(In reply to Chris Cooper [:coop] from comment #1)
> Ben: are we waiting/blocked on investigations in bug 1204898? We should mark
> it as a dependency if so.

I'm not sure. We should probably handle this case better anyways though.