1331096 - Use queue.claimWork in Buildbot Bridge

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

This will allow the bridge to handle task priorities properly, which we need to support running tasks on hardware.

This will replace the existing queue.claimTask process.  It's already supported in the queue.

Don't deploy this until bug 1331094 is done, as Jonas will use that bug to monitor load on the queue as this new approach is deployed.

Comment 1

[Rail Aliiev [:rail]]

This may require updating the taskcluster library at the same time. I can look at this.

Comment 2

[Rail Aliiev [:rail]]

Back to the pool until the dep is resolved

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

I believe the dep is in place because we were worried about load, but that seems to be non-problematic.  Greg, can you confirm that it's OK to proceed on this now?

Comment 4

[Greg Arndt [:garndt]]

Over the past couple of weeks we have been running claimWork with three of our gecko test worker types that account for ~80% of our overall load.  I will be rolling new AMIs out to the other worker types in the coming days.  For reference, looking at the numbers for the buildbot-bridge worker type, it accounts for about 3% of our load so this shouldn't be a problem.

From my perspective if we want to start migrating things over, then that should be fine, but I am going to add Jonas to this discussion so he can weigh in about the queue side of things.

Comment 5

[Jonas Finnemann Jensen (:jonasfj)]

Yeah, I agree.

Let use queue.claimWork()

Note: this should greatly simplify workers. And if bbb is relying on pulse, then this will add further stability.

Comment 6

[Chris AtLee [:catlee]]

I just spoke to Greg, and we agreed that claimWork isn't a good fit for bbb.

BBB needs to know when a task is first pending, so it can schedule the job in buildbot. Once the job starts in buildbot, BBB claims the task in TC.

As I understand it, queue.claimWork actually claims the task(s), but we shouldn't claim the task until the job actually starts in buildbot.