870061 - Add report of running builds on a per-slave basis to buildapi

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect)

Description

[John Hopkins (:jhopkins)]

buildapi doesn't support reporting running builds on a per-slave basis.

For example:

 https://secure.pub.build.mozilla.org/buildapi/running/talos-r3-w7-088

This returns 0 records even when a build is running on that machine.

We could make use of this feature in kittenherder so it can double-check that a build slave is not running a build before doing a reboot.  That would guard against this scenario:
- slave is idle for 6 hours (the threshold for kittenherder to take action)
- slave starts a build
- kittenherder checks the slave but cannot SSH in to look at the twistd.log timestamps
- kittenherder assumes the slave needs a reboot and uses PDU or IPMI to reset it.
- this kills the build in progress

Comment 1

[Nick Thomas [:nthomas] (UTC+13)]

Unfortunately the buildbot scheduler db doesn't record which slave is running a job, so we don't have a way to do this cheaply. You can get the master that claimed a buildrequest, but have to hit the web ui on that master to figure out which slave is doing it. 

At which point you're probably better parsing pages like that
 http://buildbot-master80.srv.releng.usw2.mozilla.com:8201/buildslaves?no_builders=1
But sometimes the master gets confused and that will tell you lies, and it can be slow to load if the master is busy.

Comment 2

[Justin Wood (:Callek)]

Is there a way we can coerce the scheduler DB to know the slavename relatively cheaply?

Comment 3

[Nick Thomas [:nthomas] (UTC+13)]

302 Dustin.

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

If it's not in there, it's not in there :)

You can, of course, hack the code to put it in there..

Comment 5

[John Hopkins (:jhopkins)]

Adding a buildslave field is part of the planned 'nine' release of buildbot:

 http://trac.buildbot.net/ticket/1771

Comment 6

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Found in triage.

aiui, the buildapi would first need the data to be stored in the scheduler db... a feature upcoming in buildbot0.9?

Comment 7

[Dustin J. Mitchell [:dustin] (he/him)]

Yes, in Buildbot nine there is a buildslave associated with each build in the Buildbot database.