1449282 - Create an endpoint that will report back the number of jobs currently in the jobqueue

Status: RESOLVED FIXED

(bugzilla.mozilla.org :: API, enhancement)

Description

[Chris Kolosiwsky [:ckolos] (ckolos has left the building)]

if possible, please make it possible to query a REST endpoint to return the number of jobs currently waiting in the queue for monitoring purposes.

Comment 1

[David Lawrence [:dkl]]

I would imagine the result of the API method would look something like:

{
  "waiting":12,
  "deferred":0,
  "errors":5
}

where the sum of the three is the total jobqueue current entries. I am not even sure we would need authentication for the method as it doesn't really give out anything sensitive. Unless maybe someone is trying to DoS BMO and they want to see how high the numbers go. We could wrap a group access around it or just require caller to be logged in. 

imadueme, would you want to take a look at this?

dkl

Comment 2

[Israel Madueme [:imadueme]]

Hey dkl, I would be happy to look into adding this. I will need some guidance on how to get started or a reference to exisiting code that is similar. Thanks.

Comment 3

[David Lawrence [:dkl]]

(In reply to Israel Madueme [:imadueme] from comment #2)
> Hey dkl, I would be happy to look into adding this. I will need some
> guidance on how to get started or a reference to exisiting code that is
> similar. Thanks.

bug 1204683 is a good reference on how to add a REST API endpoint to the different files involved.

Instead of returning user data, you will need to query the DB directly to get the different counts from the jobqueue. And then return a hash containing the count values.

# UNTESTED
sub jobqueue_status {
    my ( $self, $params ) = @_;

    Bugzilla->login(LOGIN_REQUIRED);

    my $query = "
        SELECT
            COUNT(*) AS total
            (SELECT COUNT(*)
               FROM ts_error
              WHERE ts_error.jobid = j.jobid
            ) AS errors,
        FROM
            ts_job j
            INNER JOIN ts_funcmap f
                ON f.funcid = j.funcid";

    return $dbh->selectall_arrayref($query, { Slice => {} });   
}

Comment 4

[Israel Madueme [:imadueme]]

Attachment: Github PR #514

Comment 5

[Israel Madueme [:imadueme]]

The endpoint will be at /rest/jobqueue_status next deploy.
@dkl: This only includes the 'total' and 'errors' fields; do you still think the 'deferred' field is needed? If so can you point me to the relevant SQL tables that that information can be pulled from? Thanks.

Comment 6

[David Lawrence [:dkl]]

(In reply to Israel Madueme [:imadueme] from comment #5)
> The endpoint will be at /rest/jobqueue_status next deploy.
> @dkl: This only includes the 'total' and 'errors' fields; do you still think
> the 'deferred' field is needed? If so can you point me to the relevant SQL
> tables that that information can be pulled from? Thanks.

Thinking about this more, the entry only stays in the deferred state for a very short period. For monitoring, we are only really interested in total and number of errors so leave out deferred.

dkl