735948 - Expose trending data via REST

Status: RESOLVED WONTFIX

(Webtools Graveyard :: BzAPI, enhancement)

Description

[Lawrence Mandel [:lmandel] (use needinfo)]

This is a request to enhance the Bugzilla REST API to be able to pull trending data from Bugzilla like the Bugzilla charts allow. (Think of the request as wanting the data to reproduce the Bugzilla trending charts outside of Bugzilla.) This data is structured as counts over time. The output should include dates with counts for each date. 

For example, I should be able to query the number of open bugs in the Fennec Native product from 20120101 to the current date.

Comment 1

[Gervase Markham [:gerv]]

This would effectively mean building a RESTful front end to the data currently accessible through here:
https://bugzilla.mozilla.org/chart.cgi
right?

Currently, Bugzilla itself has no API for exposing the raw numbers. So firstly, Bugzilla would need enhancing to do that, and then the REST API would need enhancing to take that data and re-present it to you. So the first thing to do would be to file a bug for enhancing Bugzilla to offer raw data access, and make this one dependent on it.

An added issue is that it's not like doing a search - you can't just define an arbitrary query and say "give me historical data for this". It has to have been set up beforehand, and you have to know the special name that you assigned to the data set. That makes any REST API to the data dependent on external knowledge.

Add to that the fact that the REST API is currently in maintenance mode, pending the deployment of dkl's reimplementation which is a Bugzilla add-on, I'd suggest you not hold your breath for this to happen. Sorry :-|

Gerv

Comment 2

[Lawrence Mandel [:lmandel] (use needinfo)]

(In reply to Gervase Markham [:gerv] from comment #1)
> This would effectively mean building a RESTful front end to the data
> currently accessible through here:
> https://bugzilla.mozilla.org/chart.cgi
> right?

Right.

> Currently, Bugzilla itself has no API for exposing the raw numbers. So
> firstly, Bugzilla would need enhancing to do that, and then the REST API
> would need enhancing to take that data and re-present it to you. So the
> first thing to do would be to file a bug for enhancing Bugzilla to offer raw
> data access, and make this one dependent on it.
> 
> An added issue is that it's not like doing a search - you can't just define
> an arbitrary query and say "give me historical data for this". It has to
> have been set up beforehand, and you have to know the special name that you
> assigned to the data set. That makes any REST API to the data dependent on
> external knowledge.
> 
> Add to that the fact that the REST API is currently in maintenance mode,
> pending the deployment of dkl's reimplementation which is a Bugzilla add-on,
> I'd suggest you not hold your breath for this to happen. Sorry :-|

If the REST API is in maintenance mode I wouldn't expect this new feature to make it in. Do you have any further details about dkl's reimplementation? Is there a Bugzilla product/component for this work? Is there a timeline? Will the reimplementation support the current Bugzilla REST API? Will the new add-on approach require changes to Bugzilla itself in order to implement this feature?

Comment 3

[Gervase Markham [:gerv]]

dkl is the man to give you the info. My understanding is that it's API-compatible with the current one, but faster. Of course, there's nothing to stop additional APIs being added to it.

Gerv

Comment 4

[JP Rosevear [:jpr]]

Perhaps its easier to make this available via the metrics database for bugzilla?

Comment 5

[Martin Best (:mbest)]

Yes, it might be good to use the Elastic Search database that the metrics team has developed.  We are in the process of debugging it now.  Basically the database pulls all the meta data on all bugs, including security bugs (stripped of all sensitive information), so that large long term statistical queries can be run without slowing down Bugzilla.  The database schema was designed with the intention to make it public but security will have to sign off on that.  It's my intention to push for that as soon as the data is verified as consistent with Bugzilla.

Comment 6

[Martin Best (:mbest)]

Just gave Lawrence a briefing on the database and showed him examples of what is stored.  I'm going to provide him with tools so he can browse through what's in there.  Currently MPT vpn access is required to use the database.

Comment 7

[Gervase Markham [:gerv]]

OK; looks like Metrics is the way to go.

Gerv