Closed
Bug 1301541
Opened 4 years ago
Closed 4 years ago
BzAPI bug search query takes twice as long as one using the native REST API
Categories
(bugzilla.mozilla.org :: Extensions, defect)
Tracking
()
RESOLVED
DUPLICATE
of bug 1035804
People
(Reporter: emorley, Unassigned)
Details
This may end up being wontfix, but if so, I think we need to more aggressively switch people off of the BzAPI compatibility shim, since I'm presuming it's not just slowing down consumers, but causing greater load on BMO as well. $ PARAMS='keywords=intermittent-failure&include_fields=id,summary,status,whiteboard&chfield from=-6m&limit=500' $ time curl -sSfo /dev/null "https://bugzilla.mozilla.org/bzapi/bug?${PARAMS}" real 0m15.203s user 0m0.046s sys 0m0.046s $ time curl -sSfo /dev/null "https://bugzilla.mozilla.org/rest/bug?${PARAMS}" real 0m8.728s user 0m0.046s sys 0m0.000s The query in bug 1301144 is using BzAPI, so this bug is one of the reasons it's so slow. (OrangeFactor uses bzcache, which has now been been partly converted over to the new REST API in https://github.com/jonallengriffin/bzcache/pull/4)
Reporter | ||
Updated•4 years ago
|
Summary: BzAPI query takes twice as long as the native REST API → BzAPI bug search query takes twice as long as one using the native REST API
Reporter | ||
Comment 1•4 years ago
|
||
The increase compared to the native REST API is even more pronounced with queries that return a large number of bugs: PARAMS='keywords=intermittent-failure&include_fields=id&chfieldfrom=-6m' $ time curl -sSfo /dev/null "https://bugzilla.mozilla.org/bzapi/bug?${PARAMS}" real 2m11.149s user 0m0.031s sys 0m0.015s $ time curl -sSfo /dev/null "https://bugzilla.mozilla.org/rest/bug?${PARAMS}" real 0m19.533s user 0m0.031s sys 0m0.031s (both still return the same number of bugs)
Comment 2•4 years ago
|
||
It is not a good use of developer time to do this, however I'll see about splitting off the requests to a dedicated web-head.
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → WONTFIX
Reporter | ||
Comment 3•4 years ago
|
||
(In reply to Dylan Hardison [:dylan] from comment #2) > It is not a good use of developer time to do this I agree, however this leaves: (In reply to Ed Morley [:emorley] from comment #0) > This may end up being wontfix, but if so, I think we need to more > aggressively switch people off of the BzAPI compatibility shim, since I'm > presuming it's not just slowing down consumers, but causing greater load on > BMO as well. Would you prefer to track this via: * reopening this bug * filing a new bug * newsgroup discussion?
Flags: needinfo?(dylan)
Reporter | ||
Comment 4•4 years ago
|
||
ie: As the unfortunate 'last person to touch' a few ancient tools that use Bugzilla's API, I was vaguely aware they used BzAPI not the REST API, but (a) wasn't 100% sure which were migrated, (b) had no idea they were experiencing a 2-6x slowdown by doing so. As such, I hadn't given it a second thought. If I had known, I would have prioritised migration to the native REST API. Suggested actions: * Creating (or improving an existing) "API best practices" page, which warns against using BzAPI (plus suggests general tips like using pagination, last_modified, sensible include_fields, setting a helpful User Agent etc) * Adding a warning banner to all BzAPI resources saying EOL plus in many cases many times slower * Blogging/emailing newsgroups apart BzAPI perf issues - pointing to new best practices page * Performing inspection of BzAPI web logs (via referrer, query strings, User Agent) to identify top N consumers, and file bugs against them * (Optionally, in the future, depending on usage dropping) Setting an EOL date for BzAPI entirely. I'm happy to spend time on the web logs part fwiw.
Comment 5•4 years ago
|
||
FYI there is a deprecation notice at the top of https://wiki.mozilla.org/Bugzilla:BzAPI, but you are right that it doesn't mention speed. I'll update it.
Comment 6•4 years ago
|
||
We can see which users are using bzapi (For the most part) and start contacting them. I'm also going to work on forcing a limit, and we're getting a node dedicated to bzapi queries. User education in the form of best practices is good. It may also be obviated by the elasticsearch code. That code (which wraps the internal search API) is so much faster than bzapi that most searches should be IO-bound... A discussion on tools-bmo is probably the way of handling this.
Flags: needinfo?(dylan)
Comment 7•4 years ago
|
||
sorry, "is much faster than the native searches", not "faster than bzapi". bzapi will make use if when it lands, without us doing anything. It will be an even better selling point, as a search for intermittent-failure on native rest using elastic search takes a tenth of a second. :-)
Reporter | ||
Comment 8•4 years ago
|
||
So this is pretty funny - I'd forgotten about filing pretty much the same bug two years ago: bug 1035804. Also, let's move discussion of BzAPI->native REST persuasion to bug 1003236.
Comment 9•4 years ago
|
||
performing that query results in 104,961,987 function calls. that is largest reason it is slow. I'm posting the profiling data just for funs.
Reporter | ||
Comment 10•4 years ago
|
||
This is being fixed now in bug 1035804.
Resolution: WONTFIX → DUPLICATE
See Also: 1035804 →
Duplicate of bug: 1035804
Updated•1 year ago
|
Component: Extensions: BzAPI Compatibility → Extensions
You need to log in
before you can comment on or make changes to this bug.
Description
•