Document that GET /rest/bug returns a maximum of $max_search_results bugs by default (default: 10000)

NEW
Unassigned

Status

()

Bugzilla
Documentation
a year ago
a year ago

People

(Reporter: Monika, Unassigned)

Tracking

Details

(Reporter)

Description

a year ago
User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36

Steps to reproduce:

Type the URL: https://bugzilla.mozilla.org/rest/bug?product=core&include_fields=id


Actual results:

Only top 1000 entries retrieved 


Expected results:

Should have got a list of all the tickets for product Firefox.
(Reporter)

Updated

a year ago
Severity: normal → major
(Reporter)

Updated

a year ago
Summary: https://bugzilla.mozilla.org/rest/bug?product=core&include_fields=id returns only first 1000 entries while total tickets for Firefox product are more than 5000 → https://bugzilla.mozilla.org/rest/bug?product=core&include_fields=id returns only first 10000 entries while total tickets for Firefox product are more than 50000
(Reporter)

Comment 1

a year ago
Sorry, its 10000 instead of 1000 and 50000 instead of 5000

Comment 2

a year ago
If limit=0 is not passed as argument, then the buglist is limited to $max_search_results bugs by default. This limit is set to 10000 bugs by default, unless admins edited this parameter.
Severity: major → normal
Status: UNCONFIRMED → NEW
Ever confirmed: true
Summary: https://bugzilla.mozilla.org/rest/bug?product=core&include_fields=id returns only first 10000 entries while total tickets for Firefox product are more than 50000 → Document that GET /rest/bug returns a maximum of $max_search_results bugs by default (default: 10000)
(Reporter)

Comment 3

a year ago
(In reply to Frédéric Buclin from comment #2)
> If limit=0 is not passed as argument, then the buglist is limited to
> $max_search_results bugs by default. This limit is set to 10000 bugs by
> default, unless admins edited this parameter.

If I use the URL: https://bugzilla.mozilla.org/rest/bug?product=firefox&include_fields=id&limit=0; I still see 10000 ids only. Please correct me if I am passing argument wrongly. However, if I pass value as 5 that is, https://bugzilla.mozilla.org/rest/bug?product=firefox&include_fields=id&limit=5, I get top 5 only.
(Reporter)

Updated

a year ago
Severity: normal → major

Comment 4

a year ago
Please leave the severity alone. This bug has been triaged. Thanks!
Severity: major → normal
(Reporter)

Comment 5

a year ago
Sure Frederic. The issue is sort of critical for my project. If you can resolve the issues at the earliest possible, it will be of great help.

Comment 6

a year ago
This is not an issue. As I said in comment 2, this is the correct behavior if no limit is specified. If you want all bugs, you must pass limit=0 to the request.
(Reporter)

Comment 7

a year ago
(In reply to Frédéric Buclin from comment #6)
> This is not an issue. As I said in comment 2, this is the correct behavior
> if no limit is specified. If you want all bugs, you must pass limit=0 to the
> request.

Please refer my comment 3 (reply to your comment 2). I tried passing limit=0 still got only top 10000 ids list.
(Reporter)

Comment 8

a year ago
Please refer my comment 3 (reply to your comment 2). I tried passing limit=0 still got only top 10000 ids list.
(Reporter)

Comment 9

a year ago
Effectively, there is no way that we can get more that 10000 transactions. I think if we pass limit value less than default, it overwrites the default value. However, if we pass value as 0 or greater than 10000, it gives output according to default limit only. Therefore, I think this is an issues not just documentation point.

Comment 10

a year ago
OK, I did some testing and the Search.pm code indeed has a hard limit of 10000 bugs when allow_unlimited is not set to 1.

@dkl: what is the desired behavior when limit=0? Returning all bugs would be a good way to DoS Bugzilla.

@Monika: as a workaround, you can pass the offset argument to the URL to get all bugs in a loop:

https://bugzilla.mozilla.org/rest/bug?product=firefox&include_fields=id&offset=10000
https://bugzilla.mozilla.org/rest/bug?product=firefox&include_fields=id&offset=20000
etc...
Flags: needinfo?(dkl)
(Reporter)

Comment 11

a year ago
Frederic, if I try using the offset URLs you have shared, I see following error and no list is retrieved. 
{
   "code" : 50,
   "documentation" : "http://www.bugzilla.org/docs/tip/en/html/api/",
   "error" : true,
   "message" : "The function Bug.search() requires a limit argument, and that argument was not set."
}

Please try clicking on the links that you have shared. It seems offset should be used in combination with limit which also doesn't work. Therefore, the suggested workaround doesn't help. I request to please resolve this with high priority.
(Reporter)

Comment 12

a year ago
Hey Frederic, the link below works:
https://bugzilla.mozilla.org/rest/bug?product=firefox&include_fields=id&offset=10000&limit=10000

Using this, I get a list of next 10000 ids. I think there is no way to get more than 10000 entries irrespective of what is passed as value for limit attribute. The offset workaround helps if used with limit parameter.
(In reply to Frédéric Buclin from comment #10)
> OK, I did some testing and the Search.pm code indeed has a hard limit of
> 10000 bugs when allow_unlimited is not set to 1.
> 
> @dkl: what is the desired behavior when limit=0? Returning all bugs would be
> a good way to DoS Bugzilla.

I agree that allowing all bugs to be retrieved from the DB with a single call is a bad idea. Even most of them is bad. We should keep the max_search_results limit as the *real* limit and encourage the use of offset param if more than max_search_results is needed. We need to update the docs to state that is limit is excluded, or equals 0, or is greater than max_search_results, then max_search_results will be used instead.

dkl
Flags: needinfo?(dkl)
You need to log in before you can comment on or make changes to this bug.