1251047 - /rest/bug/field takes 15-25 seconds to return

Status: RESOLVED FIXED

(bugzilla.mozilla.org :: General, defect)

Description

[Dylan Hardison [:dylan] (he/him)]

Comment from bzdeck: https://github.com/bzdeck/bzdeck/issues/312#issuecomment-188454029

> I think we can drop BzAPI now. /field/bug offers the fields, classifications, >
> products, components as well as keywords. It takes 15+ seconds to load 
> https://bugzilla.mozilla.org/rest/field/bug but it's okay because it also takes 
> long to retrieve bugs at firstrun.

time http https://bugzilla.mozilla.org/rest/field/bug > /dev/null 
http https://bugzilla.mozilla.org/rest/field/bug > /dev/null  0.13s user 0.07s system 0% cpu 23.313 total

Comment 1

[Dylan Hardison [:dylan] (he/him)]

Attachment: 1251047_1.patch

Hey, I think we can do something right away about this.

Here's what profiling shows right now:

https://hardison.net/~dylan/bug-1251047/nytprof-fields/

Bugzilla::Object->name is called 820,515 times. Wow!

The body of that method is pretty simple. Maybe nothing we can do about it. But what about Bugzilla::User::can_see_product?

> return scalar(grep {$_->name eq $product_name} @{$self->get_selectable_products});

Ah, grep in scalar context... that's really not ideal. We should use any { } there.
What that code does is call $_->name and compare it on every selectable product. if we change it to any { }, we'll only call it until one of the products match. Still not ideal, but maybe it improves things?

https://hardison.net/~dylan/bug-1251047/nytprof-fields2/

Now there's only 421,453 calls to Bugzilla::Object->name!

Comment 2

[Dylan Hardison [:dylan] (he/him)]

So with this + bug 1251221 + bug 1251208, the differences against my bmo vm:

Before:
dylan@mac-thor ~(master)1? time http http://bmo.local/rest/field/bug >/dev/null
http http://bmo.local/rest/field/bug > /dev/null  0.09s user 0.04s system 0% cpu 13.699 total

After:
dylan@mac-thor ~(master)% time http http://bmo.local/rest/field/bug >/dev/null
http http://bmo.local/rest/field/bug > /dev/null  0.09s user 0.04s system 1% cpu 7.511 total

Comment 3

[Dylan Hardison [:dylan] (he/him)]

(In reply to Dylan William Hardison [:dylan] from comment #0)
> time http https://bugzilla.mozilla.org/rest/field/bug > /dev/null 
> http https://bugzilla.mozilla.org/rest/field/bug > /dev/null  0.13s user
> 0.07s system 0% cpu 23.313 total

After caching more objects:

time http https://bugzilla.mozilla.org/rest/field/bug > /dev/null
http https://bugzilla.mozilla.org/rest/field/bug > /dev/null  0.13s user 0.04s system 5% cpu 3.235 total

After this patch and the request cache one, I predict further improvement. :-)

Comment 4

[David Lawrence [:dkl]]

Comment on attachment 8723369 [details] [diff] [review]
1251047_1.patch

Review of attachment 8723369 [details] [diff] [review]:
-----------------------------------------------------------------

r=dkl

Comment 5

[Dylan Hardison [:dylan] (he/him)]

To ssh://[e-mail]/webtools/bmo/bugzilla.git
   e9b54b1..5129486  master -> master

Comment 6

[Dylan Hardison [:dylan] (he/him)]

broken some tests because of using bsearch rather than any {}. Fixed by commit below.

To ssh://[e-mail]/webtools/bmo/bugzilla.git
   5129486..f6782b1  master -> master

Comment 7

[Dylan Hardison [:dylan] (he/him)]

And more bustage caused by the fact that I'm an idiot.

To ssh://[e-mail]/webtools/bmo/bugzilla.git
   f6782b1..0e0fd13  master -> master