Last Comment Bug 987032 - allow memcached to cache bugzilla configuration information
: allow memcached to cache bugzilla configuration information
: bmo-goal, perf
Product: Bugzilla
Classification: Server Software
Component: Bugzilla-General (show other bugs)
: unspecified
: All All
-- enhancement (vote)
: Bugzilla 5.0
Assigned To: Byron Jones ‹:glob›
: default-qa
Depends on:
  Show dependency treegraph
Reported: 2014-03-24 01:02 PDT by Byron Jones ‹:glob›
Modified: 2014-04-30 00:59 PDT (History)
2 users (show)
glob: approval+
See Also:
QA Whiteboard:
Iteration: ---
Points: ---

987032_1.patch (17.32 KB, patch)
2014-03-26 09:58 PDT, Byron Jones ‹:glob›
dkl: review-
Details | Diff | Splinter Review
987032_2.patch (17.39 KB, patch)
2014-04-03 00:27 PDT, Byron Jones ‹:glob›
dkl: review+
Details | Diff | Splinter Review

Description User image Byron Jones ‹:glob› 2014-03-24 01:02:37 PDT
we need to be able to cache bugzilla's cache data in memcached, with a means to invalid it.

there's quite a few database calls which are executed during show_bug which are prime candidates for caching with memcached:

> SELECT id,name,description,type,custom,mailhead,sortkey,obsolete,enter_bug,buglist,visibility_field_id,value_field_id,reverse_desc,is_mandatory,is_numeric FROM fielddefs ORDER BY sortkey, name
> SELECT DISTINCT grantor_id, member_id FROM group_group_map WHERE grant_type = 0
> SELECT value FROM bug_status WHERE is_open = 1
> SELECT id,value,sortkey,isactive,visibility_value_id FROM resolution ORDER BY sortkey, value
> SELECT,,keyworddefs.description FROM keyworddefs ORDER BY name
> SELECT id,name,classification_id,description,isactive,defaultmilestone,allows_unconfirmed,nag_interval,reviewer_required,votesperuser,maxvotesperbug,votestoconfirm FROM products ORDER BY name
> SELECT id,name,description,sortkey FROM classifications ORDER BY sortkey, name
> SELECT id,value,sortkey,isactive,visibility_value_id FROM rep_platform ORDER BY sortkey, value
> SELECT id,value,sortkey,isactive,visibility_value_id FROM op_sys ORDER BY sortkey, value
> SELECT id,value,sortkey,isactive,visibility_value_id FROM priority ORDER BY sortkey, value
> SELECT id,value,sortkey,isactive,visibility_value_id FROM bug_severity ORDER BY sortkey, value
> SELECT id,value,sortkey,isactive,visibility_value_id,is_open FROM bug_status ORDER BY sortkey, value
> SELECT id,value,sortkey,isactive,visibility_value_id FROM resolution ORDER BY sortkey, value

in order to achieve this, we should:
- add a 'config_prefix' property to Bugzilla::Memcached
  - similar implementation to the default "prefix"
- any time an administrator updates a product, component, group, field, flag, etc, inc the config_prefix value to invalidate all prior entries
- use the config_prefix value when building keys for caching the lists of products, statuses, resolutions, etc
Comment 1 User image Byron Jones ‹:glob› 2014-03-26 09:58:36 PDT
Created attachment 8397189 [details] [diff] [review]

- adds an IS_CONFIG constant to bugzilla::object
- sets IS_CONFIG to true on the following objects:
  - bugzilla::group
  - bugzilla::classification
  - bugzilla::product
  - bugzilla::keyword
  - bugzilla::field
  - bugzilla::field::choice
- adds helper methods to bugzilla::memcached for setting/getting config values
- adds a 'config prefix' which is incremented to invalid all cached config values
- updates bugzilla::object::_do_list_select() to automatically cache results of IS_CONFIG queries which are unconditional
- stores the list of open bug states in memcached
- stores the list of group_group_map's with grant_type = MEMBERSHIP in memcached

this removes the following database queries during show_bug:
- 10 selects for all rows in a table (!)
- 1 select for open bug states
- 1 full table select per custom single/multi value field
- 1 table select per user object (group_group_map)
Comment 2 User image David Lawrence [:dkl] 2014-04-02 13:35:48 PDT
Comment on attachment 8397189 [details] [diff] [review]

Review of attachment 8397189 [details] [diff] [review]:

Mostly works fine except for a small annoyance. You need to update Bugzilla->{request_cache}->{memcache_prefix_config} (and probably the other types as well) with the new prefix value when doing Bugzilla->memcache->clear_config(). I noticed this when editing values of drop downs such as 'bug_severity' and others. When you edit it's value or disable the value and hit submit, the list in the next page still has the old settings as it is still has the old prefix in request_cache. I don't think this would cause real issues in general use but it is just an annoyance.

Comment 3 User image Byron Jones ‹:glob› 2014-04-03 00:27:41 PDT
Created attachment 8401118 [details] [diff] [review]

ah, well spotted.  this revision fixes that.
Comment 4 User image David Lawrence [:dkl] 2014-04-04 10:24:50 PDT
Comment on attachment 8401118 [details] [diff] [review]

Comment 5 User image Byron Jones ‹:glob› 2014-04-07 01:43:29 PDT
To ssh://
   dc9486b..3ef245b  master -> master

Note You need to log in before you can comment on or make changes to this bug.