987032 - allow memcached to cache bugzilla configuration information

Status: RESOLVED FIXED

(Bugzilla :: Bugzilla-General, enhancement)

Description

[:glob ✱]

we need to be able to cache bugzilla's cache data in memcached, with a means to invalid it.


there's quite a few database calls which are executed during show_bug which are prime candidates for caching with memcached:

> SELECT id,name,description,type,custom,mailhead,sortkey,obsolete,enter_bug,buglist,visibility_field_id,value_field_id,reverse_desc,is_mandatory,is_numeric FROM fielddefs ORDER BY sortkey, name
> SELECT DISTINCT grantor_id, member_id FROM group_group_map WHERE grant_type = 0
> SELECT value FROM bug_status WHERE is_open = 1
> SELECT id,value,sortkey,isactive,visibility_value_id FROM resolution ORDER BY sortkey, value
> SELECT keyworddefs.id,keyworddefs.name,keyworddefs.description FROM keyworddefs ORDER BY name
> SELECT id,name,classification_id,description,isactive,defaultmilestone,allows_unconfirmed,nag_interval,reviewer_required,votesperuser,maxvotesperbug,votestoconfirm FROM products ORDER BY name
> SELECT id,name,description,sortkey FROM classifications ORDER BY sortkey, name
> SELECT id,value,sortkey,isactive,visibility_value_id FROM rep_platform ORDER BY sortkey, value
> SELECT id,value,sortkey,isactive,visibility_value_id FROM op_sys ORDER BY sortkey, value
> SELECT id,value,sortkey,isactive,visibility_value_id FROM priority ORDER BY sortkey, value
> SELECT id,value,sortkey,isactive,visibility_value_id FROM bug_severity ORDER BY sortkey, value
> SELECT id,value,sortkey,isactive,visibility_value_id,is_open FROM bug_status ORDER BY sortkey, value
> SELECT id,value,sortkey,isactive,visibility_value_id FROM resolution ORDER BY sortkey, value

in order to achieve this, we should:
- add a 'config_prefix' property to Bugzilla::Memcached
  - similar implementation to the default "prefix"
- any time an administrator updates a product, component, group, field, flag, etc, inc the config_prefix value to invalidate all prior entries
- use the config_prefix value when building keys for caching the lists of products, statuses, resolutions, etc

Comment 1

[:glob ✱]

Attachment: 987032_1.patch

- adds an IS_CONFIG constant to bugzilla::object
- sets IS_CONFIG to true on the following objects:
  - bugzilla::group
  - bugzilla::classification
  - bugzilla::product
  - bugzilla::keyword
  - bugzilla::field
  - bugzilla::field::choice
- adds helper methods to bugzilla::memcached for setting/getting config values
- adds a 'config prefix' which is incremented to invalid all cached config values
- updates bugzilla::object::_do_list_select() to automatically cache results of IS_CONFIG queries which are unconditional
- stores the list of open bug states in memcached
- stores the list of group_group_map's with grant_type = MEMBERSHIP in memcached

this removes the following database queries during show_bug:
- 10 selects for all rows in a table (!)
- 1 select for open bug states
- 1 full table select per custom single/multi value field
- 1 table select per user object (group_group_map)

Comment 2

[David Lawrence [:dkl]]

Comment on attachment 8397189 [details] [diff] [review]
987032_1.patch

Review of attachment 8397189 [details] [diff] [review]:
-----------------------------------------------------------------

Mostly works fine except for a small annoyance. You need to update Bugzilla->{request_cache}->{memcache_prefix_config} (and probably the other types as well) with the new prefix value when doing Bugzilla->memcache->clear_config(). I noticed this when editing values of drop downs such as 'bug_severity' and others. When you edit it's value or disable the value and hit submit, the list in the next page still has the old settings as it is still has the old prefix in request_cache. I don't think this would cause real issues in general use but it is just an annoyance.

dkl

Comment 3

[:glob ✱]

Attachment: 987032_2.patch

ah, well spotted.  this revision fixes that.

Comment 4

[David Lawrence [:dkl]]

Comment on attachment 8401118 [details] [diff] [review]
987032_2.patch

r=dkl

Comment 5

[:glob ✱]

To ssh://[e-mail]/bugzilla/bugzilla.git
   dc9486b..3ef245b  master -> master