502245 - Move memcache configuration to an administrative panel

Status: RESOLVED FIXED

(support.mozilla.org :: Knowledge Base Software, task)

Description

[E. Cooper [:ecooper]]

As part of bug 502087, we need to move the memcache configuration to the admin panel in SUMO.

Or, alternatively (and also what I'm in favor of), is that we retool cachelib to become a wrapper of for all the caching methods available. Mediawiki currently uses this sort of method, and we could expand on it pretty easily: http://svn.wikimedia.org/doc/ObjectCache_8php-source.html

Comment 1

[Laura Thomson :laura]

I like option B but the QA scares me.

I vote for option a) and we'll do b) later (file a bug for Future).

Comment 2

[E. Cooper [:ecooper]]

Attachment: v0.1

This moves _all_ memcache related configuration to the administrative sections of tiki, which is a little hairy.

I followed tiki protocol for the most part, which means that administration is spread across multiple panels. It's enabled under Admin->Features. It's config'd at webroot/tiki-admin_memcache.php. Memcache can be purged at webroot/tiki-admin_system.php, also where the other cache 'exterminators' are.

I ran memcached with the -vv flag and everything seems to be working as it should. While testing locally, remember to comment out the memcache config in db/local.php.

webroot/tiki-admin_memcache.php supports all the options we use for SUMO and allows for multiple memcache servers to be added/removed in a rather rudimentary way.

Comment 3

[Laura Thomson :laura]

Comment on attachment 387526 [details] [diff] [review]
v0.1

I'm ok with all the code.  You'll need to spend some time with stephend to explain how to test this, and it will need extensive QA.

Comment 4

[E. Cooper [:ecooper]]

How should we introduce this code into the tree? Should we wait until after 1.2.1 is shipped?

As part of bug 502101, I'm going to do a simple write-up on how the panel works (which should also explain how to tell if it's not working). I plan on doing this as I commit each new feature in bug 502087.

Comment 5

[Laura Thomson :laura]

Eric: yes, I think we should wait until 1.2.1 is tagged at least.  That will happen later today.

Comment 6

[E. Cooper [:ecooper]]

Before I can commit this, do we know the settings on stage? We'll need those to set up the new memcache panel.

Adding this patch will go like this:
*Add patch to codebase
*Enable memcache and add configuration in tiki-admin_memcache.php
*Remove memcache settings from db/local.php

This will provide an overlap where the settings will be loaded in one way or another.

Comment 7

[Laura Thomson :laura]

(In reply to comment #6)
> Before I can commit this, do we know the settings on stage? We'll need those to
> set up the new memcache panel.
> 

Jeremy should be able to help with this.

Comment 8

[Reed Loden [:reed]]

Why can't the memcache config be in the config file? Putting it in the database makes it a pain for IT to change later if needed.

Comment 9

[Paul Craciunoiu [:paulc]]

Perhaps it can be in a config file that's changeable through web UI. As a webdev, I can definitely see the benefit of being able to change it without requiring IT intervention.

Comment 10

[Jeremy Orem [:oremj]]

What kind of memcache settings will be stored? Server location and port should probably be kept in the config file. If we need to quickly switch memcache servers, because one is malfunctioning and the site is down we'll be in trouble.

Comment 11

[Reed Loden [:reed]]

WebDev would have no reason to change the memcache configuration, though. Only IT knows how that's set up.

Comment 12

[Laura Thomson :laura]

We could have the admin panel write to the config file. This is part of the productization effort, which means we don't actually necessarily want it for *sumo*.  We just want to make things easier for our users.  Also from a debugging perspective it's easier for devs if we can switch memcache on and off easily.

Comment 13

[Reed Loden [:reed]]

(In reply to comment #12)
> We could have the admin panel write to the config file.

That wouldn't work, as there's a cluster backing SUMO. Having the admin panel change a config would just mean that 1) only that one webhead is updated; 2) the change would be wiped out the next time git is run.

Comment 14

[Laura Thomson :laura]

How about this: Admin panel gives you a choice of 
1) config from file (use command line to configure) - have this set for SUMO
2) config from database - for other users of the product

We also need to be able to disable it from the UI, so that will be a setting in the DB.

Comment 15

[E. Cooper [:ecooper]]

We can just update the patch to have backwards compatibility with having the settings in db/local.php (it practically works that way now, anyhow). That is, if the settings are defined in local.php, then those settings are used regardless of whether or not the tiki memcached preferences are set. If the memcache config isn't present in local.php, then the tiki preferences are used.

Make sense?

Comment 16

[E. Cooper [:ecooper]]

Another option is to keep the memcached config in a file as it is now, and just keep the master on/off preference in this patch.

Comment 17

[E. Cooper [:ecooper]]

Reed, Laura, Jeremy? 

I'm leaning towards comment 16.

Comment 18

[Paul Craciunoiu [:paulc]]

As a general concern, we may want to not include memcachelib when disabled. Even disabling memcache still requires the server to have the memcache libraries installed. I just tried uninstalling it locally and got all sorts of problems.

Comment 19

[Paul Craciunoiu [:paulc]]

Attachment: patch, v2

* memcache settings in local.php overwrite db settings
* turning off memcache feature (feature_memcached) or setting memcache enabled to false causes memcache to be entirely disabled -- tested on machine with no memcached installed
* turning off memcache feature causes memcachelib.php to not be included at all

Comment 20

[Laura Thomson :laura]

Comment on attachment 396052 [details] [diff] [review]
patch, v2

Need extra QA on this but I <3 this feature.  Paul, please provide test cases for QA.

Comment 21

[Laura Thomson :laura]

Comment on attachment 396052 [details] [diff] [review]
patch, v2

I forgot to hit r+

Comment 22

[Paul Craciunoiu [:paulc]]

r50330 / r50331

To test the admin, you need to *NOT* have a db/local.php file (!) -- filed bug 513104 for the testcase of existing vs not existing file. Stephen: you'll need to work with IT for this part, if you wish to test changes when going from local.php to db.

Once local.php is removed, go to: tiki-admin_memcache.php, switch enabled feature on/off, try different seconds for caching and play with the options for cache wiki/forum/sessions to see if content gets updated instantly or you have to wait until cache expires.

I can imagine this will be fairly annoying to test if you were to wait around :)

Laura or IT: for the "Servers" part, perhaps you can give Stephen some pointers? I'm not sure what, if anything but the default, can be tested.

Comment 23

[Paul Craciunoiu [:paulc]]

r50454 / r50455
Forgot the memcache page:
tiki-admin_memcache.php

Comment 24

[Stephen Donner [:stephend] Not actively reading bugmail]

Post bug 513104, I load https://support-stage.mozilla.org/tiki-admin_memcache.php while logged in as an Admin, and get, "Error

Memcache is currently disabled. It can be enabled in the 'Features' panel."

I think I'm hitting bug 513840, so sorry I couldn't test more tonight; I'll look at it tomorrow.

Comment 25

[Marc Laporte]

There has been no tiki-admin_memcache.php upstreamed.

LP has upstreamed an admin panel which you can see here:
http://tiki-trunk.mozilla.com/tiki-admin.php?page=performance

But does this cover it all? (I see no "empty cache") If not, please ask LP to generalize and upstream.

Thanks!