395271 - Use memcache to cache output pages for non logged in users

Status: VERIFIED FIXED

(support.mozilla.org :: General, defect, P1)

Description

[JT Batson]

Memcache and http header improvements w/ netscaler in place by Sept 21st (at the very least, we need these)

Let's use this as a tracking bug. File additional details as necessary.

Comment 1

[JT Batson]

also get netscaler in place

Comment 2

[Michael Morgan [:morgamic]]

Adding TM 0.2 -- NS/cookie fixes buy us time.

Comment 3

[David Tenser [:djst]]

This is not a reasonable target per discussion with morgamic. Moving to 0.5.

Comment 4

[David Tenser [:djst]]

morgamic, any updates on this bug? It's currently targetted for 0.5, which is end of January. Unreasonable?

Comment 5

[David Tenser [:djst]]

-> 0.6. Final target though.

Comment 6

[Laura Thomson :laura]

The goal is to put all non personalized pages that are generated in memcache.

For bonus points, invalidate pages on an individual basis as they are updated.

Comment 8

[Les Orchard [:lorchard]]

Attachment: first round-trip stab at quick and dirty output memcache with expiration on write

I dug into the edit side of things and included what I think is working expiration logic for the caching on the output side of things.  Haven't seen this on any staging servers yet, but let me know how this looks.

Also, per oremj, moving the patch over to this bug for review.

As for locale stuff - I can't find any source of locale info other than the request parameters, which get supplied by the mod_rewrite rules

Comment 9

[Nelson Ko [:nkoth]]

Attachment 321178 [details] [diff] looks ok on code reading, but need to install on my dev server to test. My only comment for now is that instead of gutting the existing Tiki file system cache for wiki, we could add this as an option, and change the option setting for wiki cache to allow for use of memcache in the settings.

Comment 10

[Les Orchard [:lorchard]]

Nelson: I've got some code in there to either (or both) cache page output or wiki page data.  The wiki page data caching happens pretty much in the same spot as the file system cache - though, yeah, it's kind of hacked around the existing cache.

The thing is, though, caching just wiki page data still results in around 110+ DB queries - whereas page output caching results in around 19+ DB queries.  I could be missing something though

Comment 11

[Laura Thomson :laura]

In the process of reviewing this as well...I'd recommend that for pushing it back upstream we put it in as an option.  For now I'm happy to get it working and tested on our servers, and once we're happy with that we should float the changes back upstream.

Comment 12

[Nelson Ko [:nkoth]]

I think also that tiki-rollback.php should invalidate the cache.

I am also wondering about the invalidating of the cache for a page when the staging copy is updated (due to the wikiapproval_prefix stripping). Maybe the invalidation should be when the page is approved, i.e. in tiki-approve_staging_page.php instead.

Comment 13

[Les Orchard [:lorchard]]

Sure - I'll try throwing the cache invalidation into those spots instead.  I was a little unsure of the spot where I'd stuck it, thanks to lack of familiarity with the Tiki code.  I had a sense that there might be better spots in the flow to do that.

Comment 14

[Laura Thomson :laura]

Comment on attachment 321178 [details] [diff] [review]
first round-trip stab at quick and dirty output memcache with expiration on write

Good for me.  Go ahead and commit this, and let's get it up on a) load test and b) support-stage for wider QA.

Comment 15

[Les Orchard [:lorchard]]

Attachment: Another tweak to the patch, moved cache invalidation to approval and rollback pages.

Trying to sort out my SVN check-in.  Until then, here's another patch with what I think should be good to go for testing

Comment 16

[David Tenser [:djst]]

What's the scope of these memcache checkins -- do they apply to the knowledge base articles only, or do they also include forum thread views, e.g. http://support.mozilla.com/tiki-view_forum_thread.php?locale=sv-SE&forumId=1&comments_parentId=54828 ?

Comment 17

[Les Orchard [:lorchard]]

So far, just wiki pages in the knowledge base.

Comment 18

[Les Orchard [:lorchard]]

The code is now checked into SVN, r13362

[svn (projects/sumo)]: 13362: [[e-mail]] Bug 395271: Use memcache to cache output from wiki pages for non logged in users

Comment 19

[Louis-Philippe Huberdeau]

Upstreamed. Implementation changed to avoid some code duplication, but should have an identical result.

Comment 20

[Louis-Philippe Huberdeau]

The part of the commit in the staging approval page is no longer required. I moved the invalidation to the standard page cache invalidation function (yes, tiki got one of those, who would have guessed?), so simply updating the page clears the cache.