417337 - Upgrade cake to latest stable (1.1.19)

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Public Pages, defect)

Description

[Wil Clouser [:clouserw]]

I'm recommending we upgrade to 1.1.19, but we should hit at least 1.1.15.  The query speed improvements brought tears to my eyes (over 2000% on the front page).

Fred mentioned we modified core cake code for our login/session stuff.  We should double check that.

Also, when we do load testing, lets make sure we're not sacrificing webhead load for db load. kthx.

Comment 1

[Basil Hashem [:baz]]

Are you recommending we target v3.2 for this?

Comment 2

[Fred Wenzel [:wenzel]]

I personally would suggest so. The performance seems to have increased very much; and since 3.2 has to be QAed (or is being QAed) anyway, this would be a good time to do it if we want possible minor breakage (gotta love making up words) not go unnoticed.

As for the session stuff, I added code in order for Cake not to start a session for every single user automatically but rather make a session only when you log in. This is allegedly fixed though. Also, it'd only be a problem anyway if the auto-created session kept the Netscalers from caching.

Comment 3

[Wil Clouser [:clouserw]]

(In reply to comment #1)
> Are you recommending we target v3.2 for this?
> 

At the risk of scope creep, yep.  I don't think it will take long to do this. (famous last words!)

(In reply to comment #2)
\> As for the session stuff, I added code in order for Cake not to start a session
> for every single user automatically but rather make a session only when you log
> in. This is allegedly fixed though. Also, it'd only be a problem anyway if the
> auto-created session kept the Netscalers from caching.
> 

I tried it without and the session stuff didn't work.  I applied your patch verbatim, and it works great.  Now I'm checking if I can alter/reduce your patch (which was only a few lines anyway).  This has been the only thing I've found that hasn't worked out of the box so far.

Comment 4

[Michael Morgan [:morgamic]]

Would it be possible to get this in on trunk prior to 3.2?  This is a top priority...

Comment 5

[Wil Clouser [:clouserw]]

(In reply to comment #4)
> Would it be possible to get this in on trunk prior to 3.2?  This is a top
> priority...
> 

Will do.

Comment 6

[Fred Wenzel [:wenzel]]

If you need any assistance re: User session patch, please let me know.

Comment 7

[Wil Clouser [:clouserw]]

Looking at our test suite, 1.1.19 is only 1 or 2 tests short of passing the same number of tests as our current code.  There is something wrong with the no-cache headers test and the rollback() code.  I think they are pretty fixable, which brings us to the todo list:

1) Get at least as many tests passing on 1.1.19 as pass on the current code. (almost done)
2) Start getting people clicking through the new code to make sure it works
3) Do some basic performance testing of the current code
4) Do some basic performance testing of the new code

For steps 3 and 4 morgamic says we should use the "test cluster which should be up sometime today" - is there a bug for that?  How do we get access to that and what kind of perf testing tools are we going to use?

Comment 8

[Fred Wenzel [:wenzel]]

(In reply to comment #7)
> what kind of perf testing tools are we going to use?

A basic form would be replaying logs from the live site; we should make sure to do this without caching, so we can determine the actual performance of Cake rather than getting results that are biased by memcache and netscalers.

Comment 9

[Wil Clouser [:clouserw]]

oremj:  Can you give me and/or use your replay script and a typical log file on the old and new code?

Comment 10

[Jeremy Orem [:oremj]]

I can do this, but the script is at http://code.google.com/p/logreplay/ in case you get around to it before me.  Talk to me on IRC about getting a log file.

Comment 11

[Wil Clouser [:clouserw]]

I'm blocked on this until I can find a place to do load testing.  It sounds like the test cluster and khan are both running php5 and we're not ready to make that leap yet.  Any other ideas?

Comment 12

[Justin Fitzhugh]

Think oremj was talking about testing it against the staging setup, which is php4...

Comment 13

[Jeremy Orem [:oremj]]

Yep, stating setup wfm.  Just let me know what you want the set up to look like.

Comment 14

[Stephen Donner [:stephend] Not actively reading bugmail]

I'll do functional testing there, too...

Comment 15

[Wil Clouser [:clouserw]]

This is landed on AMO trunk in:

r10573: remove current cake
r10574: add new cake w/ mozilla patches
r10575: add compatibility changes to AMO code base

oremj is currently setting up our production tag on the stage box to test.

Comment 16

[Wil Clouser [:clouserw]]

This went live tonight with bug 420350.  We didn't see the awesome perf gains on the webheads when load testing, but the new code has a lot of bug fixes so I think it was a good thing to upgrade.  I'm still hoping to see some perf gains on the db boxes once we've had a full day of logging.  Thanks to everyone for the help.