523955 - Add memcache config for authstage.mozillla.com & mozilla.com

Status: RESOLVED WONTFIX

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Ryan Doherty (:rdoherty)]

Can we get memcache configuration added to includes/config.inc.php on authstage.mozilla.com & mozilla.com?

This is preparation for pushing a patch for bug 500849 (improving yslow scores by concatenation css & js). The concatenation script uses memcache to avoid reading from disk on every request.

Memcache config example can be viewed at the bottom of includes/config.inc.php-dist on trunk and authstage.

Let me know if there's questions

Comment 1

[Derek Moore [:dmoore]]

Ryan,

I've updated the config for http://www.authstage.mozilla.com

www.mozilla.com is a different matter, however, as the memcache config is more complicated. We're discussing this internally, and it's not yet deployed.

Comment 2

[Ryan Doherty (:rdoherty)]

(In reply to comment #1)
> www.mozilla.com is a different matter, however, as the memcache config is more
> complicated. We're discussing this internally, and it's not yet deployed.

Thanks, let us know when it's ready and we'll push.

Comment 3

[Michael Morgan [:morgamic]]

How long do you think it'll take?  Can we get an ETA?

Comment 4

[Derek Moore [:dmoore]]

mozilla.com is geographically distributed, so we're going to need to decide how to properly integrate a memcache configuration for multiple locations. Staging is simple, as memcached runs locally on the webserver itself. We do not run memcache on the production webservers, though... we have dedicated memcache servers.

Comment 5

[Michael Morgan [:morgamic]]

It's not proxied?  So we have actual mozilla.com webheads elsewhere?

If that's the case you could spin up small memcache partitions locally just for this.  I don't think we need much memory for this, honestly.  And if memcache is a problem we can engineer around it and use local files instead.  Would spinning of a 128M memcache partition be difficult to do in our other datacenters?

Comment 6

[Dave Miller [:justdave]]

For reference... there are nodes for www.mozilla.com in Amsterdam and Beijing.  Those two locations do not have any memcache servers.  We'll need to deploy memcache in those two locations first (which could be months - depends on hardware delivery, etc).

The other viable option would be to stop serving www.mozilla.com from those locations and just serve it from San Jose, but that seems like a step backwards.

If there's enough application backend to need memcache, the site is really straying from what the static cluster was intended for, and it really needs to be moved to another cluster that has support for web applications.

Comment 7

[Dave Miller [:justdave]]

I midaired with morgamic on that, and submitted anyway.

We *could* change it to proxy from those locations instead of actually serving it from there (similar to how AMO works).  That'd be easy to do I suspect.

Comment 8

[Michael Morgan [:morgamic]]

The idea is just having a small bucket to place the pre-compiled output for the minified JS and CSS.  We're not having load issues with mozilla.com at all, but this is an easy way to reduce overall traffic that we think is worthwhile.

Comment 9

[Michael Morgan [:morgamic]]

For Amsterdam and Beijing I think we can modify the minification script to simply fallback to file cache if memcache does not exist.  Doesn't seem like it's really worth it to set up memcache just for this.

Comment 10

[Derek Moore [:dmoore]]

To echo justdave, the dependency on memcache does imply we should move this to a cluster designated for dynamic webapps. As an upside, this would make it easier to integrate into the global load balancing / proxy system currently in place.

Comment 11

[Michael Morgan [:morgamic]]

Nevermind -- the minify code already falls back to /tmp if memcache doesn't exist.  Like I said in comment #8, redoing the static cluster to support memcache is not a good use of time.

Comment 12

[matthew zeier [:mrz]]

fyi, we -only- have origin servers in San Jose and Beijing.  Amsterdam was rebuilt - proxy/cache only.