421424 - Deploy bouncer GeoIP patch

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Jeremy Orem [:oremj]]

Bouncer should now have all the features needed to launch its GeoIP functionality.  

We need to:
 * Come up with a deployment plan
 * Load test http://svn.mozilla.org/projects/bouncer/1.0/branches/production
 * Anything else?
 * Push it live

Our options for deploying are probably:
 * svn up download.mozilla.org
 * Create download-geo.mozilla.org
   * Host a release or ?

Thoughts?

Comment 1

[Jeremy Orem [:oremj]]

All changes between what is now in production and HEAD of the production branch: http://viewvc.svn.mozilla.org/vc/projects/bouncer/1.0/branches/production/?view=query&dir=&file=&file_match=exact&who=&who_match=exact&querysort=date&hours=2&date=explicit&mindate=2008-03-01+00%3A00%3A00&maxdate=&limit_changes=0

Comment 2

[Jeremy Orem [:oremj]]

GeoIP Off = define('GEOIP',false);
GeoIP On = define('GEOIP',true);

Memcache Off = define('CACHE',false);
Memcache On = define('CACHE',true);  

Ran load tests:

Memcache on: 
  GeoIP off: 914 req/s
  GeoIP on: 774 req/s 

  774 / 914 = ~85%

Memcache off:
  GeoIP off: 566 req/s
  GeoIP on: 524 req/s

  524 / 566 = ~93%

Since we should always be running with memcache off we will probably suffer about a 15% performance hit, but this deployment will add a slave database option. Since all of these queries are slave eligible we have the ability to scale bouncer almost horizontally.

Comment 3

[Michael Morgan [:morgamic]]

Do you mean with memcache on?  We run with it on all the time, right???

Comment 4

[Jeremy Orem [:oremj]]

Oops, yes typo s/running with memcache off/running with memcache on/

Comment 5

[Jeremy Orem [:oremj]]

Attachment: Bouncer prod -> GeoIP migration SQL

We also need to run http://people.mozilla.org/~oremj/mirror_ip_to_country.sql.bz2 to import the IP mappings.

Comment 6

[Jeremy Orem [:oremj]]

I think everything is ready to go here.  When would be a good day to do this?  Tuesday is addons php5 and Thur is addons 3.2.  We could either do it 3/20 (with the, possible, 3.2 update) or move it to 3/25?

Comment 7

[Michael Morgan [:morgamic]]

Any reason we can't do it on the 18th?

Comment 8

[Jeremy Orem [:oremj]]

If we are doing the php5 migration on the 18th I'd rather not.  That migration will probably end up taking 4-6 hours.

Comment 9

[Jeremy Orem [:oremj]]

This is out number of mirrors and total "bandwidth" per region:

mysql> select count(region_name), region_name, sum(mirror_rating) from mirror_mirrors as mm join mirror_mirror_region_map as rm on rm.mirror_id = mm.mirror_id join mirror_regions as mr on mr.region_id = rm.region_id group by region_name;
+--------------------+---------------+--------------------+
| count(region_name) | region_name   | sum(mirror_rating) |
+--------------------+---------------+--------------------+
|                  1 | Africa        |                500 |
|                 17 | Asia          |               6203 |
|                  2 | Australia     |                530 |
|                 32 | Europe        |              12460 |
|                 36 | North America |              27863 |
|                  3 | South America |               1100 |
+--------------------+---------------+--------------------+
6 rows in set (0.00 sec)

Comment 10

[Justin Fitzhugh]

Can we get a page in the app to show this breakdown - helpful I think

Comment 11

[Jeremy Orem [:oremj]]

Does this bug depend on that?

Comment 12

[Jeremy Orem [:oremj]]

Anybody object to 3/25?

Comment 13

[Michael Morgan [:morgamic]]

No objection.

Comment 14

[Jeremy Orem [:oremj]]

Code and database have been updated, but GEOIP is set to false for now.