600482 - PHP to Python migration

Status: RESOLVED WONTFIX

(Cloud Services :: Operations: Miscellaneous, task)

Description

[Tarek Ziadé (:tarek)]

Once the Python server will be ready for production, we will probably want to switch gradually node after node to limit the risks. The Python application is already compatible with the DB so launching it on an existing node will work. But will be incompatible with the cache (see Bug 599018.)

If we want to make the switch transparent for our users so they don't lose their tabs etc., we can add a compatibility layer in Python so it reads the old PHP cache and rewrite new keys on the fly when the user connects. The other solution is to drop the cache altogether and schedule a downtime node by node, and tell our users they'll have to resync their tabs, which is probably not a big deal.

Comment 2

[Tarek Ziadé (:tarek)]

Added JSON support at http://hg.mozilla.org/services/server-storage/rev/31ba351b7b7d

Next step is to control interop with PHP

Comment 3

[Tarek Ziadé (:tarek)]

As discussed with Toby then Mike, a simple way to handle this would be:

- write a script that reads the PHP data and push them back in Python data
- schedule a downtime on the PHP cluster before we turn it into a Python one
- run the script to convert the memcached data

This will avoid any conversion headaches within the servers code.

I suggest that we write this script in Python and train it using a dump of one of the PHP memcached server.

Comment 4

[:Atoll]

I talked with Toby a couple weeks ago about switching ten nodes at a time from PHP to Python, which is totally doable within Zeus.  Since clients always go to the same node, this would effectively lock them to either PHP *or* Python, without ever requiring memcache interop.

It would be *nice* to pre-fill the Python memcache when we switch the users over from PHP.  However, it is not required from an ops perspective, and if we switch users to an empty cache, we avoid a great deal of potential risk related to migrating cache data.

So my vote is slightly in favor of NOT doing anything about this, but either path is acceptable.

Comment 5

[Mike Connor [:mconnor]]

Even better, let's just delete all of the user's data, then we'll have no migration problem at all!</sarcasm>

Barring this being especially hard, we should do proper migration here.

Comment 6

[Tarek Ziadé (:tarek)]

I am +1 for no cache migration at all, with the proper end-user communication.

I guess the next step could be to start an Ops wiki page with the migration scenario, where we can all give some feedback etc.

Comment 7

[Tarek Ziadé (:tarek)]

So, PhP and Python will never share data in memcached. \o/