We talked about this in our meeting, but I guess I never filed the bug. :( Anyway, AMO would like to use Gearman (http://gearman.org/) for job processing so the front end doesn't have to deal with it. I installed it on khan with minor issues (it required a newer libevent, but that was in yum) so I think it should be relatively painless. The gearman model is to have a server that tracks jobs, and workers that actually run the jobs. My guess is that we could run the server on the admin box (the same one that runs the cron jobs and hudson) and be all right. If we want to run the workers on there too I'd be willing to give it a shot, but it might be getting crowded - I don't know the specs on the admin box. I wrote a redhat-compatible init script that you can use if you like at khan:/etc/init.d/gearmand
How soon do you need this?
We have a patch ready to go on AMO in early January that could use it. If we get it on preview sooner we can make sure the patch works and make others.
What kind of volume are you expecting? We should probably have at least 2 boxes for redundancy, and I'm hoping to put at least one on a vm.
It's hard to estimate since we haven't done this before. Right now we have one script and, if it works well, I expect to add to it as we get the time. Eventually I'd like to have the majority of "unimportant" write traffic on the box, meaning anything that can be delayed (e.g. all the stats updates).
I'll try to get a box up by the 30th.
Whiteboard: Target: 12/30
Fail, reset target.
Whiteboard: Target: 12/30 → Target: 1/5
1/5 was our code freeze date. This is going to bump our schedule. :-/
What if I set it up on Monday instead?
The servers are up and I've installed the newest gearman pear package. Do you guys have a script to start up the workers?
Also, that init script didn't seem to fully work. http://pastebin.mozilla.org/694692 works if you are having problems with it.
I don't have a script, I just started one manually to test on my dev box. Do you need a script? If you have a generic one you can adapt I'd like to do that since we're low on dev time. (I don't know how plausible that is ;)
This is set up in production. It still needs the worker process monitor, but I'll close this one for now.
Status: NEW → RESOLVED
Last Resolved: 9 years ago
Resolution: --- → FIXED
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.