Status

task
RESOLVED FIXED
10 years ago
4 years ago

People

(Reporter: clouserw, Assigned: oremj)

Tracking

Details

(Whiteboard: Target: 1/5)

Reporter

Description

10 years ago
We talked about this in our meeting, but I guess I never filed the bug. :(

Anyway, AMO would like to use Gearman (http://gearman.org/) for job processing so the front end doesn't have to deal with it.  I installed it on khan with minor issues (it required a newer libevent, but that was in yum) so I think it should be relatively painless.

The gearman model is to have a server that tracks jobs, and workers that actually run the jobs.  My guess is that we could run the server on the admin box (the same one that runs the cron jobs and hudson) and be all right.  If we want to run the workers on there too I'd be willing to give it a shot, but it might be getting crowded - I don't know the specs on the admin box.

I wrote a redhat-compatible init script that you can use if you like at khan:/etc/init.d/gearmand
Assignee

Updated

10 years ago
Assignee: server-ops → jeremy.orem+bugs
Assignee

Comment 1

10 years ago
How soon do you need this?
Reporter

Comment 2

10 years ago
We have a patch ready to go on AMO in early January that could use it.  If we get it on preview sooner we can make sure the patch works and make others.
Assignee

Comment 3

10 years ago
What kind of volume are you expecting? We should probably have at least 2 boxes for redundancy, and I'm hoping to put at least one on a vm.
Reporter

Comment 4

10 years ago
It's hard to estimate since we haven't done this before.  Right now we have one script and, if it works well, I expect to add to it as we get the time.  Eventually I'd like to have the majority of "unimportant" write traffic on the box, meaning anything that can be delayed (e.g. all the stats updates).
Reporter

Updated

10 years ago
Blocks: 536143
Assignee

Comment 5

10 years ago
I'll try to get a box up by the 30th.
Whiteboard: Target: 12/30
Assignee

Comment 6

10 years ago
Fail, reset target.
Whiteboard: Target: 12/30 → Target: 1/5
Reporter

Comment 7

10 years ago
1/5 was our code freeze date.  This is going to bump our schedule. :-/
Assignee

Comment 8

10 years ago
What if I set it up on Monday instead?
Assignee

Comment 9

10 years ago
The servers are up and I've installed the newest gearman pear package. Do you guys have a script to start up the workers?
Assignee

Comment 10

10 years ago
Also, that init script didn't seem to fully work. http://pastebin.mozilla.org/694692 works if you are having problems with it.
Reporter

Comment 11

10 years ago
I don't have a script, I just started one manually to test on my dev box.  Do you need a script?  If you have a generic one you can adapt I'd like to do that since we're low on dev time.  (I don't know how plausible that is ;)
Assignee

Updated

10 years ago
Depends on: 537951
Assignee

Comment 12

10 years ago
This is set up in production. It still needs the worker process monitor, but I'll close this one for now.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.