Status

RESOLVED FIXED
9 years ago
4 years ago

People

(Reporter: clouserw, Assigned: oremj)

Tracking

Details

(Whiteboard: Target: 1/5)

(Reporter)

Description

9 years ago
We talked about this in our meeting, but I guess I never filed the bug. :(

Anyway, AMO would like to use Gearman (http://gearman.org/) for job processing so the front end doesn't have to deal with it.  I installed it on khan with minor issues (it required a newer libevent, but that was in yum) so I think it should be relatively painless.

The gearman model is to have a server that tracks jobs, and workers that actually run the jobs.  My guess is that we could run the server on the admin box (the same one that runs the cron jobs and hudson) and be all right.  If we want to run the workers on there too I'd be willing to give it a shot, but it might be getting crowded - I don't know the specs on the admin box.

I wrote a redhat-compatible init script that you can use if you like at khan:/etc/init.d/gearmand
(Assignee)

Updated

9 years ago
Assignee: server-ops → jeremy.orem+bugs
(Assignee)

Comment 1

9 years ago
How soon do you need this?
(Reporter)

Comment 2

9 years ago
We have a patch ready to go on AMO in early January that could use it.  If we get it on preview sooner we can make sure the patch works and make others.
(Assignee)

Comment 3

9 years ago
What kind of volume are you expecting? We should probably have at least 2 boxes for redundancy, and I'm hoping to put at least one on a vm.
(Reporter)

Comment 4

9 years ago
It's hard to estimate since we haven't done this before.  Right now we have one script and, if it works well, I expect to add to it as we get the time.  Eventually I'd like to have the majority of "unimportant" write traffic on the box, meaning anything that can be delayed (e.g. all the stats updates).
(Reporter)

Updated

9 years ago
Blocks: 536143
(Assignee)

Comment 5

9 years ago
I'll try to get a box up by the 30th.
Whiteboard: Target: 12/30
(Assignee)

Comment 6

9 years ago
Fail, reset target.
Whiteboard: Target: 12/30 → Target: 1/5
(Reporter)

Comment 7

9 years ago
1/5 was our code freeze date.  This is going to bump our schedule. :-/
(Assignee)

Comment 8

9 years ago
What if I set it up on Monday instead?
(Assignee)

Comment 9

9 years ago
The servers are up and I've installed the newest gearman pear package. Do you guys have a script to start up the workers?
(Assignee)

Comment 10

9 years ago
Also, that init script didn't seem to fully work. http://pastebin.mozilla.org/694692 works if you are having problems with it.
(Reporter)

Comment 11

9 years ago
I don't have a script, I just started one manually to test on my dev box.  Do you need a script?  If you have a generic one you can adapt I'd like to do that since we're low on dev time.  (I don't know how plausible that is ;)
(Assignee)

Updated

9 years ago
Depends on: 537951
(Assignee)

Comment 12

9 years ago
This is set up in production. It still needs the worker process monitor, but I'll close this one for now.
Status: NEW → RESOLVED
Last Resolved: 9 years ago
Resolution: --- → FIXED
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.