Closed Bug 962424 Opened 10 years ago Closed 10 years ago

jobqueue's worker process should process messages in batches

Categories

(Bugzilla :: Email Notifications, defect)

defect
Not set
normal

Tracking

()

RESOLVED FIXED
Bugzilla 5.0

People

(Reporter: glob, Assigned: glob)

Details

Attachments

(1 file)

recently on bmo we had an incident where someone made a lot of changes within a small period of time, resulting in just over 60,000 emails being inserted into the jobqueue for delivery.

while processing this queue, the jobqueue worker process's memory usage grew to about 1.5g per process, which if problematic as we run 16 jobqueue daemons on our servers.

if there are messages to deliver, we spawn a worker process which executes until the queue is empty (see bug 832893).  a simple fix would be to put an upper limit on the number of messages a worker will process.  when that limit is hit, it terminates, and a new worker will be spawned to continue processing the queue.
Attached patch 962424_1.patchSplinter Review
Attachment #8363494 - Flags: review?(dkl)
Comment on attachment 8363494 [details] [diff] [review]
962424_1.patch

Review of attachment 8363494 [details] [diff] [review]:
-----------------------------------------------------------------

r=dkl
Attachment #8363494 - Flags: review?(dkl) → review+
Flags: approval?
Flags: approval? → approval+
Target Milestone: --- → Bugzilla 5.0
To ssh://gitolite3@git.mozilla.org/bugzilla/bugzilla.git
   617eeba..4d5e362  master -> master
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: