979727 - increase the mod_perl sizelimit to 700_000 on production

Status: RESOLVED FIXED

(bugzilla.mozilla.org :: General, defect)

Description

[:glob ✱]

currently we're only averaging 12 requests per httpd worker before it exceeds our per-worker memory limit, with a 4 minute lifetime.

this is poor :(

due to bugzilla's bursty memory usage, it's difficult to determine an appropriate value that wouldn't cause us to occasionally hit swap (see bug 849721).  as monitoring tools work on averages, spikes tend to hard to see.

i think we should bump it up to 700_000 and keep an eye on things.

Comment 1

[:glob ✱]

Committing to: bzr+ssh://[e-mail]/bmo/4.2/
modified mod_perl.pl
Committed revision 9265.

Comment 2

[Philippe M. Chiasson (:gozer)]

(In reply to Byron Jones ‹:glob› from comment #0)
> currently we're only averaging 12 requests per httpd worker before it
> exceeds our per-worker memory limit, with a 4 minute lifetime.
> 
> this is poor :(

Indeed.

> due to bugzilla's bursty memory usage

From anecdotal evidence poking at bugzilla, I think a lot of that memory usage is caused by not making
enough use of preloading and getting the benefits of sharing the httpd parent's memory in the spawned
children.

Just my 0.02$

Comment 3

[:glob ✱]

we're now averaging 23 minutes before a worker is killed, after processing an average of 47 requests.
as expected average memory usage has increased on the webheads, to approx 4g (up from 2.5g).

Comment 4

[Kendall Libby [:fubar] (he/him)]

Mentioned elsewhere, but changes to the size limit need corresponding changes to the kill_pigs cron job, or the cron will wreak havoc.