Closed
Bug 713213
Opened 13 years ago
Closed 13 years ago
increase the Apache2::SizeLimit limit
Categories
(bugzilla.mozilla.org :: General, defect)
Tracking
()
RESOLVED
FIXED
People
(Reporter: glob, Unassigned)
Details
the default Apache2::SizeLimit limit on bugzilla is set to 70,000kb. wicked pointed out that this may be too small for bmo, and i think he's right.
right now the webheads have about 6g of ram each, however we never use more than about 2.5g according to ganglia.
we should see a performance boost if we increase the per-process sizelimit.
i suggest increasing it to 150,000kb.
justdave/dkl/ashish thoughts?
note: the main apache error log will contain messages show when a process is terminated. on bugzilla-stage-tip it looks like:
[Fri Dec 23 05:34:58 2011] (10699) Apache2::SizeLimit httpd process too big, exiting at SIZE=114472 KB SHARE=40504 KB UNSHARED=73968 REQUESTS=91 LIFETIME=5157 seconds
most of the processes on stage-tip when killed are at about this size (110,000kb).
Comment 1•13 years ago
|
||
I agree that increasing the limit would be a good thing to try and see if it gives a boost in performance. We can monitor it closely and make sure nothing bad happens but with 6G of ram 150000k should be safe.
dkl
i grabbed a copy of all the sizelimit kills from one prod webhead, and *every* request is failing after serving 2 requests, with an average lifetime of 0.1 seconds:
count: 3,674,951
avg requests: 2.0
avg lifetime: 0.1
avg unshared: 382,745
[Sun Jan 1 04:02:04 2012] (4769) Apache2::SizeLimit httpd process too big, exiting at SIZE=450236 KB SHARE=80092 KB UNSHARED=370144 REQUESTS=2 LIFETIME=0 seconds
unshared is the value that we need to pay attention to, however bug 633061 states that ASL calculates this incorrectly (it's too high). this isn't completely relevant, as when we upgrade the minimum version to 0.96, the default limit is dropped from 70,000 to 45,000.
there's a big gap between the current limit of 70,000 and the average of 382,745; and i'm not sure we have enough ram on the phx webheads to increase the limit to avoid this situation.
how about we:
- bump up the current limit slowly and measure the impact on memory usage, requests per process, and process lifetime
- use these measurements to determine if we need to add ram to the webheads (currently phx webheads have 6gb, and sjc webheads have 12gb)
Comment 3•13 years ago
|
||
The incremental approach works for me.
bump to 100,000
Committing to: bzr+ssh://bjones%40mozilla.com@bzr.mozilla.org/bmo/4.0/
modified mod_perl.pl
Committed revision 8002.
i'm going to wait for the ram in the webheads to be increased (bug 715425), and then increase the sizelimit to 400,000 and monitor closely.
bump to 400,000
Committing to: bzr+ssh://bjones%40mozilla.com@bzr.mozilla.org/bmo/4.0/
modified mod_perl.pl
Committed revision 8060.
the memory usage increased significantly less after the bump to 400,000. it's now running with a limit of 1,600,000 and there's a large amount of unused memory still.
Committing to: bzr+ssh://bjones%40mozilla.com@bzr.mozilla.org/bmo/4.0/
modified mod_perl.pl
Committed revision 8061.
Committing to: bzr+ssh://bjones%40mozilla.com@bzr.mozilla.org/bmo/4.2/
modified mod_perl.pl
Committed revision 8048.
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•