Require Apache2::SizeLimit 0.96 for proper operation on Linux

RESOLVED FIXED in Bugzilla 4.2

Status

()

Bugzilla
Installation & Upgrading
RESOLVED FIXED
7 years ago
6 years ago

People

(Reporter: Max Kanat-Alexander, Assigned: Max Kanat-Alexander)

Tracking

Bugzilla 4.2
Bug Flags:
approval +
approval4.2 +
blocking4.2 +

Details

(URL)

Attachments

(1 attachment)

(Assignee)

Description

7 years ago
Currently, Apache::SizeLimit determines the wrong "unshared" size for Apache processes on Linux. I just submitted a patch that fixes this to the Apache::SizeLimit maintainer that he says that he will be integrating as part of 0.96. He says that 0.96 should be out tomorrow.

Once I applied this patch, I also noticed that 70MB was a *much* too large size for the set_max_unshared_size value. Apache quickly takes up all of my memory if I use 70MB for that value and run the webservice test suite. Upon experimentation, I've found that about 45MB is a good size. This lets httpd children live for a long time (about 80 requests, in my tests), but doesn't take up all my RAM when running the webservice QA tests.
(Assignee)

Comment 1

7 years ago
Created attachment 511268 [details] [diff] [review]
v1
Assignee: installation → mkanat
Status: NEW → ASSIGNED
Attachment #511268 - Flags: review?(dkl)
Still waiting on 0.96 to show up in CPAN. Is it there for you?
(Assignee)

Comment 3

7 years ago
Ah, I don't think it's out, and it may not actually resolve this problem, as it turns out. There's a small debate happening about it.
(Assignee)

Comment 4

7 years ago
0.96 rc1 is out, and it does have the fix in it so the final 0.96 should be out in a few days.
(Assignee)

Updated

7 years ago
Blocks: 532928

Comment 5

7 years ago
There's also another problem with all these SizeLimit stuff. mod_perl 2.0.4 ships 0.03 version of SizeLimit. When you install mod_perl and SizeLimit both, the SizeLimit one is not detected and the 0.03 is used. You have to manually remove it from mod_perl and everything works Ok.

But this is unmaintainable for distros. mod_perl just released 2.0.5 which includes SizeLimit 0.95. Using that version fixes all the problems.
(Assignee)

Comment 6

7 years ago
(In reply to comment #5)
> There's also another problem with all these SizeLimit stuff. mod_perl 2.0.4
> ships 0.03 version of SizeLimit. When you install mod_perl and SizeLimit both,
> the SizeLimit one is not detected and the 0.03 is used. You have to manually
> remove it from mod_perl and everything works Ok.

  Yes, in normal Bugzilla installation situations, this is overcome by using install-module.pl.

> But this is unmaintainable for distros.

  I do understand that, it's a difficult situation we've run into before.

> mod_perl just released 2.0.5 which
> includes SizeLimit 0.95. Using that version fixes all the problems.

  0.95 does not have the fix required to make it actually work properly on Linux. 0.96 will have it.
(Assignee)

Comment 7

7 years ago
I've added a link to the URL field that explains in high technical detail exactly how memory consumption works on Linux, so that anybody interested in the very technical details of this bug can understand them. (Also, so that we can easily find this link and send people to it in the future.)

Comment 8

7 years ago
I think it should block 4.2rc1 (to have the version bump before the RC stage). If mkanat disagrees, feel free to remove the flag.
Flags: blocking4.2+
Unless I missing something, I still only see 0.95 in CPAN. Any idea when the author is going to publish?

dkl

Comment 10

7 years ago
I just looked on CPAN, and I realized that Apache2::SizeLimit is part of mod_perl. So isn't it enough to require mod_perl to have this module automatically installed?

Comment 11

7 years ago
Also, I don't think this should go into 4.0.x as some distros may not want to upgrade their mod_perl package just for Bugzilla (or for some other reasons), and this would prevent them from updating their Bugzilla package.
(Assignee)

Comment 12

7 years ago
No, it's "dual-lifed" like core Perl modules are. It has a CPAN module and it's *also* part of mod_perl. The newer versions are CPAN modules, the older versions are part of mod_perl.

Comment 13

7 years ago
The most recent Apache2::SizeLimit I can find available for download on CPAN is 0.95.  And that is part of the modperl 2.0.5, not a separate package.

http://search.cpan.org/search?query=Apache2%3A%3ASizeLimit&mode=all
(Assignee)

Comment 14

7 years ago
Right. 0.96 isn't out yet.

Comment 15

7 years ago
Retargetting to 4.2 as we shouldn't bump the min version required on stable branches.
Target Milestone: Bugzilla 4.0 → Bugzilla 4.2

Comment 16

7 years ago
Not a blocker as 0.96 isn't out yet. We will take it when it's available, if this happens before 4.2 final.
Flags: blocking4.2+ → blocking4.2-
0.96 was released on 2011-12-21.
Flags: blocking4.2- → blocking4.2?

Comment 18

7 years ago
The doc says that reading /proc/self/smaps is expensive and that we should consider using Apache2::SizeLimit->set_check_interval(). I see it's not used by Bugzilla. May we should?

Comment 19

7 years ago
While you are on it, shouldn't you also bump the min version of mod_perl to 2.0.2 (released on October 20, 2005)? Its changelog says:

- Add support for the newer Smaps (/proc/self/statm) on Linux systems that support it to accurately count the amount of shared memory.

Also, there is a comment in mod_perl.pl which says:

    # $0 is broken under mod_perl before 2.0.2, so we have to set it
    # here explicitly or init_page's shutdownhtml code won't work right.
(Assignee)

Comment 20

7 years ago
(In reply to Frédéric Buclin from comment #18)
> The doc says that reading /proc/self/smaps is expensive and that we should
> consider using Apache2::SizeLimit->set_check_interval(). I see it's not used
> by Bugzilla. May we should?

  It hasn't presented any measurable problem yet, and we need to kill large processes as quickly as possible. Otherwise they could sit around until they get called again.

(In reply to Frédéric Buclin from comment #19)
> While you are on it, shouldn't you also bump the min version of mod_perl to
> 2.0.2 (released on October 20, 2005)? Its changelog says:
> 
> - Add support for the newer Smaps (/proc/self/statm) on Linux systems that
> support it to accurately count the amount of shared memory.

  That's actually a note about Apache::SizeLimit, which is bundled with mod_perl. In other words, it's earlier release notes of this library. The release didn't actually work right with smaps, only the most recent Apache::SizeLimit does.

  So for now, there's not enough reason to change the min requirement.
(Assignee)

Updated

7 years ago
Flags: blocking4.2? → blocking4.2+
Comment on attachment 511268 [details] [diff] [review]
v1

Review of attachment 511268 [details] [diff] [review]:
-----------------------------------------------------------------

Works fine for me. install-module.pl pulls down the correct 0.96 version. After proper configuration
I am able to observe proper behavior in the logs when adjusting the max_unshared_size value where
Apache kills off children once the max has been passed. r=dkl
Attachment #511268 - Flags: review?(dkl) → review+

Updated

6 years ago
Flags: approval?
Flags: approval4.2?
(Assignee)

Comment 22

6 years ago
Sweet! We should include this in rc2 in case people have trouble installing it or other problems. (Installing mod_perl-related modules requires you to have mod_perl-devel installed on CentOS, which can confuse people.)

If it causes people too much trouble, we might have to back it out and just point major installs at it, for 4.2.
Flags: approval?
Flags: approval4.2?
Flags: approval4.2+
Flags: approval+

Comment 23

6 years ago
We will have to update the relnotes before releasing rc2 to mention the 0.93 -> 0.96 change. Committing it now to see what really still need patches/reviews.

Committing to: bzr+ssh://lpsolit%40gmail.com@bzr.mozilla.org/bugzilla/trunk/
modified mod_perl.pl
modified Bugzilla/Install/Requirements.pm
Committed revision 8089.

Committing to: bzr+ssh://lpsolit%40gmail.com@bzr.mozilla.org/bugzilla/4.2/
modified mod_perl.pl
modified Bugzilla/Install/Requirements.pm
Committed revision 8011.
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago
Keywords: relnote
Resolution: --- → FIXED

Comment 24

6 years ago
Added to relnotes for 4.2rc2.
Keywords: relnote
You need to log in before you can comment on or make changes to this bug.