11062 - Page manager takes too much memory

Status: VERIFIED FIXED

(Core :: Networking, defect, P3)

Description

[Colin Blake]

Discussed in the xpcom newsgroup. I am entering a bug report so that
it does get fixed. I think this needs fixing before M9 freezes. Its having
a serious impact on at least two platforms.


In nsPageMgr.cpp, InitPages attempts to allocate a BIG chunk of memory
for itself (32767 pages of 4K each, that's 134MB). If this fails, then
it keeps trying smaller amounts until it eventually succeeds (or reaches
a too-low threshold). Is this really the desired behavior? If the page
manager consumes all available memory, then don't we run the (likely)
risk that a future malloc of some minuscule amount will fail?

Would a better solution be to take some percentage (50%?) of whatever is
the maximum we can get?

How are we ever going to load any more components or plugins if there's
no more memory!!

Comment 1

[Warren Harris]

Mine.

Comment 2

[Warren Harris]

I have a fix for Windows, BeOS and Linux -- making it divide the amount of
memory it's trying to get in 2 instead of subtracting a page and trying too
many times.

But the VMS implementation looks pretty different and I'm afraid to touch it
without being able to verify on VMS. Cc'ing dp since it looks like he inserted
the VMS code from somewhere.

Comment 3

[Suresh Duddi (gone)]

Colin Blake, the person who filed this bug, submitted all the VMS changes.

Comment 4

[Colin Blake]

The code is going to have to remain different for OpenVMS, since we don't have
a /dev/zero device. The $EXPREG system calls expands process memory and is
functionally as close as you can get.

But I fear that trying (n/2) each time until success is only half the fix.
Using this technique, the Page Manager is still likely to consume a large
portion of available memory (anywhere from 50% to 100%) and hence starve
everyone else. The fix I coded for VMS yesterday (but haven't checked in
yet) performs the (n/2) loop to find out how much it can grab, but then when
it succeeds puts THAT memory back and allocates (n/2) one last time, so we
know that we are taking AT MOST 50% of available memory. I can send you this
code if you wish, but its pretty trivial.

Once you have finalized the code for the other platforms, assign the bug to
me and I'l make the VMS changes so that they mimic your new code in
functionality/algorithm as much as possible. Note I'll be on vacation 8/5
thru 8/12 so I won't be checking anything in for a while.

Comment 5

[Warren Harris]

I reduced the max page count which should help a bit, but I want to revisit
this later. Setting milestone for m10.

Comment 6

[Warren Harris]

*** Bug 10850 has been marked as a duplicate of this bug. ***

Comment 7

[Warren Harris]

I've disabled the page manager until we have more cycles to make it work
properly.

Comment 8

[leger]

Bulk move of all Necko (to be deleted component) bugs to new Networking

component.