76329 - nsComponentManager uses too much memory for registry

Status: RESOLVED FIXED

(Core :: XPCOM, defect)

Description

[Daniel Veditz [:dveditz]]

To speed startup time the component manager sets a very large buffer for the 
component registry. It was shrinking it at the end of XPCOM initialization, but 
then lots of other startup tasks (intl charset converters and various category 
things) re-read the same data repeatedly.

nsComponentManager should be a memory-pressure listener and shrink the registry 
buffer at some time after startup.

Better yet, the component registration data could be moved into a flat file or 
two that could be read and parsed much quicker than the libreg format. the 
Component manager is not really benefitting from the libreg hierarchical 
structure.

Comment 1

[Daniel Veditz [:dveditz]]

*** Bug 101961 has been marked as a duplicate of this bug. ***

Comment 2

[Chris Waterson]

Shaver attached a patch in bug 101961 (attachment 51241 [details] [diff] [review]). Is there anything
wrong with it? If not, let's get it checked in. This is the single largest
consumer of memory in the product.

Type                    Count    Bytes %Total   %Cov
TOTAL                  134673  6864031 100.00   0.00
registry-Buffer             1   512000   7.46   7.46
xpti-unclassified         263   288976   4.21  11.67
JS-script                4158   280874   4.09  15.76
nsImageGTK                264   239236   3.49  19.25
gtk-unclassified         4402   208766   3.04  22.29
JSScopeProperty          5718   205848   3.00  25.29
nsString                 4465   201869   2.94  28.23
nsZipArchive             4049   182925   2.66  30.89
js_MatchToken            6505   174616   2.54  33.44
OTHER                  104848  4568921  66.56 100.00

Comment 3

[Daniel Veditz [:dveditz]]

attachment 51241 [details] [diff] [review] from bug 101961 deletes too much, it should simply uncomment
the two lines that shrink the buffer.

If applied we get back to startup registry churn because the i18n folks are
still grovelling for charset converters. It's a 2-3% startup hit so that needs
to get fixed first.

Comment 4

[Daniel Veditz [:dveditz]]

Attachment: patch that eliminates registry footprint problem

Shrink the big registry buffer back to default size before the end of XPCOM
init. Explicitly initialize the Category manager during that time so its
registry enumerations happen while we've already got the data in memory.

Also changed the default registry buffer size from 64K to 8K (for other
nsIRegistry consumers)

Comment 5

[Mike Shaver (:shaver emeritus)]

Comment on attachment 59777 [details] [diff] [review]
patch that eliminates registry footprint problem

>-#define BUFIO_BUFSIZE_DEFAULT   0x10000
>+#define BUFIO_BUFSIZE_DEFAULT   0x2000

Clearer as decimal?  Use PAGE_SIZE to tune it?	Just thinking aloud.

>+    // Create the category manage to ensure prepopulation of categories
>+    nsCOMPtr<nsICategoryManager> catman = do_GetService(NS_CATEGORYMANAGER_CONTRACTID);

Do we need to use the service manager for this?  Hmm, probably not
worth making a more direct call.  Never mind.

sr=shaver.

Comment 6

[Doug Turner (:dougt)]

Comment on attachment 59777 [details] [diff] [review]
patch that eliminates registry footprint problem

add a comment why this buffer has to be the size it is. with that r=dougt

Comment 7

[Doug Turner (:dougt)]

Comment on attachment 59777 [details] [diff] [review]
patch that eliminates registry footprint problem

add a comment why this buffer has to be the size it is. with that r=dougt

Comment 8

[Daniel Veditz [:dveditz]]

Made dougt's suggested comment change and checked in.

Comment 9

[Daniel Bratell]

Oh, this dropped MH (max heap size) by half a MB! Way to go!