191970 - necko configuration options for footprint reduction

Status: RESOLVED FIXED

(Core :: Networking, defect, P2)

Description

[Darin Fisher]

necko should have options to disable: disk cache, dns cache, and various
protocol handlers.  there should also be configuration options (or prefs?) to
adjust neckos default buffer segment size, buffer segment count, and buffer
recycle count.

Comment 1

[Darin Fisher]

Attachment: v1 patch

this patch adds the following configure options:

  --disable-disk-cache
  --disable-dns-cache
  --enable-protocols=... (works like --enable-image-decoders)

it also changes necko's hard coded buffer sizes:

  buffer segment size  = 2000
  buffer segment count = 4
  buffer recycle count = 10

i have no idea if these numbers make sense, but they should reduce our heap
requirements by 76KB.. whatever that is worth.

Comment 2

[Darin Fisher]

by 76KB heap reduction, i really meant a reduction in our steady-state, idle
heap usage.  the big reduction is probably during page load.  instead of
allocating up to 64K per transaction, we'll now only allocate up to 8K per
transaction.  this number will of course need to be tuned for the embedding
application.

Comment 3

[Darin Fisher]

this patch has been checked in on the branch.  some form of this patch will
hopefully make its way into the 1.4 trunk.

Comment 4

[Alec Flett]

great stuff!

I'm beginning to wonder though if we should think about something beyond
configure.in, since we're going to be adding more and more options like which
modules to build, and so forth. (obviously the subject of another bug)

adding seawood for a configure.in review, once we go to the trunk

Comment 5

[hacker formerly known as [e-mail]]

Comment on attachment 113569 [details] [diff] [review]
v1 patch

The disk cache & dns cache tests should look like:
MOZ_DISK_CACHE=1
MOZ_ARG_DISABLE_BOOL(disk-cache,
[  --disable-disk-cache    Disable disk cache],
    MOZ_DISK_CACHE=,
    MOZ_DISK_CACHE=1)

so that we do the correct thing when someone specifies --disable-disk-cache
--enable-disk-cache on the same configure line. (It happens.)

--enable-protocols should be --enable-necko-protocols.

Why do we bother having about listed in the option list if it's a mandatory
protocol?

Does the cache directory get rebuilt properly if you -enable/disable disk cache
between builds?  I don't think it will.  The dependencies for
nsCacheService.cpp, which is built always, will not be regenerated when you
flip the switch so the file will not be rebuilt even though the MOZ_DISK_CACHE
define is added or removed.  You'll have to add a hack similar to the export
rule used for the protocol list to work around that problem and force the files
to be rebuilt.

It seems like a waste to have to traverse each of the protocol subdir just to
export the handful of idl files used but I guess we can wait until bug 167254
lands to worry about that.

Comment 6

[Darin Fisher]

cls: thanks for the tips.

about:blank is mandatory, but about:foo is not.  so my patch just makes it so
that disabling about: keeps about:blank but eliminates all the others.  not sure
that is so clean, but :-/

Comment 7

[Darin Fisher]

ok, i talked to cls about adding a netwerk specific header.h.in file that will
be autogenerated like mozilla-config.h.  this will be a lot nicer than all of
the Makefile hackery in this patch already.

Comment 8

[Darin Fisher]

*** Bug 60725 has been marked as a duplicate of this bug. ***

Comment 9

[Darin Fisher]

Attachment: v2 patch

ok, this patch is better.  i haven't put it through all the paces yet, but it's
complete enough such that i'd really like to get feedback on the new way of
handling necko specific configuration options.

i bagged the idea of a .h.in file since it was getting awkward, but maybe i
overlooked something that would have simplified it.  anyhow, i ended up just
writing a shell script to generate necko-config.h.  also, instead of trying to
get this included on the command line, i just #include it where needed.

i also realized that i was suppressing some things that didn't really need to
be suppressed.	for example, if FTP is disabled, i only really need to not link
the FTP libs... i don't necessarily need to not build the FTP libs.  being lax
in this regard helps minimize the number of changes, which is a good thing i
think.

i decided not to include the "disable dns cache" option since there is little
code associated with it and moreover the size is configurable from prefs.

finally, i threw in a new option "--enable-necko-small-buffers" ... this just
configures necko to minimize the size of its i/o buffers.  i thought about
making this option take an argument, but this just seems sufficient to me (for
now).

Comment 10

[hacker formerly known as [e-mail]]

Comment on attachment 120808 [details] [diff] [review]
v2 patch

+  _NECKO_DEFS_H=necko-config.h
+  ( cd netwerk &&
+    cat << EOF > $_NECKO_TMP

That's going to break in objdir builds.  The directories do not get created
until the Makefiles are generated at the end of the configure run.

You shouldn't be generating the header by hand anyway. (Neither shoud
mozilla-config.h.)   You should use the AC_CONFIG_HEADER to generate the header
from an existing template at netwerk/.../necko-config.h.in .  Including the
header is definitely the way to go.  We should have made mozilla-config.h a
required include a long time ago.

Comment 11

[Darin Fisher]

Attachment: v2.1 patch

revised per comments from seawood.  uses new _NON_GLOBAL_ACDEFINES configure.in
variable.

Comment 12

[hacker formerly known as [e-mail]]

Comment on attachment 121262 [details] [diff] [review]
v2.1 patch

Use AC_DEFINE_UNQUOTED(NECKO_PROTOCOL_$p) instead of manually adding those
defines to confdefs.h.	necko-config.h.in should have a license header. :-/

Comment 13

[Darin Fisher]

Attachment: v2.2 patch

Comment 14

[hacker formerly known as [e-mail]]

Comment on attachment 121295 [details] [diff] [review]
v2.2 patch

r=cls

Comment 15

[Darin Fisher]

ok, the patch is in the tree.  marking FIXED.

Comment 16

[hacker formerly known as [e-mail]]

Darin, I just noticed in my tree that the build breaks if no protocols are
specified.  This probably isn't a supported configuration anyway, in which case
the configure test should bail.  MSVC complains about the empty structure in
netwerk/build2/nsNetModule2.cpp .

Comment 17

[Darin Fisher]

hmm... build2/Makefile.in has a check to not do anything if none of the
protocols for that library are enabled.  it is working in my linux tree, and it
is a supported configuration (note: the core protocols are in build/ and very
little will work without them).

Comment 18

[Jim Dunn]

Attachment: Patch to fix AIX/HP-UX

The previous patch broke AIX tinderbox... (and hp-ux builds)
there is confusion as to which "log" protype to use.

Comment 19

[Jim Dunn]

Comment on attachment 121425 [details] [diff] [review]
Patch to fix AIX/HP-UX

UGH- wrong bug, ignore (sorry)

Comment 20

[Jim Dunn]

Comment on attachment 121425 [details] [diff] [review]
Patch to fix AIX/HP-UX

UGH- wrong bug, ignore (sorry)