611781 - Project to reduce the size of NSS libraries included in Firefox distributions

Status: RESOLVED WONTFIX

(Firefox Build System :: General, enhancement)

Description

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

Here is a summary of the research I did on this a few weeks ago. Some of these points may require changes in NSS.

(0) Simply building NSS libraries with LTO on Windows resulted in a ~3% size reduction on Windows, IIRC. This is NSS Bug 608040.

(1) I found that there is over 100KB of object code that seems like purely dead code as far as Firefox is concerned. I simply removed unused (by Firefox and NSS) symbols in the *.def files of the NSS libraries, rebuilt them with LTO, and observed a >100KB size decrease.

(2) Robert Relyea pointed out that the S/MIME library contains code that Firefox doesn't use. But, we expose an interface to extensions in PSM that allows them to use it. Removing those interfaces is a ~90KB size decrease, IIRC.
This is bug 611752 in PSM.

(3) Removing bypass mode and the libssl server-side code was a very small win. But, I want to revisit this because I think the way I removed them might not have made the code look "dead enough" to the optimizer.

(4) Robert also pointed out that if we switch the SQLite from DBM as the database storage format (bug ???), we can (eventually) remove the ~100KB nssdbm3 library. But, we won't be able to remove nssdbm3 because we need it for migrating existing profiles to the SQLite format. Conversely, if we create patches to make the SQLite support in Softoken optional (since we don't use SQLite), like (3) above, but I suspect the size win will be small and very likely not worth the effort.

(5) Robert and Wan-Teh pointed out that when we switch to libpkix for all certificate validation (bug 479393), we will be able to remove the dependencies on the old validation code. I have not measures how much of a size win this would be. But, AFAICT, to get the size win we will either have to modify NSS to remove the old validation code or we will have to use the new validation APIs; the proposed patch for bug 479393 on its own won't have any benefit w.r.t. to size.

(6) I did not have time to investigate this, but my understanding is that there are a few "default" implementations of the customizable aspects of NSS. I do not know how large these "default" implementations are.

(7) Linking NSS libraries (other than FreeBL and Softoken) together and/or into libxul (bug 534471) will also likely result in size reductions as LTO will be more effective.

(8) Linking FreeBL and Softoken together on platforms where this isn't problematic would probably also result in a small win similar to (7).

(9) More investigation beyond the above is warranted.

All together, I think we are looking at a ~300KB-500KB size reduction.

Comment 1

[Kai Engert [:KaiE:]]

for the record, LTO = link time optimization

Comment 2

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

Any changes in NSS would have to be configurable at build time in NSS so that system NSS libraries used on desktop/server Linux and other platforms can still provide all the functionality while allowing us to ship a subset of the functionality for Fennec (for size and startup time) and desktop Firefox (for startup time) on Windows and Mac OS X. The idea is that we would get these build-time options included into the NSS trunk and then change the Mozilla build system to pass the configuration options to the NSS build system when Firefox and/or other products are built.

Comment 3

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

From Bug 537857 Comment 0

> On Linux  -fdata-sections -ffunction-sections
> gcc flags and -Wl,--gc-sections
> results in a 5% smaller libxul.so
> 
> Windows has similar options accessible via /Gy
> compiler flag and /OPT:REF linker flags.

Comment 4

[Nathan Froyd [:froydnj]]

I looked into this a little bit for possible size reductions in B2G builds.  Essentially, I think I repeated what Brian had done in part 1 of comment 0.  Note that I did this on Linux x86-64, not Windows, so my numbers might be slightly different strictly from a different processor architecture/compiler standpoint.

- Find the set of symbols that libnss3 exports that are unused from libxul, libsmime3, and libssl3.  Since various utilities (certutil, pk12util, modutil) also use libnss3, we have to consider symbols from those too, even though they aren't strictly necessary for Firefox (?).
- Remove those symbols from nss.def;
- Remove any no-longer needed symbol versioning namespaces from nss.def;
- Compile NSS with -ffunction-sections -Wl,--gc-sections so the linker cleans up after us.

So, some numbers.  Plain opt build, x86-64 Linux.  Unmodified first:

[froydnj@cerebro build-mc]$ size dist/lib/libnss3.so 
   text	   data	    bss	    dec	    hex	filename
1056779	  26272	   5988	1089039	 109e0f	dist/lib/libnss3.so

Just building with -ffunction-sections, -Wl,--gc-sections:

[froydnj@cerebro build-mc]$ size dist/lib/libnss3.so 
   text	   data	    bss	    dec	    hex	filename
 952509	  25280	   5924	 983713	  f02a1	dist/lib/libnss3.so

That's a 100K size win just from changing compiler options.  I have no idea what got removed there!

Building with -ffunction-sections, -Wl,--gc-sections, and modifying nss.def.  Please note that this could likely be less if we restricted it to just what firefox/libssl3/libsmime3 needed.  I had to add back quite a few things so tests and other utilities would build:

[froydnj@cerebro build-mc]$ size dist/lib/libnss3.so 
   text	   data	    bss	    dec	    hex	filename
 891523	  21976	   5924	 919423	  e077f	dist/lib/libnss3.so

So another ~60K on top of our earlier changes.  That's pretty nice, though not quite as much as comment 0 suggests.  It's possible that several years of development has simply made things that would have gotten removed when the original measurements were done used by functions that we now must export.

It looks like we should be changing our compilation options for NSS on Linux.  I am less sure what "compiling with LTO" referenced in comment 0 and bug 608040 means in the context of Windows.

Doing all of this for Android doesn't seem to have any effect, possibly because there's no attempt to export symbols there (I think?).  Therefore, the linker sees everything as exported, and can't GC anything.  We should probably fix that.  I'm assuming that the same sort of thing happens and would be beneficial on B2G as well.

Modifying nss.def seems quite a bit more difficult for not quite as much gain.  Is there precedent for controlling bits of NSS according to Firefox's needs like this?  --enable-minimal-nss or something?

Comment 5

[Nathan Froyd [:froydnj]]

Attachment: pare down nss.def to the minimal set of symbols for a successful build and firefox's needs

This is the nss.def I built with on Linux for the wins in comment 4.

Comment 6

[Nicholas Nethercote [inactive]]

> - Remove any no-longer needed symbol versioning namespaces from nss.def;
> - Compile NSS with -ffunction-sections -Wl,--gc-sections so the linker
> cleans up after us.

Sounds like --gc-sections is pure win -- is there any reason not to enable it by default? Though maybe it doesn't have much effect without -ffunction-sections, and the effects of that look more mixed. (And all this isn't relevant on Windows, of course.)

> That's a 100K size win just from changing compiler options.  I have no idea
> what got removed there!

Could you find out what has been removed by doing some kind of process where you objdump libnss3.so, extract the function names, sort them, and then diff? Assuming that information is even useful...

Comment 7

[Mike Hommey [:glandium]]

Just use -Wl,--print-gc-sections

Comment 8

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

(In reply to Nathan Froyd (:froydnj) from comment #4)
> - Find the set of symbols that libnss3 exports that are unused from libxul,
> libsmime3, and libssl3.  Since various utilities (certutil, pk12util,
> modutil) also use libnss3, we have to consider symbols from those too, even
> though they aren't strictly necessary for Firefox (?).

It should be possible to change things so that we don't need to build certutil, pk12util, and modutil. In particular, it seems unlikely to me that we should need those utilities to be built *for* B2G; we may need them around in the *host*, but not the *target*, but even then I bet we can get rid of the need for them completely. if you build NSS with all the functionality that those utilities require, then you're going to be building an NSS that has significantly more stuff than necessary.

> - Remove those symbols from nss.def;
> - Remove any no-longer needed symbol versioning namespaces from nss.def;
> - Compile NSS with -ffunction-sections -Wl,--gc-sections so the linker
> cleans up after us.

IIRC, you also need to build with LTO to get the biggest effects.

> So another ~60K on top of our earlier changes.  That's pretty nice, though
> not quite as much as comment 0 suggests.  It's possible that several years
> of development has simply made things that would have gotten removed when
> the original measurements were done used by functions that we now must
> export.

Your results are consistent with item #1 in comment 0.

When I did my last round of experiments with this, I did a build without libpkix (i.e. without exporting CERT_PKIXVerifyCert and with the code in certvfy.c that calls that function #ifdef'd out.) IIRC, CERT_PKIXVerifyCert should save a couple of hundred KB just on its own.

> It looks like we should be changing our compilation options for NSS on
> Linux.  I am less sure what "compiling with LTO" referenced in comment 0 and
> bug 608040 means in the context of Windows.

/GL /LTCG http://msdn.microsoft.com/en-us/library/xbf3tbeh.aspx

> Doing all of this for Android doesn't seem to have any effect, possibly
> because there's no attempt to export symbols there (I think?).  Therefore,
> the linker sees everything as exported, and can't GC anything.  We should
> probably fix that.  I'm assuming that the same sort of thing happens and
> would be beneficial on B2G as well.

And Mac OS (everything is exported from NSS, accidentally).

> Modifying nss.def seems quite a bit more difficult for not quite as much
> gain.  Is there precedent for controlling bits of NSS according to Firefox's
> needs like this?  --enable-minimal-nss or something?

No, there is nothing like that now. Ultimately the biggest gains are going to come from doing something like that, especially in conjunction with the other bugs that this bug depends on (see updated dependency list). Removing CERT_Verify* (which requires fixing bug 975229 and maybe finding and removing other uses, such as moving nsICMS* to comm-central) should be a significant savings.

If I were to do this, I would first write moz.build files for building all of security/nss/lib except security/nss/lib/softoken and security/nss/lib/freebl, using link-time optimization (I wouldn't use PGO, since NSS isn't exercised by profiling runs). Then I would modify those moz.build files to use a custom, minimal *.def file. I would probably also keep the existing traditional Make-based build of NSS so that the utilities (certutil and friends) can link against *that* NSS, while Gecko links against the minimized version. Then I would fix the bugs in the dependency list.

I am happy to review patches for removing nsICMS* from Gecko and for removing the old NSS-based validation (bug 975229).

Comment 9

[Nathan Froyd [:froydnj]]

(In reply to Nicholas Nethercote [:njn] from comment #6)
> > - Remove any no-longer needed symbol versioning namespaces from nss.def;
> > - Compile NSS with -ffunction-sections -Wl,--gc-sections so the linker
> > cleans up after us.
> 
> Sounds like --gc-sections is pure win -- is there any reason not to enable
> it by default? Though maybe it doesn't have much effect without
> -ffunction-sections, and the effects of that look more mixed. (And all this
> isn't relevant on Windows, of course.)

Yeah, you really need -ffunction-sections for any sort of win from --gc-sections, otherwise all the functions from a file get shoved into .text and you can't separate any of them out.

> > That's a 100K size win just from changing compiler options.  I have no idea
> > what got removed there!
> 
> Could you find out what has been removed by doing some kind of process where
> you objdump libnss3.so, extract the function names, sort them, and then
> diff? Assuming that information is even useful...

Yeah, that'd be the way to go.

Comment 10

[Nicholas Nethercote [inactive]]

> Yeah, that'd be the way to go.

But -Wl,--print-gc-sections sounds a lot easier, as Mike suggested.

Comment 11

[Nathan Froyd [:froydnj]]

Attachment: deleted sections according to -Wl,--print-gc-sections

(In reply to Nicholas Nethercote [:njn] from comment #10)
> > Yeah, that'd be the way to go.
> 
> But -Wl,--print-gc-sections sounds a lot easier, as Mike suggested.

Here's the output from that.  Looks like a lot of unused certificate stuff, as well as some out-and-out debugging stuff (e.g. the MyFuncName bits).

This stuff only adds up to ~85K, not the 100K change mentioned in comment 4.  But they're the same objects, so I'm still not sure what happened to the other 15K.  Perhaps we're just getting very lucky in how the padding segments to page alignment works out in this case.

Comment 13

[Richard Newman [:rnewman]]

ekr, Martin: in Bug 856791, briansmith suggested:

> If you think a ~500KB size win is important, then
> I suggest you ask Eric Rescorla or Martin Thompson
> to help instead.

We certainly think a 500KB size win is important; Fennec's ever-expanding waistline is a huge barrier to our adoption on resource-constrained Android devices, and libxul and libnss together occupy 15MB.

Are either of you able to pick up some of this work, or point to someone who can?

Comment 14

[Eric Rescorla (:ekr)]

I doubt we will be able to. I would suggest asking Richard Barnes, whose team works on this stuff.

Comment 15

[Richard Barnes [:rbarnes]]

How urgent is this?  We might be able to pick it up, but possibly not until Q4.

Comment 16

[Richard Newman [:rnewman]]

(In reply to Richard Barnes [:rbarnes] from comment #15)
> How urgent is this?  We might be able to pick it up, but possibly not until
> Q4.

Low urgency -- we already ship the current build output -- but pretty high value. The smaller NSS (and libxul, the main offender) gets, the happier we'll all be.

I think the current state of affairs is that libnss3.so for ARMv7 is 795KB.

Comment 17

[Kim Moir [:kmoir] ET]

:briansmith is this still an active project, if not I'm going to close.  (Triaging this bucket)

Comment 18

[Kim Moir [:kmoir] ET]

closing, please reopen if this is still an issue