7748 - DLL bloat in M6

Status: VERIFIED FIXED

(Core :: Internationalization, defect, P3)

Description

[bobj]

Somehow, most of the I18N modules (even those
that had little or no changes), bloated from M5
to M6.

 lwbrk nslocale strres  uconv   ucvcn   ucvja  ucvja2  ucvlatin unicharutil
  9488    25312   6800  23808  243120   60464  111360     46496    9360
 25344    42192  21120  35104  255360   70656  124112     58336   21328

Something happened between 5/11 and 5/12.  Possibly XPCom related changes?

1999-05-11-09-M6/size_matters.txt

     -rw-r--r--   1 544      everyone     9488 May 11 09:48 lwbrk.dll

1999-05-12-16-M6/size_matters.txt

     -rw-r--r--   1 544      everyone    25344 May 12 17:55 lwbrk.dll

Comment 1

[Suresh Duddi (gone)]

No big XPCOM changes went in then.

I tried looking for what changed during that time frame but didn't find anything
standing out.

I will let frank figure it out.

Comment 2

[bobj]

ChrisY and ChrisH,
We made little or no changes to most of these modules.  Can we get some
build/release help to help figure out what happened?

Has anyone else's modules been affected like this?

Comment 3

[Frank Tang]

I have talk to namachi, he believe this is caused by the change waterson check
in rev 3.17 of mozilla/config/WIN32 for "Clean up MOZ_PROFILE rules to get
working with Quantify." However, that check in is in 6/3, not 5/11.

Reassign this bug to namachi

Comment 4

[chris hofmann]

Frank,

Shiva works on talkback.  Unless we believe that this is related
to talkback in any way this bug should not be assigned to him.

The following CVS query can provide a listing of all the changes
that were made around the time period of the increase.

http://cvs-mirror.mozilla.org/webtools/bonsai/cvsquery.cgi?module=SeaMonkeyAll&b
ranch=HEAD&branchtype=match&dir=&file=&filetype=match&who=&whotype=match&sortby=
Date&hours=2&date=explicit&mindate=05%2F10%2F99+00%3A00%3A00&maxdate=05%2F12%2F9
9+00%3A00%3A00&cvsroot=%2Fcvsroot

I see you made a number of changes in related areas
during that time Have we confirmed the following are not related to
the increase?

add MappingCache mozilla/intl/uconv/src/ and public...
add nsMappingCache.cpp
add HankakuToZenkaku
add nsIMappingCache.h to export list
add progid to a number of files mozilla/ intl/ uconv/ public/
add progid. Add nsHankakuToZenkaku.h to the export list

Can you do a detailed review of the cvs query to
identify any other likely sources?

Comment 5

[Chris Waterson]

The theory was that one of the compiler switches caused the increase in size,
IIRC. Specifically, for MOZ_PROFILE, I changed the compiler to use
/PDB:$(PDBFILE) instead of /PDB:NONE. This creates a program database (debugger
info) for each DLL in the optimized build, allowing you to actually play with
the build in VC++. Since the PDB is a separate file, it seems strange that it
would increase the size of the release DLLs, but, I've seen stranger things
happen.

Could somebody with some spare machine cycles play around with this flag to
test the theory?

http://lxr.mozilla.org/seamonkey/source/config/WIN32#72

Comment 6

[bobj]

Chris,
It's unlikely to be due to any source change because the bloat only happened
on Windows.  It seems more likely that something changed in the way it is built
or linked.

Comment 7

[Frank Tang]

chofmann: None of the change you mention build into lwbrk.dll except I add
progid which should be several bytes, not several thousand bytes. We are
sure the reason it blow is out side our source directory (e.g.
mozilla/intl/ ) but we are not sure where. That dll only contains the code
inside the mozilla/intl/lwbrk directory. Of course, it also link against xpcom.
Add namachi to the CC list.

Comment 8

[Chris Waterson]

Okay, you guys were right. Bloat came from using /PDB:$(PDBFILE) in
MOZ_PROFILE. Turns out that a DLL that is built with the PDB on is slightly
larger than a splitsym-ed DLL.

So I think the right thing to do is change back to /PDB:NONE in WIN32.

Comment 9

[Chris Waterson]

I should add that the difference is really not all that significant (like, 1%
maybe). I'm attaching my results from today's build (1999-06-11).
pdb-none-splitsym.txt lists the DLLs that were built with /PDB:NONE and then
splitsym-ed. pdb-pdbfiles.txt lists the DLLs that were build with
/PDB:$(PDBFILE).

Comment 10

[Chris Waterson]

Attachment: DLL sizes for /PDB:NONE + splitsym

Comment 11

[Chris Waterson]

Attachment: DLL sizes with /PDB:$(PDBFILE)

Comment 12

[Shiva Thirumazhusai]

Leaf is working on the change. Tomorrow's build will not generate pdb files
for Release build. Instead it will generate debug info in .dll's and splitsym
will seperate the release .dll and generate debug info as .dbg files. If for
some reason we need .pdb files. We should create different tag than MOZ_PROFILE.

Comment 13

[Frank Tang]

We find out there are very small changes to the PROFILE and pdb. This must not
the real problem.

I download the May11 build (lwbrk.dll is 9K) and May12 build (lwbrk.dll is 25K).
There are no source difference between that two build for this Dll. And then I
do a dumpbin /headers to this two build and the result is the following

May-11
Microsoft (R) COFF Binary File Dumper Version 5.00.7022
Copyright (C) Microsoft Corp 1992-1997. All rights reserved.


Dump of file lwbrk.dll

PE signature found

File Type: DLL

FILE HEADER VALUES
     14C machine (i386)
       4 number of sections
3737E842 time date stamp Tue May 11 01:20:18 1999
       0 file pointer to symbol table
       0 number of symbols
      E0 size of optional header
    230A characteristics
            Executable
            Symbols stripped
            32 bit word machine
            Debug information stripped
            DLL

OPTIONAL HEADER VALUES
     10B magic #
    5.10 linker version
    1200 size of code
     C00 size of initialized data
       0 size of uninitialized data
    2010 address of entry point
    1000 base of code
    3000 base of data
         ----- new -----
10000000 image base
    1000 section alignment
     200 file alignment
       2 subsystem (Windows GUI)
    4.00 operating system version
    0.00 image version
    4.00 subsystem version
    6000 size of image
     400 size of headers
    D4DC checksum
  100000 size of stack reserve
    1000 size of stack commit
  100000 size of heap reserve
    1000 size of heap commit
    3430 [      93] address [size] of Export Directory
    3208 [      78] address [size] of Import Directory
       0 [       0] address [size] of Resource Directory
       0 [       0] address [size] of Exception Directory
       0 [       0] address [size] of Security Directory
    5000 [     168] address [size] of Base Relocation Directory
    3050 [      38] address [size] of Debug Directory
       0 [       0] address [size] of Description Directory
       0 [       0] address [size] of Special Directory
       0 [       0] address [size] of Thread Storage Directory
       0 [       0] address [size] of Load Configuration Directory
       0 [       0] address [size] of Bound Import Directory
    3000 [      48] address [size] of Import Address Table Directory
       0 [       0] address [size] of Reserved Directory
       0 [       0] address [size] of Reserved Directory
       0 [       0] address [size] of Reserved Directory


SECTION HEADER #1
   .text name
    1100 virtual size
    1000 virtual address
    1200 size of raw data
     400 file pointer to raw data
       0 file pointer to relocation table
       0 file pointer to line numbers
       0 number of relocations
       0 number of line numbers
60000020 flags
         Code
         (no align specified)
         Execute Read

SECTION HEADER #2
  .rdata name
     4C3 virtual size
    3000 virtual address
     600 size of raw data
    1600 file pointer to raw data
       0 file pointer to relocation table
       0 file pointer to line numbers
       0 number of relocations
       0 number of line numbers
40000040 flags
         Initialized Data
         (no align specified)
         Read Only


Debug Directories(2)
        Type       Size     Address  Pointer

        misc        110    00000000     2200    Image Name: lwbrk.dbg
        fpo         200    00000000     2310

SECTION HEADER #3
   .data name
     310 virtual size
    4000 virtual address
     400 size of raw data
    1C00 file pointer to raw data
       0 file pointer to relocation table
       0 file pointer to line numbers
       0 number of relocations
       0 number of line numbers
C0000040 flags
         Initialized Data
         (no align specified)
         Read Write

SECTION HEADER #4
  .reloc name
     18A virtual size
    5000 virtual address
     200 size of raw data
    2000 file pointer to raw data
       0 file pointer to relocation table
       0 file pointer to line numbers
       0 number of relocations
       0 number of line numbers
42000040 flags
         Initialized Data
         Discardable
         (no align specified)
         Read Only

     Summary

        1000 .data
        1000 .rdata
        1000 .reloc
        2000 .text

May 12

Microsoft (R) COFF Binary File Dumper Version 5.00.7022
Copyright (C) Microsoft Corp 1992-1997. All rights reserved.


Dump of file lwbrk.dll

PE signature found

File Type: DLL

FILE HEADER VALUES
     14C machine (i386)
       4 number of sections
3739E5F1 time date stamp Wed May 12 13:34:57 1999
       0 file pointer to symbol table
       0 number of symbols
      E0 size of optional header
    230E characteristics
            Executable
            Line numbers stripped
            Symbols stripped
            32 bit word machine
            Debug information stripped
            DLL

OPTIONAL HEADER VALUES
     10B magic #
    6.00 linker version
    2000 size of code
    3000 size of initialized data
       0 size of uninitialized data
    1FC5 address of entry point
    1000 base of code
    3000 base of data
         ----- new -----
10000000 image base
    1000 section alignment
    1000 file alignment
       2 subsystem (Windows GUI)
    4.00 operating system version
    0.00 image version
    4.00 subsystem version
    6000 size of image
    1000 size of headers
   1017B checksum
  100000 size of stack reserve
    1000 size of stack commit
  100000 size of heap reserve
    1000 size of heap commit
    3430 [      93] address [size] of Export Directory
    3208 [      78] address [size] of Import Directory
       0 [       0] address [size] of Resource Directory
       0 [       0] address [size] of Exception Directory
       0 [       0] address [size] of Security Directory
    5000 [     168] address [size] of Base Relocation Directory
    3050 [      38] address [size] of Debug Directory
       0 [       0] address [size] of Description Directory
       0 [       0] address [size] of Special Directory
       0 [       0] address [size] of Thread Storage Directory
       0 [       0] address [size] of Load Configuration Directory
       0 [       0] address [size] of Bound Import Directory
    3000 [      48] address [size] of Import Address Table Directory
       0 [       0] address [size] of Reserved Directory
       0 [       0] address [size] of Reserved Directory
       0 [       0] address [size] of Reserved Directory


SECTION HEADER #1
   .text name
    1088 virtual size
    1000 virtual address
    2000 size of raw data
    1000 file pointer to raw data
       0 file pointer to relocation table
       0 file pointer to line numbers
       0 number of relocations
       0 number of line numbers
60000020 flags
         Code
         (no align specified)
         Execute Read

SECTION HEADER #2
  .rdata name
     4C3 virtual size
    3000 virtual address
    1000 size of raw data
    3000 file pointer to raw data
       0 file pointer to relocation table
       0 file pointer to line numbers
       0 number of relocations
       0 number of line numbers
40000040 flags
         Initialized Data
         (no align specified)
         Read Only


Debug Directories(2)
        Type       Size     Address  Pointer

        misc        110    00000000     6000    Image Name: lwbrk.dbg
        fpo         1F0    00000000     6110

SECTION HEADER #3
   .data name
     310 virtual size
    4000 virtual address
    1000 size of raw data
    4000 file pointer to raw data
       0 file pointer to relocation table
       0 file pointer to line numbers
       0 number of relocations
       0 number of line numbers
C0000040 flags
         Initialized Data
         (no align specified)
         Read Write

SECTION HEADER #4
  .reloc name
     188 virtual size
    5000 virtual address
    1000 size of raw data
    5000 file pointer to raw data
       0 file pointer to relocation table
       0 file pointer to line numbers
       0 number of relocations
       0 number of line numbers
42000040 flags
         Initialized Data
         Discardable
         (no align specified)
         Read Only

     Summary

        1000 .data
        1000 .rdata
        1000 .reloc
        2000 .text

Notice that the linker version are different. And a lot of size are different
there.

I then check the size of of today's build in our group. Tague and my local debug
build is ~30K (Both of us use VC++ 5.0) and Naoki's debug build are ~45K (He use
VC++ 6.0)

I believe the reason it bloat is the change of build tool not source or
configuration change.

Mark it "WONTFIX"

Comment 14

[Frank Tang]

bobj thinks we should reopen this because we should fix it. Reopen this and
reassign this to chofmann. chofmann, you need to find someone to investigate the
linker and make decision which version we should use.

BTW, both version of the lwbrk.dll waterson built are small (~11K). Waterson,
can you do a dumpbin /headers lwbrk.dll and tell us the Linker Version ? Is that
5.10 ?

Comment 15

[Chris Waterson]

Frank: I'm using VC5.

Comment 16

[bobj]

ChrisH,
   Are you having someone look into the possible linker issue?  The
smoking gun seems to be pointing to the switch to the 6.0 linker.
I want to look into linking the I18N libs with the xpcom DLL instead
of xpcom.lib, but 5.0 needs to solve 7748 first (instead of hiding
this problem by changing something else.)

Comment 17

[Scott MacGregor]

Just to add a few more notes to this report. I was investigating some DLL bloat
that also occurred between 5/11 and 5/12 on our mailnews DLLs. We had three dlls
(rawemitter.dll, htmlemitter.dll, xmlemitter.dll) that were all 8K on 5/11. On
5/12 they jumped to over 20K. I was never able to figure out why as there were
no code changes in these libraries. It was probably the change in linker that
caused the I18N dll bloat as well.

Comment 18

[chris hofmann]

over to jevevering to see if he can find someone to help on figuring
out if a compiler/linker swicth might help us to recover some footprint.

Comment 19

[bobj]

Is anything going to happen with this bug?

Comment 20

[Dawn Endico]

reassigning jevering's old bugs to jar for adoption or to
find them a new home

Comment 21

[Jim Roskind]

Since bobj (long ago) asked for a re-open, I'm passing it to him.  My guess is
that he'll say this is not an issue, and close it... but he's welcome to push it
to a better person than me.

Thanks,

Jim

Comment 22

[bobj]

Bouncing it back to jar.

I logged this bug because I thought it was significant for downloading size.
If we are adding 10K to every DLL because of some linker flag, that adds
up.  Just for the I18N DLLs mentioned at the beginning of this bug report,
the combined bloat is over 100KB.  And from mscott's comment, this problem
was not restricted to the I18N DLLs.  If size matters, someone should look
into this again.  Maybe since this is a very old bug, the problem has gone
away????

Comment 23

[Jim Roskind]

Chris,
As you can tell by the recent commentary, this bug was assigned to me, an I'm 
trying to find a better home for it. Can you make a suggestion, and pass it 
along, or perhaps argue that this is no longer a problem, and mark it resolved?

Thanks,

Jim

Comment 24

[bobj]

cc'd valeski.  I have no idea of the status of this bug.

At the time I filed it, we suddenly bloated over 100K for a handful of i18n
components and probably had similar bloat in other components (e.g., mail).

The last theory was something related to the linker...

Comment 25

[Judson Valeski]

linking w/ "/opt:nowin98" removes some vc6 linker cruft from shared libs. here's
the savings in i18n:

as we build today:  16 File(s)      1,725,529 bytes
w/ the nowin98 linker option: 16 File(s) 867,328 bytes

as you can see the savings is 858,201 bytes (release) which is a huge win.

Comment 26

[Judson Valeski]

this is fixed as a result of 54199 being fixed (trunk).

Comment 27

[Phil Peterson]

Should we consider this for Seamonkey RTM?

Comment 28

[Judson Valeski]

IMO, yes.

Comment 29

[bobj]

Definitely for RTM.  See comment in bug 54199
   ------- Additional Comments From Judson Valeski 2000-09-27 07:54 -------

   all the info has already been posted to mozilla-seamonkey and
   mozilla-porkjockeys (last week). reposting here.

  ... I built win32
  using the "/opt:nowin98" linker option and got some interesting results.

  release build using existing linker flags for win32 produced shared libraries
  who's total size is 9,326,592 bytes.

  release build using existing linker flags plus the nowin98 linker flag  
  produced shared libraries who's total size is 8,482,816 bytes.

  Under current conditions, that's a disk space savings of 10%.

Comment 30

[Teruko Kobayashi]

Verified.

Comment 31

[Teruko Kobayashi]

I should not mark verify this since I cannot verify.  I changed QA contact to
the engineer.

Comment 32

[Chris Waterson]

thbbbt!

Comment 33

[Chris Waterson]

unh