130372 - Win32 executables should be 4KB aligned (reduces load time on Win9x at the cost of code size increase)

Status: RESOLVED INVALID

(Firefox Build System :: General, enhancement)

Description

[Rastignac]

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.9) Gecko/20020311
BuildID:    2002031104

Intel and Microsoft prefer 4KB aligned executables. But Mozilla's EXE and DLLs
are not 4KB aligned ! Aligned executables are better: they load faster, they
better fit into memory / swap, etc... Defragmentation tools (MS or Norton) can
also greatly optimize aligned executables.

I made a few tests with my brand new Mozilla 0.9.9 executables. I "manually"
aligned the EXE and the DLLs (using MS's standard WinAlign). And the results are
good ! On my old and slow machine, the gain can really be seen. A few percents
gain, yes.

So, Mozilla's releases should be aligned. They will perform better. People will
feel it.

It's easy to do: just add a "command line switch" to your compiler and it will
align the executables at build's time. Or just align them after the compilation
(the way I did). It's easy and fast, and gives a few more percents of
performance to Mozilla !

Thanks

Reproducible: Always
Steps to Reproduce:
1. Default executables are not aligned.
2. After alignment, executables perform better.

Comment 1

[Warner Young]

Not necessarily a bad idea, but just remember that changing the File Alignment
from its current 512-byte to a 4K boundary will introduce noticeable file bloat.

Comment 2

[Bernard Alleysson]

this is Build Config

what is the "command line switch" that you use ?
what program do you use to align after compilation ?

Comment 3

[Rastignac]

About the file bloat: the file bloat is very small indeed. It's not a problem.

Here are my figures:
MOZILLA.EXE went from 104 928 bytes to 110 592, XPCOM.DLL from 510 272 to 520
192, GKLAYOUT.DLL from 1 173 744 to 1 183 744, GKCONTENT.DLL from 1 462 272 to 1
474 560, etc... A very little change !

The difference is only a few more KB, even with the biggest files.

Also, a few files won't be aligned at all (because they are already aligned
(?)). So, the total change will be quite negligeable.

Note that when compressed, the size is nearly the same between unaligned and
aligned.

Comment 4

[Olivier Cahagne]

-> Build Config

Comment 5

[Rastignac]

> what is the "command line switch" that you use ?

I don't know. I'm just a "user", not a expert programmer... (sorry). But such
switch does exist. (At least there is one in Intel or Microsoft compiler).


> what program do you use to align after compilation ?

I'm using Microsoft's Walign / WinAlign. There are two versions of this program.
One is free and included into Win98 and/or Office97. The other one is included
in MS's Resource Kit (so you must pay to get it). See
http://support.microsoft.com/default.aspx?scid=kb;EN-US;q191655

I'm using the free one (of course); there's a tip to use it onto every EXE or
DLL (a little ".bat" can do the work). It's as good / fast / powerfull as the
paid one (same result but without the cost !).

Here's my "aligne.bat":
---begin---
 @echo off
 xcopy %1 *.??$
 move %1 c:\windows\system
 echo %1 >>c:\windows\system\winali.ini
 start /w c:\windows\system\walign.exe
 move c:\windows\system\%1 .
 edit c:\windows\system\wina*.txt
---end---


Just run "aligne mozilla.exe" and it will align the program. So, run this
command line for every executable. (A simple batch can be made to perform that
easily).

What does this ".bat" do ?
- It backups the executable as "*.??$"
- It copies it into "c:\windows\system" (must be changed if "c:\winnt"). Only
programs in "system" can be optimized. So we must move our program there.
- It adds the program's name into "winali.ini". Only programs contained in this
list are optimized by Walign, so we must add our programs' names.
- It runs walign (and waits for its end).
- It moves the brand new aligned/modifed executable to replace the current one.
- The last line ("edit") is just here to show the report log (to verify success
/ failure) and is not really needed.

PS: I've seen many so-called-enhanced ".bat" file doing the same thing. Just
search the Net. They are just a little more complex / clean, but they perform
the same thing.


Thanks.

Comment 6

[Jesse Ruderman]

Confirming (not a duplicate).

Comment 7

[hacker formerly known as [e-mail]]

Over to leaf.

Comment 8

[Bernard Alleysson]

this is /OPT:NOWIN98 linker option, which is in config\WIN32

see url:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore/html/_core_.2f.OPT.asp

it's all about win98 *or* winNT (NT4, 2000, XP)

Comment 9

[Jon Granrose]

1.2 unless cathleen nominates.

Comment 10

[Sebastian Späth]

Not sure why you were able to achieve some gain. The 4kb alignment is on by
default on the MSVC++ 6 compilers (it was not on version 5). It is not on by
default (see below for the relevant MSDN part) if    
* /ALIGN is used.
* /MACHINE does not target x86.
* /SUBSYSTEM specifies something other than WINDOWS or CONSOLE.

------------
"WIN98 | NOWIN98
    WIN98 and NOWIN98 control the section alignment in the final image. For
Windows 98 applications, it is optimal to align sections on a 4K boundary to
improve load time (allows Windows 98 memory manager to cache executable images
with a minimum of wasted space). This is on by default in the linker, so you
need to specify /OPT:NOWIN98 to get a trimmed-down (but slower on Windows 98)
version of the application.

    WIN98 is on by default. WIN98 is not on when:
        * /ALIGN is used.
        * /MACHINE does not target x86.
        * /SUBSYSTEM specifies something other than WINDOWS or CONSOLE."
---------------

Comment 11

[Rastignac]

I've just used "Windows' Quick view" ("aperçu rapide" in french) onto old and
new executables (right click on the file and choose "quick view") to see
technical infos about the executables.

I've looked "MOZILLA.EXE" first. The notable differences I've seen:
- OLD: "alignement des fichiers = 00000200" (files alignment) (look the second
page of infos).
- NEW: "00001000".

So, this alignment really went from 512 bytes to 4096 bytes. So there has been a
optimisation. The executables were not perfectly aligned before I performed.

I've also looked a few ".DLL" I've also optimized. Same thing applies for them
(visible changes from 0200 to 1000).

Comment 12

[Christian :Biesinger (don't email me, ping me on IRC)]

>Not sure why you were able to achieve some gain. The 4kb alignment is on by
>default on the MSVC++ 6 compilers 

that's because Mozilla explicitly specifies to disable the 4kb alignment (using
/OPT:NOWIN98)

see http://lxr.mozilla.org/seamonkey/search?string=NOWIN98

Comment 13

[Rastignac]

> that's because Mozilla explicitly specifies to disable the 4kb
> alignment (using /OPT:NOWIN98)

Is there a good reason why the "4KB alignment" is disabled by purpose ? I can't
see any reason to do so. (The only drawback: the files's size is a few KB bigger
with than without alignment).

Conclusion:
- It's very easy to put back the optimization (just change the /OPT in the
makefiles)
- There's no valid reason to remove the optimization
- The optimzation add speed/performance to Mozilla.

===> Mozilla should really be 4KB aligned !

(I made a few serious tests on my slow computer: before, Mozilla loaded in 15
seconds; after 4KB alignment, it loads now in 12 seconds).

Comment 14

[Markus Hübner]

How much performance gain can be expected?

Comment 15

[Rastignac]

> How much performance gain can be expected?

On my old and slow computer (200mhz), Mozilla loads now in 12 seconds with 4KB
aligned executables. It was 15 seconds before 4KB alignment. Yes, there's a real
gain, and you can see and feel it.
On my new and fast computer (XP2900+), Mozilla is faster. The gain is very
little, because Mozilla is already fast ! But there's also a gain. Mozilla is
now faster than fast !
4KB alignment is a MUST !    ;)

Comment 16

[Olivier Cahagne]

*** Bug 196476 has been marked as a duplicate of this bug. ***

Comment 17

[Eric Towers]

I believe that when completely aligned, both the Section Alignment and the File
Alignment will be 0x1000.  Mozilla 1.0.1 is section aligned at 0x1000 but file
aligned at 0x0200.  (No data on 1.0.2; it was so buggy for my usage model I had
to regress.)

Comment 18

[Rastignac]

Why programmers first choosed to build Mozilla without 4KB alignment ?

Every single application nowadays is 4KB aligned. I don't see why Mozilla didn't
try this optimisation. There's no risk at all to make Mozilla 4KB aligned. The
only drawback is a few more KB in the executables size (very few).

Windows does act much better with 4KB aligned programs. They are loaded faster,
they are swapped faster, etc. Mozilla really reacts much better and smoother
when it's 4KB aligned.

I think this optimisation should really be tried. It can give a good boost to
Mozilla's performance; users can feel the gain. And it's easy to do (just change
one compilation parameter, from NOWIN98 to WIN98), and there's no risk.

The target milestone is now very far away (1.7a)...

Comment 19

[Olivier Cahagne]

We'd need to build a special Win98 build (not sure how mozilla.org can support
this additional build) because the documentation mentions that Win2k & NT (to a
greater extent, XP) NOWIN98 shouldn't use /opt:nowin98:

"If you are building components that run only on Windows NT or Windows 2000, you
should use /OPT:NOWIN98."
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore/html/_core_.2f.OPT.asp

So, if the perf win mentioned in comment 15 applies for Win98 only, then this
bug should be morphed into a Win98 build request.
Do we know if there's perf drop on Win2k/NT with this particular optimization flag ?

btw, Win98 won't be supported by MS next year:
http://www.microsoft.com/windows/lifecycle.mspx
does this optimization also apply to WinME (I believe so but I'd better ask) ?

Comment 20

[Ali Ebrahim]

Well according to MS, for NT/2000 (and I assume XP) platforms, /OPT:NOWIN98 is
the way to go. Since people are slowly moving off Windows 98, I don't think it
makes a whole lot of sense to regress back to support Windows 98 at the expense
of (much) newer and increasingly more common platforms.

Comment 21

[Rastignac]

OK, again, let's be clear.

The 4KB alignment gives a perfomance boost because Microsoft and Intel have
implemented a better handling of this type of executables inside Windows. The
4KB alignment makes the executables' boundaries "compatible" with harddisk cache
management and memory allocation management and swap management. So:
- A 4KB aligned executable can be loaded into memory directly from hard disk
cache (it's very faster and it eats less memory !).
- Also, it can be swapped out and swapped in much faster.
- Also, the executables can be reorganised in an optimized way on the hard disk:
the defrag programs will place these executables on the hard disk in the exact
same order they are loaded in memory by Windows, so they can be then loaded in
one pass only (no hard disk seeking at all).

This implementation exists in:
- Windows 98. (Let's look for the "applog" directory).
- Windows ME also.
- Windows XP. (Let's look for the "prefetch" directory).
- Windows 95 + Norton Speedstart (this little tool from the Norton Utilities is
acting roughly the same way and also gives a big performance boost).

This implementation does not exist in:
- Windows NT.
- Windows 2000.

So, that's the problems ?
- Windows NT and 2000 won't perform better with 4KB aligned programs. But they
WON'T PERFORM WORSE ! There's no impact at all: the programs won't gain
anything, but they won't loose anything. NT and 2K handle all executables the
same way (no more, no less).
- 4KB aligned programs are a little bigger than the non aligned ones. A few KB
are added to the executables. But it's very very few more KB. Also, these added
KB are zero filled, so the executables are bigger but they can be compressed as
well as not aligned ones, so the installation program won't be bigger to download.

So, what's the deal with Microsoft warnings ? Microsoft said: "if you build
NT/2000 programs ONLY, don't do OPT WIN98." Why ? Because there's no performance
gain; it just make the executables bigger and don't help performance with
NT/2000. For pure NT/2000 programs, there's no interest at all to use OPT WIN98.
But it doesn't harm !

If the executables are made for EVERY system (98+ME+NT+2K+XP), you must add the
OPT WIN98 option. Every single program made nowadays is 4KB aligned and there's
only one executable for every platform (programmers never give two version of
the same program, one for 98/ME/XP and one for NT/2K ! The same executable is
given for all the OSes). And they don't perform worse on NT/2000. These two OSes
don't perform better but they don't perform worse: they handle these executables
the same way other executables are handled (no more, no less). But 98/ME/XP DOES
handle them differently: they boost them !

So, this optimization should really be tried.


(Sorry to repeat myself again and again with the same points).

Comment 22

[Rastignac]

Target milestone was "1.7 alpha"...

Somebody should try to build Mozilla with 4KB-aligned executables and test:
- is there a speed-up ?
- are there regressions ?

The speed-up should be very visible on slow computers. But fast computers will
fell it also.

Comment 23

[Rastignac]

Why can't I change the target ?
I got this error: " You tried to change the Target Milestone field from
mozilla1.7alpha to mozilla1.8beta, but only the owner or submitter of the bug,
or a sufficiently empowered user, may change that field."
I am the submitter of this bug. Why can't I change the target ? A Bugzilla's bug ?

Comment 24

[Olivier Cahagne]

Can someone resolve this as per bug 263512 (dupe, wontfix, etc.) ?

Comment 25

[Jo Hermans]

This seems to be a good thing to do, espcially because it helps Windows XP.

Comment 26

[Chase Phillips]

Mass reassign of open bugs for [e-mail] to [e-mail].

Comment 27

[J. Paul Reed [:preed]]

Mass re-assign of bugs that aren't on the build team radar, so bugs assigned to [e-mail] reflects reality.

If there is a bug you really think we need to be looking at, please *email* [e-mail] with a bug number and explanation.

Comment 28

[Wayne Mery (:wsmwk)]

jo.hermans in comment #25)
> This seems to be a good thing to do, espcially because it helps Windows XP.

jo,

do you mean "good thing" to dupe to bug 263512, or "good thing" to go with 4k alignment?
 

Comment 29

[Steffen Wilberg]

Support for Win9x/Me/NT4 has been dropped in trunk builds (Firefox 3 etc.), see bug 330276. So there's no reason to optimize those trunk builds for Win9x. According to bug 263512 comment 12, this optimization for Win9x (4k-aligning by building without /opt:nowin98) would increase codesize.

"If you are building components that run only on Windows NT or Windows 2000, you should use /OPT:NOWIN98."
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore/html/_core_.2f.OPT.asp

So I'd suggest to WONTFIX this bug.

Comment 30

[Rastignac]

4KB-aligning helps also WinXP ! With WinNT and Win200, it's neutral (no gain but no loss). With WinXP, there's a gain !
WinXP's prefetching (see %windir%\prefetch) is better done with 4KB-aligned EXEs and DLLs. That helps. WinXP's defrag does also a better job with them. That helps.
Do not close this bug, please.

Comment 31

[Steffen Wilberg]

Same info for VisualStudio 2005: http://msdn2.microsoft.com/en-us/library/bxwfs976(VS.80).aspx

Comment 32

[Benjamin Smedberg]

Rastignac, can you provide documentation links for your claim that 4kb aligning helps the WinXP cache?

Comment 33

[Serge Gautherie (:sgautherie)]

(3 years later)

No reply from reporter.

***

After a brief look, I couldn't find official documentation about this option and WinXP:
especially, Microsoft seems not to have updated its documentation which only cite 98/NT/2K :-/

Yet note that (wrt future):

http://msdn.microsoft.com/en-us/library/bb531344.aspx
{
Visual C++ 2008

The Windows 95, Windows 98, Windows ME, and Windows NT platforms are no longer supported. These operating systems have been removed from the list of targeted platforms.
}

http://msdn.microsoft.com/en-us/library/bxwfs976.aspx
Microsoft Visual Studio 2008/.NET Framework 3.5
still lists this option.

http://msdn.microsoft.com/en-us/library/bxwfs976(VS.100).aspx
Microsoft Visual Studio 2010/.NET Framework 4
doesn't list the option anymore.

***

http://www.microsoft.com/whdc/archive/XP_kernel.mspx
"Logical Prefetcher for Faster Boot and Application Launch

When a Windows XP-based system is booted, data is saved about all logical disk read operations. On later boots, this information is used to pre-fetch these files in parallel with other boot operations. [...]"

This is not a cache and is unrelated to the /OPT option.

***

Windows 98 is (as) not supported anymore (by (Microsoft and) Mozilla).

(May have been "WontFix", now is) R.Invalid

Comment 34

[(not currently active) Ted Mielczarek]

Pretty sure we had -OPT:WIN98 in the build system somewhere, and I removed it along the way.

Comment 35

[Serge Gautherie (:sgautherie)]

(In reply to comment #34)

See bug 530093...