79580 - Reorder jar files to improve startup performance

Status: RESOLVED WONTFIX

(SeaMonkey :: General, defect)

Description

[Roger Chickering]

In a hallway conversation today, I agreed to prototype a tool
which would reorder the contents of .jar files so that contents
are stored in the same order in which we read them at startup.

-Roger

Comment 1

[Daniel Mario Vega (Restore MNG Support --> #18574)]

Will it produce a noticeable performance improvement ?

I suspect this change will be very difficult to maintain in future releases.
We will need to keep a list of files sorted in the "proper" order, updated to
reflect code changes and jar changes.

Does it worth the trouble ?

Comment 2

[Daniel Veditz [:dveditz]]

As with all performance optimizations you have to do the work and then 
measure to see if your guess at what would help really did.

Comment 3

[selmer (gone)]

Would it make sense to have some API where all the contents of a jar are just
dumped into the cache without using all the streams etc?  (Seems like that could
be a big win for some jars.  Pav has also been interested in getting directly at
gif files without all the overhead of using our streams...)

Comment 4

[Roger Chickering]

I think it makes sense to bypass nsIOService when we *know* we're
reading from the local disk.  Once that has been accomplished, I'm
not sure that preloading the image cache with images needed for
start-up will help performance... but it's certainly worth trying!

Comment 5

[Roger Chickering]

I have prototyped a .jar file reorderer, and run a bunch of
start-up timings.

Result: a very small improvement in cold start and no improvement at
all for warm start (The numbers actually show warm start getting
worse, but by such a small amount that it's in the noise).

These results are not surprising, although I was hoping for more
improvement in the cold start case.

I ran this on a 650MHz PIII, 512MB, NT 4.0.

Here are my timings:

    cold start reorder:
	 15.232
	 15.783
	 15.942
	 15.853
    avg: 15.703
    
    cold start original:
	 16.354
	 15.663
	 15.983
	 16.234
    avg: 16.056
    
    net diff: .356 (2.2% better)
    
    
    warm start reorder:
	 3.515
	 3.535
	 3.555
	 3.515
    avg: 3.530
    
    warm start original:
	 3.515
	 3.515
	 3.525
	 3.545
    avg: 3.525
    
    net diff: .05 (1.4% worse)

I'm going to attach some stuff to this bug so other people can try on
other platforms to see if it makes more difference.  Please contact me
if you need help!

Comment 6

[Roger Chickering]

Attachment: Changes to mozilla/modules/libjar/nsJAR.{h,cpp} for data collection

Comment 7

[Roger Chickering]

Attachment: order.zip: contains order files I generated

Comment 8

[Roger Chickering]

Attachment: order-jar.pl: perl script to reorder .jar file based on .order file

Comment 9

[Roger Chickering]

OK, here are some suggestions for using this stuff (assuming you're
running bash from cygwin; sorry, I'm a Unix guy :-):

Save order.zip and order-jar.pl in c:\temp

    cd $MOZ_SRC/dist/WIN32_O.OBJ/bin
    mv chrome chrome.orig
    cp -r chrome.orig chrome
    cd chrome
    unzip c:/temp/order.zip
    for package in US chatzilla comm en-US en-win help messenger modern toolkit
    do
	perl c:/temp/order-jar.pl $package.jar $package.order
    done

Now try measuring start-up time with the new chrome directory, and
comparing with start-up time with the old chrome directory.

(There's no need to apply the patch in the bug; that's just for
generating the .order files)

Thanks!
-Roger

Comment 10

[Cathleen]

forgot to update my results to this bug:

machine used: win98 200mHz, 128Mb RAM
I ran the test last Friday, 5/18, and used a commercial build from that day.  
Here is the average of 3 runs from a watch clock.

       original JARs      with JAR ordering
cold      27.34                 27.66
warm      10.33                 10.66

There isn't much difference I saw, but again I tried this on commercial build, 
and not all the JAR files got reordered.   

I guess I need to try apply the re-order tool to all JARs to see if that helps.  
Roger, will your re-order tool work with other JARs that's not listed in your 
comment above?

Comment 11

[Roger Chickering]

I generated those .order files using a commercial build, so
any .jar files that did not have corresponding .order files
represent .jar files that were not accessed during startup
under my instrumentation.

So re-ordering .jar files does not improve startup time.
Surprising, but good to know.

Comment 12

[Roger Chickering]

Given the data from our experiments, reordering the .jar files
does not help very much.