Closed Bug 72551 Opened 19 years ago Closed 19 years ago

Opening chrome JARs inefficient

Categories

(Core :: Networking: JAR, defect)

defect
Not set

Tracking

()

RESOLVED FIXED
mozilla0.9

People

(Reporter: dveditz, Assigned: dveditz)

References

Details

(Keywords: perf)

Attachments

(3 files)

nsZipArchive reads too much and in too many small chunks when opening JAR 
archives, and we open 9 or so on startup. Averaging three runs after reboots I 
see about 2.8 seconds spent in BuildFileList(); on a second start it averages 
about .08 seconds on WinNT. On a first load the time is roughly proportional to 
the number of files in the archive, not the size of the archive (which makes 
sense, we're processing the archive's directory). One exception is en-US.jar 
which seems to take a disproportionate amount of time to load.

The worst thing libjar does is while reading the central directory it PR_Seeks 
back to the local header for each file, reads 30 bytes (to figure out the true 
offset of the file data), and then seeks back to the central directory to get 
the next file's data.

Instead we should 1) read and process the central directory in big chunks, and 
2) leave the file data offset unset until needed. This will eliminate 3 
PR_Reads and 2 PR_Seeks for each archived file -- thousands of OS calls 
eliminated.

I'm not sure how much of the 2.8 seconds on a first start can actually be saved 
though, since the bulk of the time (as judged by the difference preloading 
makes) seems to be in reading the file data.
nominating and tying into the master perf tracking bug
Blocks: 7251
Status: NEW → ASSIGNED
Keywords: nsbeta1, perf
Target Milestone: --- → mozilla0.9
The attachment shows stack traces from PR_Read calls, sorted by unique filenam & 
stack. Note that nsZipArchive::BuildFileList() is causing huge numbers of reads 
(2051 for modern.jar).
That's what I said when I opened this bug -- 3 reads and 2 seeks for each file 
in an archive. modern has 680 or so chrome files so ~2000 reads is exactly what 
I was describing.

I'm testing my fix for this now.
Attaching a patch that reduces the first run "open jar" times from 2.8 seconds 
on average to 0.4 seconds. This was a big and pleasant surprise -- I figured 
that since the second-run time was only 0.08 seconds before that the bulk of 
the time was in the raw disk reading that we would still be doing. (second-run 
times are now around 0.02 seconds.)

For a first run this shaves about 10% off my win32 start.

The patch:
#defines in the zipstub.h and nsZipArchive.h header files are to reduce the 
clutter of #ifdef STANDALONE in the nzZipArchive code; otherwise the only 
change here is a minor signature change and removal of an unused member.

The real change is in nsZipArchive.cpp where the BuildFileList() algorithm 
changed as outlined in the opening comment of this bug, namely read in big 
chunks and leave item->offset null until needed to avoid seeking up to the top 
of the archive and back for each file.

The changes in nsJARInputStream.* are just opportunistic cleanups.
Keywords: nsbeta1nsbeta1+
wow, very nice.  even the installers (both win mozilla and win ns) seem quicker 
now during the xpinstall phase.  The bits are also getting uncompressed 
correctly.

r=ssu
waterson gave verbal sr=waterson on this at the performance meeting.
fix checked in 2001/4/3
Status: ASSIGNED → RESOLVED
Closed: 19 years ago
Resolution: --- → FIXED
this is great!
2051 reads down to 54 reads for modern and other files.

Thanks!  :-)
2051 vs 54 isn't the right way to read the attachments.

Originally there were 2051 modern.jar reads in BuildFileList and 99 in 
various InflateItem stack traces.

Now there are 18 modern.jar reads in BuildFileList, 106 in InflateItem (must 
have added 7 chrome files to startup), and 99 in SeekToItem.

Mostly my patch was to BuildFileList (2051 -> 18 reads), but that gain came in 
part by defering some reads (the 99 new SeekToItem reads).

*** Bug 69140 has been marked as a duplicate of this bug. ***
No longer blocks: 7251
Blocks: 7251
Component: XP Miscellany → General
QA Contact: brendan → general
Component: General → Networking: JAR
QA Contact: general → networking.jar
Depends on: 728457
You need to log in before you can comment on or make changes to this bug.