If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

49 dlls loaded on startup: 50% of startup time

RESOLVED INVALID

Status

()

Core
XPCOM
P4
major
RESOLVED INVALID
18 years ago
10 years ago

People

(Reporter: Suresh Duddi (gone), Unassigned)

Tracking

({topperf})

Trunk
mozilla1.2alpha
topperf
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [nav+perf])

Attachments

(3 attachments)

(Reporter)

Description

18 years ago
Here is the list of dlls in the order in which they get loaded. We have two 
objectives:

1. Reduce the number of dlls loaded by removing code that caused dlls not 
required at startup
2. Combine dlls if that causes a performance improvement.

I am primarily focussing on (1) reduce dlls loaded from this list.

components\xpinstal.dll
components\jsloader.dll
components\appshell.dll
components\chardet.dll
components\uconv.dll
components\necko.dll
components\nkres.dll
components\nkfile.dll
components\mimetype.dll
components\ucharuti.dll
components\ucvlatin.dll
gkwidget.dll
components\rdf.dll
components\profile.dll
components\xppref32.dll
components\xpc3250.dll
components\nsprefm.dll
components\nkabout.dll
gkgfxwin.dll
gkweb.dll
gkplugin.dll
components\urildr.dll
components\history.dll
components\mork.dll
components\tbmb.dll
components\msgbase.dll
jsdom.dll
components\mozbrwsr.dll
components\oji.dll
components\chrome.dll
components\caps.dll
components\gkhtml.dll
components\gkparser.dll
components\ender.dll
components\msgcompo.dll
components\msgnews.dll
components\addrbook.dll
components\wallet.dll
components\gkview.dll
components\nslocale.dll
components\nsgif.dll
components\strres.dll
components\bookmark.dll
components\search.dll
components\xpiflash.dll
components\lwbrk.dll
components\nkhttp.dll
components\cookie.dll
components\nkcache.dll
(Reporter)

Comment 1

18 years ago
11.5 secs out of 22 secs of startup is solely due to dll loading.
Summary: 49 dlls loaded on startup → 49 dlls loaded on startup: 50% of startup time
(Reporter)

Updated

18 years ago
Blocks: 7251
(Reporter)

Comment 2

18 years ago
Ccing dveditz and ftang. Are there any xpinstall or intl that dont need to get 
loaded or may be loaded lazily after the first page shows ?

Seth is already fixing msgcompose and msgnews being loaded.
Status: NEW → ASSIGNED
Priority: P3 → P1
Target Milestone: M14
**Check a commercial build -- this list is NOT all that gets loaded** (all I'll 
say for fear of the paranoid marking yet another bug private)

xpinstall must get loaded, and in fact I went to some trouble making sure it's 
the very FIRST component -- it's what checks to see if an install needs 
completing and if autoreg needs to be done.

Later on it would need to be loaded before the first page load anyway because 
it adds objects to the DOM namespace.

xpiflash could go two ways, either combined into the xpinstall library, or 
delay the loading until some future time. Either way the version checking it 
does shouldn't happen at startup. I'm not sure if this feature is actually 
working for beta1, perhaps we can just not deliver it for now. CC'ing dougt

Comment 4

18 years ago
xpiflash: http://bugzilla.mozilla.org/show_bug.cgi?id=29275

Comment 5

18 years ago
It would be good to determine what the overhead is for loading a dll. I suspect 
we'd see some formula like:

  dllLoadTime = fixedOverhead + m * dllSize + n * dllEntryPoints 

With a little experimentation, we should be able to figure out m, n and the 
fixed overhead. By reducing the number of dlls we can eliminate the overhead. 
Maybe we can cull some entry points too if n is significant.

We need to also examine which dlls necessarily must go together -- i.e. most of 
networking is needed at the same time (see bug 17031), editor, browser 
components, etc. If we put too much in a dll, we end up spending time loading a 
larger dll than we need. If we put too little in, then we end up with multiple 
dll loads and more overhead.
Of course the values for overhead, m and n will differ per platform. Should we 
compromise for build consistency or go for the custom solution?  The latter 
scares me since I just tracked down a problem that stemmed from libreg being 
a static lib on Unix and shared on Mac and Win.
(Reporter)

Comment 7

18 years ago
For putting dlls together, the order of dll loading is important. The list I 
put in the buglist is the order in which dlls are loaded.

So it should be easy to make the decision of clubbing
components\necko.dll
components\nkres.dll
components\nkfile.dll
together as these get loaded one after another.

I would also like to not loaded anymore data than what would have been loaded 
before the first window is being shown. So like if dll A and dll B are loaded 
one after another but inbetween there is the toplevel window, then putting them 
together would cause the toplevel window to show up later than before.
(Reporter)

Updated

18 years ago
Depends on: 29359
(Reporter)

Updated

18 years ago
Target Milestone: M14 → M16

Updated

18 years ago
Keywords: perf
(Reporter)

Comment 8

18 years ago
Warren did the dll aggregation for necko. Nisheeth has an excellent thread on
this on n.p.m.peformance. Ccing nisheeth. Can you add relevat information.

Comment 9

18 years ago
Can someone measure the startup time now that necko dlls have been combined and 
see if the time is any better? (Ideally, do a profile before and after pulling 
my friday evening changes.)

Yes, I agree with dp that combining the necko dlls is a no-brainer since they 
all are needed to even load in the first bit of chrome, but measuring what 
difference this has made in startup time will help us understand whether it's 
worth doing this on a larger scale.

Comment 10

18 years ago
I've gone through the issues in the post to n.p.m.performance titled "Mozilla 
Startup Time" and filed the bugs listed below on people who I thought could be 
potential owners. 

Nisheeth Ranjan wrote:

> Some questions that I could identify based on this analysis are:
>
> 1) The docshell initializes the global history service when it is
> created.  This causes history.dll and mork.dll to get loaded.  Can we
> defer global history initialization till after the browser window has
> shown up?

http://bugzilla.mozilla.org/show_bug.cgi?id=38607 assigned to waterson.

> 2) The JS environment (nsJSEnvirnoment) initializes LiveConnect() when
> it is created which causes oji.dll and gkplugin.dll to get loaded.
> Can LiveConnect() initialization be deferred?

http://bugzilla.mozilla.org/show_bug.cgi?id=18277 already existed.

> 3) When the chrome protocol handler is created to load navigator.xul,
> nsChromeRegistry::AddToCompositeDataSource() is called which loads the
> "user-skins.rdf", "user-locales.rdf", "all-locales.rdf",
> "all-packages.rdf" datasources.  Each of these datasource loads is a
> synchronous load.  Can these datasource loads be deferred.  If not,
> can they be made asynchronous?

Hyatt said that this is not possible.

> 4) cookie.dll is currently loaded when the http protocol handler
> creates objects registered under the "http-startup-category"
> category.  Can we change this so that cookie.dll is loaded after the
> browser window has shown up?

http://bugzilla.mozilla.org/show_bug.cgi?id=38612 assigned to valeski.

> 5) Can we defer wallet initialization till after the browser window
> has come up?

http://bugzilla.mozilla.org/show_bug.cgi?id=38611 assigned to morse.

> 6) For some reason, RDFServiceImpl::GetResource() cause msgimap.dll
> and msglocal.dll to get loaded.  The call stack that causes both these
> dll loads is pasted below.  We should probably not load msgimap.dll
> and msglocal.dll before the browser window has come up.

A bug for this is assigned to Scott Putterman already.

> 7) bookmark.dll, search.dll, and directry.dll all get loaded when
> datasource attributes with the corresponding "rdf:____" are
> encountered in navigator.xul while building a content model.  Can we
> load these datasources up after the browser
> window has come up?

waterson said that it isn't a good idea to defer the loading of
bookmark.dll.  http://bugzilla.mozilla.org/show_bug.cgi?id=29359 was
already assigned to waterson for search.dll.  It seems like the decision
there was to load search.dll but defer actual searches till later.  I've
filed a new bug for directry.dll
(http://bugzilla.mozilla.org/show_bug.cgi?id=38620) and assigned it to
waterson.

> 8) shistory.dll and psmglue.dll get loaded through xpconnect calls
made from script that executes when navigator.xul's > onload event
handler fires.  Can we move the script that causes these dlls to get
loaded to another place so that these dlls > don't get loaded before the
browser window comes up?

shistory.dll: http://bugzilla.mozilla.org/show_bug.cgi?id=38621 assigned
to radha.
psmglue.dll: http://bugzilla.mozilla.org/show_bug.cgi?id=38622 assigned
to mstoltz.

Comment 11

18 years ago
Oops, this is Nisheeth on Troy's machine.  The previous post was by me, not 
Troy.
(Reporter)

Updated

18 years ago
Keywords: nsbeta2

Comment 12

18 years ago
nsbeta2-. Would love to have this perf work for beta3
Whiteboard: [nsbeta2-]

Comment 13

18 years ago
One way to cut down on the number of dlls loaded at Mozilla startup would be to 
preload some dlls at Windows startup. This should be an option during the 
instalation. 
"Should Mozilla preload some files at Windows startup? Yes/No"
Users who access Mozilla many times during a scession would see a net savings of 
time (Maybe 5 seconds saved each time Mozilla is launched vs 5 seconds longer 
Windows boot ONE time.) 
Users who are concerned with system resources could still select NO.

Comment 14

18 years ago
Moving all current open XPCOM and XPCOM Registry bugs to rayw since dp is on 
sabbatical.  rayw is now default assignee for these components.
Assignee: dp → rayw
Status: ASSIGNED → NEW

Comment 15

18 years ago
A few questions:

Does this startup time include loading a page?

Should it include loading a page?

I assume that these DLLs have all been properly based to unique locations, 
right?

If this needs to become my major priority, then so be it, but other teams are 
likely to better understand dependencies within their own code.  Once XPCOM 
itself uses a single DLL, do we have any good tools to bring to bear on the 
rest?  I had a good one I wrote ages ago at a former company.  I think writing 
one again is superior to trying to do without, because you really need to 
understand what is happening, and a list of dlls, or just trying to read all the 
source code does not do justice at all to what is taking the time.  It can be 
related to lots of different things  I wrote it using the win32 debug APIs.  Is 
that as good as any other platform I might use to create it?

Comment 16

18 years ago
This list fails to take into account Microsoft DLLs that are loaded, which may 
cause a great impact, depending upon which are used.

Possibly the worst DLL being loaded, from a performance impact viewpoint, during 
startup is winmm, which is apparently being brought in as a static dependancy 
from nspr4.dll and gkwidget.dll.  I believe this is probably a DLL that we only 
should bring in dynamically in quite special circumstances.

As soon as I can get my hands back on the right tools, I'll figure out the exact 
dependencies.

Comment 17

18 years ago
In gkwidget.dll, the static call that brings in winmm.dll is PlaySound.  
Assuming this is the best method to call, a dynamic wrapper should be written 
that causes the DLL to actually be loaded when the first sound plays.

In nspr4.dll, the static call that brings in winmm.dll is timeGetTime.  There 
ought to be a better way to get time than activate the whole multimedia 
libraries and drivers.  Can we find a substitute call?

Comment 18

18 years ago
One problem call is found in:

http://lxr.mozilla.org/seamonkey/source/nsprpub/pr/src/md/windows/ntinrval.c#75

It already has work-around code for when the dll is missing.  Let's forget ever 
calling it, and it can save significant startup time (after making PlaySound 
dynamic, as well).

Comment 19

18 years ago
The second problem call is:

http://lxr.mozilla.org/seamonkey/source/widget/src/windows/nsSound.cpp#81

This could be replaced with a dynamic call to the library.

I will produce patches.

Comment 20

18 years ago
Added NSPR engineers (larryh,wtc) to the cc list.

Comment 21

18 years ago
Created attachment 10198 [details] [diff] [review]
Changes to file mozilla/nsprpub/pr/src/md/windows/ntinrval.c

Comment 22

18 years ago
Created attachment 10199 [details] [diff] [review]
Changes to file mozilla/widget/src/windows/nsSound.cpp

Comment 23

18 years ago
I added patches to eliminate WINMM.DLL from the startup profile.  This is only
a very small thing, but once when I developed a major app for another company,
this particular fix cut a huge percentage off of startup time on Windows 95.   
Both patches must be in place before the win will occur.  If memory serves me, 
at the time, it was a huge win on win95 (millions of instructions less), and 
less of a win on win nt.  Obviously, preloaded DLLs could cancel the win 
entirely.

I tested:

Mozilla runs with the 2 patches, with no visible breakage.
Mozilla no longer activates winmm and mmdrv and so on with this patch.

I did not test:

Can we still play sound?
Did we see a loss in time precision that cost us anything important?

This DLL is obviously just a drop in the bucket, and it will take quite a while 
to really make an impact, but I request that anyone who reviews this who is
proficient in performance timings to see if it made a measurable result.

Comment 24

18 years ago
Removal of NSPR's dependency on winmm.dll is tracked as bug #42714.
Depends on: 42714
Adding pavlov on a guess because of the sound playing thing

Updated

18 years ago
Status: NEW → ASSIGNED

Comment 26

18 years ago
>       loaded = true;
>       HMODULE module = ::LoadLibrary("WINMM.DLL");
>       if (!module)
>       {
>         play = (PLAY)::GetProcAddress(module, "PlaySoundA");
>       }

should be: if (module) ?

Comment 27

18 years ago
Putting on nsbeta3 radar.
Keywords: nsbeta3

Comment 28

18 years ago
M16 has been out for a while now, these bugs target milestones need to be 
updated.

Comment 29

18 years ago
Yes, it should be if module.  

As I said before, this could take a lot of time and coordination to seriously 
attack the problem.  It is not yet clear to me that I should drop all other 
priorities and work on this full time for weeks.  Changing to M20.
Target Milestone: M16 → M20

Updated

17 years ago
Depends on: 46794

Updated

17 years ago
Depends on: 46797

Updated

17 years ago
Depends on: 46800

Updated

17 years ago
Depends on: 46802

Comment 30

17 years ago
I identified a couple of dll:s that were loaded but not used and filed bugs on 
those.
It's
addrbook.dll (bug 46794)
msglocal.dll (bug 46800)
mime.dll (bug 46801)
msgbsutl.dll (bug 46797)

Also nsprefm.dll is almost not used (bug 46802)

Updated

17 years ago
Whiteboard: [nsbeta2-] → [nsbeta2-][nsbeta3-]

Comment 31

17 years ago
Updating QA Contact
QA Contact: leger → gbush

Comment 32

17 years ago
Created attachment 14632 [details]
Summary of DLL's loaded in viewer, seamonkey, nav4x, IE

Comment 33

17 years ago
WINMM is also being pulled into memory by nspr, which is calling timeGetTime().

Comment 34

17 years ago
The NSPR winmm.dll fix has been checked in on
NSPRPUB_CLIENT_BRANCH.  (See bug #42714.)

Comment 35

17 years ago
RTM+? 

PS. MOST WANTED!!!

Comment 36

17 years ago
Edward: Welcome to xpcom!
Status: ASSIGNED → NEW
QA Contact: gbush → rayw
Target Milestone: M20 → mozilla1.0

Comment 37

17 years ago
Once again... attempting to reassign from Ray to Edward.
Assignee: rayw → kandrot

Comment 38

17 years ago
Small side note for anyone who's interested: Windows Developer's Journal
published a nice little tool in their latest issue called LoadSpy for monitoring
loading/initialization times for DLLs, etc. The code is at http://www.wdj.com/code/

Updated

17 years ago
Keywords: nsbeta2, nsbeta3 → nsbeta1
Whiteboard: [nsbeta2-][nsbeta3-]

Updated

17 years ago
No longer blocks: 7251

Comment 39

17 years ago
BAsed on recent numbers, this no longer seems to be true.  On a fast machine, 
there is a 12 sec launch time, but by removing the java plugin, it drops to 3 
sec (there is a bug on this).  On a slower machine, there is a 12-15 sec launch 
time the first time, then 3-4 sec the second time.  The extra 8-12 sec are dlls 
loading.  There is a solution in the works by rickg that preloads the dlls 
under Windows, therefore removing most of this dll load time.  There is also 
research into a static app.

If there is a current list of dlls that should not be loaded at start up, 
please add them to this bug, then create bugs for them.  Currently, I am 
assuming that this is either a solved problem (or being solved), and marking it 
future for now.
Status: NEW → ASSIGNED
Priority: P1 → P4
Target Milestone: mozilla1.0 → Future

Comment 40

17 years ago
The preloading option shouldn't be a crutch. Most people are (understandably) 
annoyed by apps always running in the background and using resources.

Updated

17 years ago
Whiteboard: [nav+perf]

Comment 41

16 years ago
reassign all kandrot xpcom bug.
Assignee: kandrot → dougt
Status: ASSIGNED → NEW
Target Milestone: Future → ---

Updated

16 years ago
Blocks: 98275

Comment 42

16 years ago
to cathleen per performance meeting.
Assignee: dougt → cathleen
btw, we're down to about 41 .dll's loaded on startup (w/out Java)

Updated

16 years ago
Blocks: 104166
has this changed for the better since 9-10?,  I think startup time has gotten
much better over the last several milestones, including upto 0.9.8. One I
remember, in particular is the smaller size (kb) images for splash screens, and
some about delaying code/dlls?  

Updated

16 years ago
Target Milestone: --- → mozilla1.2

Comment 45

16 years ago
Is there a startup-graph available to track the improvements made?

Updated

16 years ago
OS: Windows NT → All
Hardware: PC → All

Updated

15 years ago
Keywords: perf → topperf

Comment 46

14 years ago
Is this bug still valid since it is so old?  If it is, I want to vote for it. 
Assignee: cathleennscp → nobody
QA Contact: rayw → xpcom

Comment 47

10 years ago
this is not valid any longer.  we are building everything into a single dll/dso.
Status: NEW → RESOLVED
Last Resolved: 10 years ago
Resolution: --- → INVALID
Isn't it still valid for Seamonkey until it starts using libxul?

Comment 49

10 years ago
i do not think it is a core xpcom problem.  Feel free to open up application specific.
You need to log in before you can comment on or make changes to this bug.