26455 - Mail startup time is slow

Status: RESOLVED WONTFIX

(MailNews Core :: Backend, defect, P2)

Description

[Phil Peterson]

On Mac: 3.3s on 4.7 vs 15.7s on mozilla. Need to be within 2x to meet beta1
criteria doc.

Comment 1

[leger]

Putting on PDT+ radar for beta1.

Comment 2

[Scott MacGregor]

All I'll say is...if the browser ain't within 2x 4.7 on startup...we aren't
going to be either...

accepting moving to m14.

Comment 3

[Phil Peterson]

This particular stat covers mail launch time *after* the browser. However, I
understand that it might be the case that DLL loading time dominates in mozilla,
since 4.x loaded all the code up front. *If* it turns out that there's nothing
we can do short of loading all the mail DLLs at startup time, we can resolve
this bug invalid.

Comment 4

[Phil Peterson]

P5 for M14.

Comment 5

[scottputterman]

I have a fix that will get us back .17 seconds on my machine on startup.  It's 
not much, but it's something.  Perhaps close to 1s on a target machine.  I was 
needlessly handling notifications before a folder was even loaded.

Comment 6

[scottputterman]

I've been Quantifying mail startup through loading my Imap Inbox over the last 
few days and another thing I see taking up time is traversing the file system to 
load local folders as well as imap folders.  

I know that someone should be able to drop in a folder into the pop folder 
hierarchy and have it show up, but I'm wondering if there's a different way to 
do this.  It seems a waste to have to traverse the directories every time when 
most of the time people won't be adding/removing folders outside of 5.0

I'm wondering if we could somehow either use the folder cache or have another 
file that knows about the pop folders and then use something like the subscribe 
dialog for people who are modifying the file system themselves.

On my machine, with about 180 folders I'm seeing close to .5 second being spent 
doing this.  I think that translates to about 2.5 - 4 seconds on the target 
machine.

Comment 7

[scottputterman]

Even though this bug is for the Mac, all of my measurements have been on 
Windows.

Comment 8

[Alec Flett]

I'm sure mac is still very I/O bound, so this is probably quite relevant.
A question: are we traversing the whole local folder hierarchy even if it isn't
open in the folder pane? seems like we could be REALLY lazy about this...

I think there are times when we just want to know if a folder HAS subfolders,
but not necessarily what they are...I'll bet we could figure that out easily
without actually creating/traversing subfolders.

Comment 9

[Phil Peterson]

Hmm. 4.x also discovers all the summary files by traversing the filesystem, and 
it seems to launch pretty quickly. 

Comment 10

[scottputterman]

I'm not saying it's our only problem.  I'm just saying that I see it standing 
out. I might have all of my folders open right now which is why the number is so 
high.

I don't know if anyone has tried this before, but I tried it this weekend.  I 
made mailnews into one dll.  That's another place where I see us spending a lot 
of time, loading dll's.  I have no idea if this makes any difference.  I'm going 
to ask Suresh to try this out for me.  It did shrink the total size of mailnews 
down by a bit over 200K on an optimized release build and by over 600K on my 
debug build.  And it eliminated 9 dll's. I'm not advocating this approach, I'm 
just curious if it will have any effect.

Comment 11

[Alec Flett]

I still think we should be moving the database "base" classes into
mailnews/base, moving the news database into the News DLL, etc, and we should
handle the inheritance issues via aggregation. 
That would eliminate the database DLL.

Next, we should eliminate the msgbaseutil DLL by moving it's classes into
mailnews.dll, and again solving inheritance issues via aggregation.

Finally, we should combine some of the MIME dlls, such as moving the default
emitters into the base library, and combining the signed and SMIME dlls

If we do all this, we condense 15 dlls down to 10.. 

Comment 12

[Scott MacGregor]

Hey Scott, Simon Fraser actually already tried doing this on the Mac (where
loading all of these libraries is more expensive than on windows) a couple
months ago.

He discovered that having just a couple shared libraries for the total system
wasn't making a huge dent in the overall startup time of the browser.

You can email him if you want more details about his results. But it didn't look
as promising as he thought it would.

Hopefully this info saves you some time....

Comment 13

[scottputterman]

Well, I have verified Simon's results :)

One dll vs 10 dll's made no difference in startup time or loading a compose 
window or pretty much anything except size.  

The things I've seen as taking a lot of time in startup based on Quantify:

Loading Dll's

Painting (a ton of time spent saving the clip rect.  I've been told that 
evaughan has a fix that might help change this. Currently 1/3 of painting is 
spent in this - that's .5 s out of 1.5 s on my machine).

File reading.  I'm not sure where this is coming from.  I'm sure it's some combo 
of reading in xul, prefs, and xpt files.

I have no proof of this one except for what I see when Quantifying because it's 
so slow.  But when I Quantify I watch the folder pane come up and see that the 
icons start off really small and then watch it get reflowed about 4 times until 
all of the icons are the right size.  The same thing happens in the thread pane.  
Even if Gecko is really fast, I imagine that this shouldn't be necessary and is 
probably taking up some time.

And then there was the folder hiearchy traversal I mentioned before.

Comment 14

[scottputterman]

Just to give some stats on file reading. This is from running Quantify starting 
up with -mail and then having my Imap Inbox loaded for me automatically.  No 
password dialog comes up. In the past I've found we can generally multiply my 
results by 5-7 times for the target machine.

nsFileTransport::Process = 1.69 seconds

of which:

nsPipe::nsPipeOutputStream::WriteFrom	= .936 seconds
nsLocalFileSystem::Open	 = .479 seconds 
nsPipe::nsPipeOutputStream::Flush  = .104 seconds
nsLocalFileSystem::GetInputStream = .076 seconds

Comment 15

[Scott MacGregor]

I believe we were punting on this for PDT. I'm voing to remove the label and let
them decide. But I'm pretty sure we talked about this bug as being a candidate
to cut. Unless scott P. has a miracle up his cap =).

Comment 16

[Phil Peterson]

PDT- for beta1

Comment 17

[Scott MacGregor]

Moving my remaining M14 bugs to M15 which is the next targeted milestone.

Comment 18

[Scott MacGregor]

triaging.

Comment 19

[selmer (gone)]

Not M16 stopper.  Marking M17.

Comment 20

[lchiang]

I guess I should change the summary to reflect beta 2.

On the 4/3 build, the timings were 21.096 with a standard deviation of                                                                          
0.104 so the timing has gotten worse.  This is mail startup after starting 
Navigator.

See http://www.mozilla.org/mailnews/mac_performance_results.html

Comment 21

[leger]

Putting on [NEED INFO] radar.  PR2 goal is now slower than PR1, how does todays 
commercial build compare to the PR1 final Mac build?  Show us the numbers! ;-)

Comment 22

[lchiang]

I didn't have info going back to Beta 1 (mid/late Feb), but I have results from 
3/7/00.

3/7/00:  17.568s w/a SD of 0.134
5/4/00:  17.322s w/a SD of 0.195

So we are slightly faster than PR1 (or thereabouts), but very slow compared to 
4.7.

Comment 23

[leger]

Putting on [nsbeta2-] radar.  

Comment 24

[scottputterman]

Mail triage marking [nsbeta3-]

Comment 25

[scottputterman]

changing summary since it's not just a beta2 problem.

Comment 26

[scottputterman]

marking nsbeta1-.  This isn't one of the top performance areas we are trying to
looking into during the nsbeta1 time period.

Comment 27

[scottputterman]

reassigning to cavin and changing platform because startup perf is slow everywhere.

Comment 28

[[e-mail] (gone - use [e-mail] instead)]

Attachment: Files loaded with mozilla.exe -mail

Comment 29

[[e-mail] (gone - use [e-mail] instead)]

Attachment: Files loaded after launching Navigator then Mail

Comment 30

[Ben Bucksch (:BenB)]

*** Bug 89061 has been marked as a duplicate of this bug. ***

Comment 31

[scottputterman]

investigation on mail startup time will be put off for a while.

Comment 32

[Martin Krzysiak]

Why does the mailer start _much_ slower than
starting the browser _and then_ the mailer?

Any ideas?

Comment 33

[[e-mail] (gone - use [e-mail] instead)]

Martin, because the Mail component uses lots of the libraries that Navigator
loads.  So, when launched by itself, it has to load those, but when Navigator
has already been launched, we just load what's left.

Comment 34

[David G King]

Changing milestone as 1.2alpha is done.

Comment 35

[kaido loor]

Is it possible, at startup of MailNews not to load Local Folders? I have in
Local Folders about 1000 subfolders and 6,3 GB mails in them, which makes
startup time of ~10-20 seconds when connected to LAN (and 5-10 minutes from home
1 MBit VPN connection). Maybe it is worthwile to postpone loading Local Folders
until user (or mail filtering) accesses the Local folders?

Comment 36

[Worcester12345]

Mail sure seems slow lately. It would be great to see this one move forward
before  RC's start coming out.

Comment 37

[Wayne Mery (:wsmwk)]

I don't see anything salvageable here.  Do you?
I think none of the suggestions held up after follow up comment(s).

(WRT comment 35, kaido is gone)

Comment 38

[David :Bienvenu]

no, I don't think there's anything salvageable here either.

Comment 39

[Wayne Mery (:wsmwk)]

David comment #38
> no, I don't think there's anything salvageable here either.

David, close bug 117130 also?

Comment 40

[David :Bienvenu]

yes, sure, that could be closed as well, since we don't really use the sidebar.

Comment 41

[Eyal Rozenberg]

Um, mail startup time is still slow, though...