HTML Mail View very slow (45++ sec) for pop and local folders - speed up FolderUriForPath first lookup when switching folders

NEW
Assigned to

Status

MailNews Core
Backend
--
major
8 years ago
a year ago

People

(Reporter: Bienvenu, Assigned: sid0)

Tracking

(Blocks: 1 bug, {helpwanted, perf, regression})

1.9.1 Branch
x86
Windows XP
helpwanted, perf, regression
Dependency tree / graph
Bug Flags:
wanted-thunderbird +

Firefox Tracking Flags

(blocking-thunderbird5.0 -, thunderbird5.0 wanted)

Details

(Whiteboard: [gs][ref comment 4, bug 536873], URL)

(Reporter)

Description

8 years ago
+++ This bug was initially created as a clone of  +++

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2) Gecko/20100115 Firefox/3.6 (.NET CLR 3.5.30729)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1

Some HTML Mail freezes Thunderbird for a while.
Mostly in large email folders but not necessarily a large email.
Emails came from Lotus-Notes-Client and contains only few text and some spacer GIF for layout.

Effect also occurs in '-safe-mode' of Thunderbird.
No anti-virus programm is activ.
Yes, Profile is in network folder but this is probably not the reason.
Text-Viw of same email is OK.
Only HTML view appears as problem.

CPU-use graph in Windows-Task-Manager shows very high peaks for a while.


Reproducible: Always

Steps to Reproduce:
1. create a larger Emailfolder (about 200 plus xx)
2. ask to get HTML-mails from different Lotus Notes User
3. Open Task-Manager for CPU-use
5. switch preview of emails an notice what happens
Actual Results:  
Email is not shown for a while. 
Can't change to another mail or function.

Expected Results:  
show the email content in preview window.
David, same problem as Bug 545126, isn't it?

(Time to take display an HTML mail) == A * B * C,
 where;
  A : Number of <img src="cid:..."> in an HTML mail
  B : Number of local mail folders before folder in which the HTML mail is kept
  C : Time to acquire directory information of a mail folder file

(Case-1) Original problem on bug opener of Bug 545126
  A: Large(Lotus Notes mail), B: Large, C: Large(remote profile, CSC of Win)
(Case-2) Minimized test case by bug opener of Bug 545126
  A: Small(reduced to 7), B: Large(same as Case-1), C: Large(same as Case-1)
(Case-3) My test with 10,000 local mail folders with test mail I attached
  A: Small(4 or 40), B: Very Large(10,000), C: Small(local profile, no CSC)
(Case-4) Bug 536873 (DUP'ed to Bug 545126)
  A: Medium, B: Larger than Case-1/Case-2, C: Small(local profile, no CSC)
(Case-5) This bug
  A: Large or Very Large (Lotus Notes mail),
  B: Medium to Large (200?),
  C: Not Small (remote profile, CSC?)

Is cached result(by change to "caching the most recent result", Bug 545126) lost if very many <img> exist(Case-5)?
Caching doesn't work well if special mail structure(not-well-formed structure, funny nesting of multipart)?
Or caching doesn't work well if wrong Content-Type?
(application/octet-stream for image data part with Content-ID:)
(Reporter)

Comment 2

8 years ago
WADA, I cloned bug 545126 so that we could do a non-caching fix, using sid's patch as a starting point. Caching doesn't help the first lookup, but subsequent lookups should be fast. This bug is about fixing the first lookup.

Comment 3

8 years ago
(In reply to comment #2)
> WADA, I cloned bug 545126 so that we could do a non-caching fix, using sid's
> patch as a starting point. Caching doesn't help the first lookup, but
> subsequent lookups should be fast. This bug is about fixing the first lookup.

bienvenu, standard8, is this bug still wanted, since the 3.0->3.1 major update has flown the coop?
(Reporter)

Comment 4

8 years ago
yes, if I remember, sid had some ideas about how to make the first lookup faster using a hash table that we keep up to date.
Assignee: bienvenu → sid.bugzilla

Comment 5

7 years ago
If someone sees slow message display on first view, but second and subsequent views are fast, then then there's a good chance they are seeing this bug?


sid, possible for you to revisit this soon?

Comment 6

7 years ago
I have a similar problem which may be related to Wada's Point B above.
B : Number of local mail folders before folder in which the HTML mail is kept
It shows itself with simple folder selection (a newsgroup about 150 folders deep)
Can take up to 30 seconds to select and show the threadpane.
Heavy disc I/O during that delay.
New bug..or related.
(Reporter)

Comment 7

7 years ago
(In reply to comment #5)
> If someone sees slow message display on first view, but second and subsequent
> views are fast, then then there's a good chance they are seeing this bug?
Yes, on a per-folder basis, iirc.  The first lookup from the first message loaded in a folder is slow; subsequent lookups on that message (i.e., the message triggers multiple path lookups), and lookups from other messages in that same folder should be fast, until you click on an other message in an other folder that does an other lookup, changing the cached lookup.

Comment 8

7 years ago
ah, as described by http://getsatisfaction.com/mozilla_messaging/topics/slower_performace#reply_4890042 for example

that's pretty severe. 

No workaround? Like keeping each affected folder open in a tab?
blocking-thunderbird5.0: --- → ?

Comment 9

7 years ago
(In reply to comment #8)
> ah, as described by
> http://getsatisfaction.com/mozilla_messaging/topics/slower_performace#reply_4890042
> for example
> 
> that's pretty severe. 
> 
> No workaround? Like keeping each affected folder open in a tab?

Yes, keeping the affected folder open in a tab fixes the problem for me.
However, it adds a problem. The msf files are loaded into memory. For system with limited memory (large folders) this is can be problematic.

Example:
1 tab open TB=86,280
4 tabs     TB=302,800

On a 512 meg box, you won't be doing much else with this workaround.

Comment 10

7 years ago
this should block - or at least get focus via "needs" - there are people leaving version 3 for version 2, etc because of this bug.
I don't think we can block 3.3 on this, mainly as we haven't got anyone to work on it and we don't have long before we branch - though it is definitely wanted (also, whilst I realise some users are affected, I'm not sure that it is significant enough that this is a stop-ship).
blocking-thunderbird5.0: ? → -
status-thunderbird5.0: --- → wanted
Flags: wanted-thunderbird+
Keywords: helpwanted

Comment 12

6 years ago
there are a few more reports of this on gsfn
Blocks: 564148
Summary: HTML Mail View very slow (45++ sec) for pop and local folders - speed up FolderUriForPath → HTML Mail View very slow (45++ sec) for pop and local folders - speed up FolderUriForPath first lookup when switching folders
Whiteboard: [gs][ref comment 4, bug 536873]

Comment 13

6 years ago
(In reply to David :Bienvenu from comment #4)
> yes, if I remember, sid had some ideas about how to make the first lookup
> faster using a hash table that we keep up to date.

Is it not time to dust this off?

I don't have time ATM to benchmark the current state of slowness, but I have to think that users are still seeing this.
Flags: needinfo?
Removing myslef on all the bugs I'm cced on. Please NI me if you need something on MailNews Core bugs from me.

Updated

a year ago
Flags: needinfo? → needinfo?(mozilla)

Updated

a year ago
Flags: needinfo?(mozilla) → needinfo?(vseerror)
(In reply to Joe Sabash [:JoeS1] from comment #6)
> I have a similar problem which may be related to Wada's Point B above.
> B : Number of local mail folders before folder in which the HTML mail is kept
> It shows itself with simple folder selection (a newsgroup about 150 folders
> deep)
> Can take up to 30 seconds to select and show the threadpane.
> Heavy disc I/O during that delay.
> New bug..or related.

Joe, can you profile using  https://developer.mozilla.org/en-US/docs/Tools/Remote_Debugging/Thunderbird ?

possible report http://forums.mozillazine.org/viewtopic.php?f=39&t=2345081&p=11618255#p11618255
Flags: needinfo?(vseerror) → needinfo?(jsabash)
(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #15)
> (In reply to Joe Sabash [:JoeS1] from comment #6)
> > I have a similar problem which may be related to Wada's Point B above.
> > B : Number of local mail folders before folder in which the HTML mail is kept
> > It shows itself with simple folder selection (a newsgroup about 150 folders
> > deep)
> > Can take up to 30 seconds to select and show the threadpane.
> > Heavy disc I/O during that delay.
> > New bug..or related.
> 
> Joe, can you profile using 
> https://developer.mozilla.org/en-US/docs/Tools/Remote_Debugging/Thunderbird ?
> 
> possible report
> http://forums.mozillazine.org/viewtopic.
> php?f=39&t=2345081&p=11618255#p11618255

Well, I am no longer reproducing the effects of this bug on any of my PC's with Trunk or any currently tested versions. Folder selection is less that a second for me.
Flags: needinfo?(jsabash)
You need to log in before you can comment on or make changes to this bug.