64764 - very long list of recipients hangs mozilla for a very long time

Status: VERIFIED FIXED

(MailNews Core :: Backend, defect)

Description

[Oliver Klee]

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; m18) Gecko/20010106
BuildID:    2001010620

This mail has an extremely long list of recipients. Trying to display the mail
hangs mozilla.

Reproducible: Always
Steps to Reproduce:
1. Put the attached mail file into an existing mail directory.
2. Open mozilla.
3. Select the mail folder you copied the mail file into.
4. Select the (only) message in that folder.

Actual Results:  Mozilla hangs with 100% CPU.

Expected Results:  The mail is displayed.

Comment 1

[Oliver Klee]

Attachment: Mail file that causes the hanger.

Comment 2

[[e-mail] (gone - use [e-mail] instead)]

It opened fine for me using Windows 2000, build 2001011104, but my machine has
128 megs of RAM, and is a PIII 750 mhz.  I'll try this on my P133 with 64 megs.

Comment 3

[[e-mail] (gone - use [e-mail] instead)]

Tried this also on my slower machine mentioned above, and viewed the message
fine.  I'm using IMAP4.  I'd feel more comfortable if someone from the Mozilla
community helped out with verifying this also.

Comment 4

[Bradley Hart]

I see this also, but on Mac OS 8.6, 266 MHz (definatly a low-end machine). 

Changing platform/OS to ALL, and CONFIMRMING.

Comment 5

[Sheela Ravindran]

change qa contact to myself, cc-> Esther

Comment 6

[Oliver Klee]

Adding hang keyword.

Comment 7

[David :Bienvenu]

could you retry this?

Comment 8

[Oliver Klee]

I'm still seeing this on 2001033004 Win2k.

Comment 9

[Oliver Klee]

Nominating for 0.9.1.

Comment 10

[a-huntwork]

On my machine, an athlon 900 with 192 MB ram running linux, displaying a message
with about 800 recipients causes mozilla to become non-responsive for about 5-10
seconds (won't even repaint).  I'm using 2001051306 on an simap account.  Kind
of annoying when it happens, but how often do you get a message with 800
recipients?  This is my first.  So I can verify that this bug exists, but it
doesn't bother me much.

Comment 11

[Oliver Klee]

*** Bug 96390 has been marked as a duplicate of this bug. ***

Comment 12

[Oliver Klee]

The testcase from duplicate bug 96390 is
http://bugzilla.mozilla.org/showattachment.cgi?attach_id=46715 (a Mozilla
MailNews file).

Comment 13

[Oliver Klee]

Ok, this technically isn't a hanger but a severe performance problem: Displaying
http://bugzilla.mozilla.org/showattachment.cgi?attach_id=46715 took me 28
seconds with 100% CPU on an Athlon 1400.

Removing hang kw, lowering severity to major, adding perf kw. Nominating for
0.9.5, 0.9.6 and 0.9.7, removing verifyme kw.

Comment 14

[Giovanni Gatti]

please, try with the theme SKYPILOT.

I installed it and seems much faster in handling long series of TOs and CCs.

Comment 15

[Giovanni Gatti]

hallo. Just received a message with 223 TOs. I use Skypilot theme. The first
time I opened the message, it took 8 seconds with my Powerbook 333 MHz and OS
8.6. Then, after opening once, the message displays very quickly.

Hope this helps

Comment 16

[Jacob Eisinger]

*** Bug 107275 has been marked as a duplicate of this bug. ***

Comment 17

[Denis Antrushin]

Mozilla spends a lot of time in function OutputEmailAddresses()
(mozilla/mailnews/base/resources/content/msgHdrViewOverlay.js),
adding emails addresses to header pane. I think it has sense to limit 
amount of addresses added to 'To:' element of header pane to, say, 300.
Anyway, adding more entries has no sense, because that pane is not
scrollable...

Comment 18

[Denis Antrushin]

Attachment: limit number of addresses in msg header pane (hardcoded)

Comment 19

[Jacob Eisinger]

I disagree that you should limit the number of addresses because then the only
way to tell who it is addressed to is to check the message source.

Also, according to the Windows 2000 Task Manager, mozilla starts off w/ about 40
mb allocated, jumps up to 76 mb allocated when the message (>1000 address) is
displayed and then jumps back down to 43 mb of ram when the message is closed. 
Clicking the down arrow to display the addresses does not cause much extra CPU
cycles/ram to be allocated.

Comment 20

[Denis Antrushin]

Well, can you see last address of 1000+ recipients? ;-)
I can't - such long list doesn't fit to my screen and this 
list not scrollable.

As for memory consumed, mozilla created new dom node for every
email address being displayed plus node for address separator
(comma). When you close this message/select another one,
this nodes are freed.

I don't see much possibilities to speedup process of creating/inserting
of node into dom tree (well, honestly, I'm not expert here). One 
possibility to speedup it a little bit is to use cloneNode() for email
separator instead of creating it from scratch everytime
(see InsertEmailAddressUnderEnclosingBox() function). (it's about 2x faster)

Also, looks like the first call to msgHeaderParser.ParseHeadersWithEnumerator
in OutputEmailAddresses() is redudant - it used only to get number of emails
(numOfEmailsInEnumerator), but I didn't find any use of this variable.
But msgHeaderParser.ParseHeadersWithEnumerator takes almost no time, compared
with time taken by dom tree operations....

Comment 21

[Denis Antrushin]

Using node.cloneNode() instead of document.createElement() and removing
redudant call msgHeaderParser.ParseHeadersWithEnumerator() gives
only 5% of performance win. (I don't attach patch, becuase I can't
connect to cvs server).
It looks like some rework needed to further improve performance....

Comment 22

[Scott MacGregor]

*** Bug 109915 has been marked as a duplicate of this bug. ***

Comment 23

[esther]

Note: this became evident again today with a message sent to many of us with 647
recipients.  Not only does it take a long time to display the message with the
large recipient list, but when you switch to another message it takes a long
time to view that message too.  

Comment 24

[Scott MacGregor]

one of my mail display perf bugs (for normal message cases) happens to also help
with this bug a little bit. But it's still slow. Marking this bug dependent on
109931.

Comment 25

[Denis Antrushin]

Attachment: Parse email address in the single pass

In this patch I try to parse address in a single pass.
Ideas behind this patch:
1. Originally, we copy all chars not inside () into addr_buf
array (dequoting them). So, when we encounter mailbox start
symbol ('<'), we have display name in addr_buf array already
and we can just copy it to name_buf (am I buggy?) - it's dequoted already.
This saves us second pass copying and dequoting again.
2. Convert ascii whitespaces ('\t', '\n', ' ') into plain space in place,
so we don't need to loop over the whole array again after the parse.

If I didn't add bugs here, it gives us some speedup :).

Also, it may be not in scope of this bug, but I noticed that email addresses
(and subject and (some) other headers) are re-parsed everytime, even when I
just
move mouse in/out of mail window (I have sloppy focus in my WM). Well,
addresses
parsed TWICE! Is it possible/reasonable to cache parsed headers? At least
To:, Cc: and Subject....

Comment 26

[Denis Antrushin]

Attachment: fix mistype in previous patch

This patch fixes typo in previous one.

Comment 27

[R.K.Aa.]

*** Bug 97930 has been marked as a duplicate of this bug. ***

Comment 28

[Scott MacGregor]

I think this is pretty much fixed. Many of my message display optmizations in
22960 also ended up addressing this problem. I have a test message with 1,000
recipients.  It now loads just as fast as the same message with only 3
recipients. Then if I click the email address toggle to go into the long view
there's a slight pause then I see the 1000 recipients. 

Moving to .9.8 and down grading the severity due to the recent optimizations. 

Comment 29

[Scott MacGregor]

I'm actually going to mark this fixed per my comments above. 

Comment 30

[[e-mail] (gone - use [e-mail] instead)]

Clicking on the test message
(http://bugzilla.mozilla.org/showattachment.cgi?attach_id=22144) is no slower
than any other message, but expanding it is very slow.  Scott, should we have a
separate, new bug for 'Expanding fields of extremely long recipient lists is too
slow'?  I can try and get comparative data from other clients (I didn't see such
a bug in bug 22960).

So, using build 2001-12-17 trunk builds, this is FIXED (since the different
behavior I believe warrants a separate bug).

Comment 31

[Oliver Klee]

Looks like the separate bug got field: Bug 194709

Comment 32

[Patrik Nilsson]

This bug is back in 17.0.2 (on Linux).