Closed Bug 88489 Opened 23 years ago Closed 16 years ago

Lag and momentary freeze adding large attachment in compose message

Categories

(MailNews Core :: Composition, defect, P1)

PowerPC
All

Tracking

(Not tracked)

RESOLVED WORKSFORME
mozilla1.2alpha

People

(Reporter: benjamin, Assigned: bugzilla)

References

Details

(Keywords: hang, perf)

Attachments

(1 file)

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Macintosh; U; PPC; en-US; rv:0.9.2) Gecko/20010629
BuildID:    2001062823

When I attache a large file (for exemple 15MB) a very long lag happens for about
4 minutes. A can move the cursor bug everything else freeze. I don't think it's
a network problem (I'm on a ethernet LAN) because I can do the exact same thing
in NS4.7 in less than 30 secs! It's probably the sequence "read file ->
assembling message -> write in sent folder" that isn't enough optimized... 

Reproducible: Always
Steps to Reproduce:
1. New message
2. Attache something *big* ex: 15mb
3. Press Send and wait...

Actual Results:  Freeze for 4 minutes...

Expected Results:  No Freeze and should be faster! (Less then 30 secs like NS4.7
at less)
Assignee: mscott → ducarroz
Component: Networking - SMTP → Composition
QA Contact: nbaca → sheelar
.
QA Contact: sheelar → esther
using branch build id: 2001070205 on mac. It took me about 4 minutes and 10 
seconds to send a 14MB attachment.  The application did not appear to freeze 
all the time. In between certain process while it was sending and copying to 
sent folder I could select other applications. But it took too long.  
I sent the same file using 4.7 with the same server and the whole process was 
done in 1 minute and 17 seconds.  
confirming 
Status: UNCONFIRMED → NEW
Ever confirmed: true
cc-ing related people. Sending process has slowed down in recent builds. As I
mentioned below for comparison same file attached and sent with 4.x took less
time than 6.x.  I don't think this has anything to do with server behavior. I am
not sure if any regressions have caused this in the recent build. I will try
this on both win98 and linux and update the bug.  
nominating this for nsbeta1. 
Keywords: nsbeta1
QA Contact: esther → sheelar
Can you find a recent build where it was faster?  Could the progress dialog
affect this?
I tested on windows98 using the same file (14MB).  It took 1min 42seconds on
trunk build 2001070206. On 4.x it took about 1min 46sec.  So this seems to be
working okay on windows. I still need to check this on linux and will update
after I do.  

I could go back a few builds before the progress dlg and check to see if it was
faster than what it is now.  
the mac probably doesn't like the async smtp feature. We block and restart
threads which the mac doesn't like. That's probably the problem. I suspect we
are just going to have to live w/ it since we don't like the old behavior of
blocking the UI.
With Build 2001070310 MacOS 8.6 14Mo attachment

Operation     Click send    Attaching       Send        Write           Completed
Freeze            |    No      |     Yes     |     No     |       Yes       |
Status Dialog     |    -       |  Attaching  | Deliv.mail | Deliv.mail100%  |
Time              0           0:36          1:36         2:50             7:56

LAN speed when Delivering mail ~340k/s on 10base-t

All the lags are with disk R/W operations...
Here are the few builds that I tried sending the same 14MB file:

Latest 2001062704 trunk build:  4mins 42sec
oldest build I had 2001060506 trunk build: 4mins 59sec
Retry under NS4.77 with exact same Network Congestion/File/Server load as at 
2001-07-03 16:03 --

Complete sequence in 1:21. The write sequence take less than 7 secs. 

Operation     Click send    Attaching       Send        Write           Completed
Time              0           0:01          0:21         1:14             1:21

NS indicate a Read speed of 4750k/s when assembling.
LAN speed when Delivering mail ~360k/s on 10base-t

And no lags/freeze effect on the OS.

Any update on this one? I can't use Moz as emailer as long as this bug is
present...  lose 10 minutes of productivity every time I send something *big*
can cost lot of time at the end of the month! I think for macuser this bug is P1.
Keywords: hang, mail1, perf
*** Bug 98264 has been marked as a duplicate of this bug. ***
OS: Mac System 8.6 → All
Please take at look at Bug 97922.

**

I'm not sure that Bug 98264 is dup of this one because -> here is with local 
folder Bug 98264 is with IMAP folder : 

> 2. Select Edit | Mail/News Account Settings, set the Sent
> folder to the folder  on an IMAP account. This problem is
> not reproducible when the Sent folder is located in local directory.

I think the time is being spent in nsMsgComposeAndSend::MimeDoFCC(), where we do 
lots of disk I/O in small chunks. I think it could be heavily optimized.
*** Bug 97922 has been marked as a duplicate of this bug. ***
This is a dup of 100353 which has been fixed.

*** This bug has been marked as a duplicate of 100353 ***
Status: NEW → RESOLVED
Closed: 23 years ago
Resolution: --- → DUPLICATE
Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---
No, it's not. The freeze described here is in disk I/O, as I said above:

>I think the time is being spent in nsMsgComposeAndSend::MimeDoFCC(), where we do 
>lots of disk I/O in small chunks. I think it could be heavily optimized.
oops, my mistake! Thanks simon for the catch.
Status: REOPENED → ASSIGNED
Target Milestone: --- → mozilla0.9.6
Blocks: 104166
moving to 0.9.7
Keywords: mail1
Target Milestone: mozilla0.9.6 → mozilla0.9.7
Keywords: nsbeta1+
I have identified several locations in the code where we are bloking:

1) When we write parts (nsMsgSendPart.cpp), we read data by block of 4K and then
write it to the temp file line per line, all this is done synchronously.
Unfortunately, it's not a simple task to fix it, it will need a major rewrite.

2) When we prepare a new temp file for the FCC operation (nsMsgSend.cpp). We
read and write line per line, again synchronously. I have a fix for that for
which I'll post a patch soon.

3) during the copy into a local folder (nsLocalMailFolder.cpp). Again a lot of
read/write done synchronously. The best way to fix this part is to be able to
stream the data to copy instead of passing a file. That will remove point #2
any luck with the patch?
Priority: -- → P1
The patch is covering only 1/3 of the problem. The full solution is way more
complexe and therefore need more time. I think we should push this one to the
next milestone.
Target Milestone: mozilla0.9.7 → mozilla0.9.8
move to 0.9.9 but it will probably not append before 1.0
Target Milestone: mozilla0.9.8 → mozilla0.9.9
Moved post 1.0
Target Milestone: mozilla0.9.9 → mozilla1.2
Blocks: 122274
Keywords: nsbeta1+nsbeta1-
No longer blocks: 122274
nominating this for the next release 
Keywords: nsbeta1-nsbeta1
QA Contact: sheelar → stephend
Blocks: 168430
With build 20020826 I am able to send a 17MB file in 62 seconds.

The only thing that is needed is a dialog while assembling the attachment (bug
168430).
Sorry, didn't mention that I'm using Windows NT4 SP6a
*** Bug 184675 has been marked as a duplicate of this bug. ***
What I don't quite understand is why this kind of thing can happen - one thing
is that the process of sending may be disk or CPU intensive or whatever, but it
should not freeze the entire application. Is Mozilla built with a classical
Windows mindset? But even in Windows, nowadays, you can put this kind of process
in a background thread, or even better: in another process. I personally can
live without a real time progress indicator - if there has to be a progress
indicator, either put this in another process, or at least detach the progress
indicator from the I/O process, which can then run in the backkground.
Mail triage team: nsbeta1-
Keywords: nsbeta1-
Keywords: nsbeta1
Product: MailNews → Core
still a problem?
Yes... and no... the write operation (15MB file) takes around 20 secs (whole
process... Click send -> Completed... takes 35 secs) on my PowerMac G4 1.25Ghz
MacOS 10.3.9 (Original report was under MacOS 8.6 PowerMac 8600 G3@400Mhz, my
mail server is now a dual PIII@1.4Ghz instead of a PowerMac 7500@200Mhz and the
network is now 100Base-t). Still during the 20 secs of writing, I can't do
anything else (in Mozilla)... also sometime I get the spinning beach ball and
BTW... Thunderbird is also affected...  but if we (in fact you... because I
don't have the knowledge to fix the problem) wait 3 to 5 more years the Moore's
law will probably do the job... ;-)
do you still see this problem?
there is also bug 358421, bug 240307, and bug 354915
Jean-Francois Ducarroz comment #22
> The patch is covering only 1/3 of the problem. The full solution is way more
> complexe and therefore need more time. I think we should push this one to the
> next milestone.

it's not clear to me what "1/3" of comment 19 is covered by attachment 57972 [details] [diff] [review].

is there more than just the Mac-only portion?

 
Severity: normal → major
QA Contact: stephend → composition
Summary: Lag and momentary freeze with large attachment → Lag and momentary freeze adding large attachment in compose message
reporter writes "I am now using Thunderbird and everything is fine."  so => WFM
Status: ASSIGNED → RESOLVED
Closed: 23 years ago16 years ago
Resolution: --- → WORKSFORME
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: