32222 - Send mail and duplicate in Sent should happen asnchronously

Status: VERIFIED FIXED

(MailNews Core :: Backend, defect, P2)

Description

[jefft]

Users cannot do anything when sending big attachment over slow momdem (say 56k 
modem). It will be a very bad first-hand experience to the user.

Comment 1

[Phil Peterson]

Rich, is this yours?

Comment 2

[jefft]

I'll take this.

Comment 3

[Richard Zach]

Jeff, you don't mean that you want to put a copy in the Sent folder before the
message was successfully sent, do you?  

Comment 4

[jefft]

No. What I meant is all the sending and copying process should happen in the 
background or at least time slicing it. We shouldn't tight up the cpu so that 
user cannot do anything other than sit there waiting...

Comment 5

[selmer (gone)]

Not M16 stopper, marking M17.  Please add beta2 keyword if you think this is a 
beta2 stopper.

Comment 6

[lchiang]

This is being talked about as an "exception" item in the PDT meetings.  Marking 
nsbeta2 keyword so it can be official whether we are going to have this for 
beta2 or now.

Comment 7

[Phil Peterson]

It's actually not on the current exception proposal list. If you think it's a
feature that we have to have for beta2, rather than a bug we can fix after
beta2, perhaps it should be on the exception list.

Comment 8

[lchiang]

Phil, I thought this was "SMTP feedback" in the exception list?

Comment 9

[leger]

Putting on [nsbeta2-] radar.  Putting on the nsbeta2 radar.

Comment 10

[lchiang]

- per mail triage

Comment 11

[jefft]

Adding mail1 keyword. We need to address this sooner than later....

Comment 12

[jefft]

*** Bug 34070 has been marked as a duplicate of this bug. ***

Comment 13

[scottputterman]

reassigning jefft's bugs to naving

Comment 14

[Azrael AOD]

Bug 34070 (Mail appears to hang upon sending large messages) is a dupllicate of 
this, verifying it as such. However the summary of this bug makes it awkward to 
compare bugs to, espcially is reporting a new bug.
Suggest summary altered.
Something like 'sending large mail messages too slow'

Comment 15

[scottputterman]

I think Jeff was going to try to fix this and then it turned out to be hard. Any
ideas on how much work this would be?

Comment 16

[scottputterman]

Scott or David, do you have any idea about the amount of work this would take?  

Comment 17

[David :Bienvenu]

It's just a matter of either doing this as part of a url (e.g., a file: url) or
using timers to simulate time slicing (e.g., start a recurring timer whose
callback does 16K of the work) and then stops the timer when done, and picks up
the send operation where it left off. The hard part might be that last bit.

Comment 18

[Scott MacGregor]

Naving, do you mind if I take this? I have to fix this in order to implement
progress on the smtp send operation.

Comment 19

[Navin Gupta]

Go ahead

Comment 20

[Scott MacGregor]

Thanks. I've been working with the necko folks to add the ability to properly
implement AsyncWrite (well they did all the work I just nodded by head =)).

We should now be able to leverage off of that to send the message in chunks
instead of sitting in a blocking while loop sending the entire message.

Comment 21

[scottputterman]

marking nsbeta1+

Comment 22

[Scott MacGregor]

I've implemented AsyncWrite for both smtp and news and re-written how we do
posting to leverage this. This should make a huge change for perceived
performance of message posting. 

Unfortunately these changes are blocked by changes I need from the necko branch
dougt is on. Without those changes AsyncWrite does not work. 

This bug blocks 65209 which is "report progress on sending messages". If the
sending isn't Async then there is never any progress to report =).

Comment 23

[Scott MacGregor]

Attachment: current fix: subclass nsMsgProtocol with a version that uses AsyncWrite

Comment 24

[David :Bienvenu]

pretty cool, Scott, sr=bienvenu.
 One thing, though, can we come up with a better error code than
NS_ERROR_FAILURE? We're going to have to get rid of all of those at some point.

Comment 25

[Scott MacGregor]

Attachment: updated fix that works with the necko branch landing today

Comment 26

[Scott MacGregor]

Great news, this actually works! Now that the necko branch landed I was able to
actually try it out. I can really notice the "perceived" performance gain when
sending large files now that we no longer block the UI for the duration of the
smtp send. 

This patch just contains some changes to make it build with the necko changes.
Many thanks to darin and doug for making AsyncWrite on a socket work!

One cool side effect: we don't need to read the message into a buffer in order
to scan for "LF." pairs before sending. I was able to write the send code to
check for this case by "peeking" into the input stream and not actually reading
the data out into a buffer.

Hopefully I'll land this, this weekend.

Comment 27

[Scott MacGregor]

*** Bug 72857 has been marked as a duplicate of this bug. ***

Comment 28

[Scott MacGregor]

I just checked a modified version of  the patch from 2/10 into the tree. I had
to add code to pause and resume the file transport used to asynchronously read
in the post data file to fix some problems with large attachments. We weren't
sending data out faster than the file transport was reading it in.


QA: to test this, first make sure that basic sending works for you =). Varada,
Jean-Francois and I have been running a variant of this code in our trees for a
couple weeks to make sure it was okay but you never know. It was tricky code. 

Try sending yourself some messages with really large attachments. Before, when
you hit the send button, the 3-pane would freeze since the UI thread blocked
until the message was sent. Now you should be able to do other operations and
the UI should be very responsive during the send operation. 

Comment 29

[Sheela Ravindran]

Does this bug verification depend on 65209? Or can I verify this without 65209 
fixed?

Comment 30

[Sheela Ravindran]

verified by sending large files which took a while to send and also made sure 
that it did not block the UI for that duration. Could perform other operations 
while it was in progress.  Cool!
2001050706-win98
2001050209-mac
2001050708-linux

verified on the dial up machine in the lab on buildid: 2001043008 

Comment 31

[Keyser Sose]

*** Bug 79790 has been marked as a duplicate of this bug. ***