Stalls for 30 sec at copying sent mail to IMAP folder on sending

RESOLVED FIXED

Status

MailNews Core
Networking: IMAP
--
major
RESOLVED FIXED
16 years ago
9 years ago

People

(Reporter: Erich Bremer, Assigned: Bienvenu)

Tracking

({fixed1.8, perf})

Trunk
x86
All
fixed1.8, perf
Bug Flags:
blocking-aviary1.0 -
blocking1.8b5 +

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(3 attachments, 1 obsolete attachment)

(Reporter)

Description

16 years ago
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.1b) Gecko/20020821
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.1b) Gecko/20020821

This bug was opened before and then closed.  I just tested the current build and
sending e-mails is very very slow.  My server is Sun One Messaging server 5.2

Reproducible: Always

Steps to Reproduce:
1.Send an e-mail (doesn't matter how big the e-mail is)
2.
3.
Sending emails is always fast for me.  It could be network delays, server delays...

Try using another email client, or better yet telnet to the smtp server and
attempt to send something.  If it's slow, then it's not mozilla.

Comment 2

15 years ago
erich@ebremer.com --

Have you tried the suggestions of bugs@zzxc.net?

Unfortunately, your bug report doesn't contain enough information for us to be
able to fix the bug. Could you please read the Bug Writing Guidelines at 
<http://www.mozilla.org/quality/bug-writing-guidelines.html> to see the
kinds of information we need in a bug report. Please report back with more
information after reading those guidelines?

Thanks for using the guided form -- we just need more information.

Feel free to re-open this bug if you can reproduce the problem on a recent build
and have more details of the situation, or perhaps have narrowed it down to what
might be causing the problem.

-M
Status: UNCONFIRMED → RESOLVED
Last Resolved: 15 years ago
Resolution: --- → INVALID
(Reporter)

Comment 3

15 years ago
This problem occurs with Mozilla 1.1 but does not occur with Netscape
Communicator 4.79.  The problem seems to be with the "copies to sent" portion of
the e-mail transmission.  If the "Place copy to Sent" option is turned off, the
e-mail will go out within 5 seconds.  When the "Place copy to Sent" is turned on
with the copy going to the *IMAP server* selected, the copy portion of the
transmission can take in excess of 30 seconds (and for a small message).  If a
local Sent folder is selected, the sending goes fast (< 5 seconds). EVERYTHING
WORKS FINE THOUGH WITH NETSCAPE 4.79 so I do not think the problem is with the
IMAP server itself which is actually a Dual Pentium III Dell PowerEdge 2500 with
very little load.  The problem only occurs with Netscape 6 & 7 and Mozilla. 
Emptying the 530 messages in my sent folder on the imap server also does not
help.  Just playing around with it now, the copy to sent process will go fast
when you send as plain text only, but goes slow when you say to send as plain
text and HTML at the same time.  Several of my people are having problems with
this.  HELP!
Status: RESOLVED → UNCONFIRMED
Resolution: INVALID → ---
(Reporter)

Comment 4

15 years ago
Ok, I found some additional info.  Sending goes slow when you REPLY to a message
with HTML text and then when you send it as HTML and Plain Text.  If you send as
Plain text only, it works fine.  Goofier, if you FORWARD the message (with
attachement inline) and then send as HTML and Plain Text, it works fine.  There
seems to be some difference with REPLY and FORWARD.  - Erich

Updated

15 years ago
QA Contact: olgam → stephend

Comment 5

15 years ago
This complaint (based on latest comments the problem is in copying to the IMAP
server) seems to be the same as what is described in bug 169608 (or vice-versa).

Updated

15 years ago
Blocks: 193931
Keywords: perf

Updated

15 years ago
Summary: Sending of e-mails (regardless of size) is slow! → Sending of e-mails is slow! (copying sent mail to IMAP folder)

Comment 6

15 years ago
*** Bug 169608 has been marked as a duplicate of this bug. ***

Comment 7

15 years ago
related: bug 123063
Summary: Sending of e-mails is slow! (copying sent mail to IMAP folder) → Stalls for 30 sec at copying sent mail to IMAP folder on sending

Comment 8

15 years ago
*** Bug 166676 has been marked as a duplicate of this bug. ***

Comment 9

15 years ago
Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4a) Gecko/20030326

This has been a long-standing bug in Mozilla 1.x.  It seems that any message
copy from local to an IMAP folder (SSL or non-SSL) takes an extremely
long time to negotiate.  To make matters worse, there is little to no status
information (i.e. "moved/copied message #4 of 20" used to be the default
behavior in Netscape 4.x).

There have been so many dups on this that I think the original bug report is
starting to be diluted!  Half of this bug relates to extreme slowness, the other
half relates to lack of notification to the user as to what Mozilla is doing
while it appears to be "idle".   Bug 80877  Bug 156471  Bug 169608

Comment 10

15 years ago
Confirming on basis of duplicates. Should be addressed in 1.4 beta.

Is this really a Windows-only bug? Maybe it's in our file handling or networking
code?
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: blocking1.4b?
Keywords: nsbeta1
Mail triage: nsbeta1-
Keywords: nsbeta1 → nsbeta1-

Updated

15 years ago
Flags: blocking1.4b? → blocking1.4b-

Comment 12

15 years ago
I have noticed stalling while trying to send mail in at least 1.2 and 1.3 of
Mozilla.  It seems that the mail is never actually sent, and it only occurs when
replying to a previous email.  If I create a new email and send it to the same
recipient it goes just fine.  According to the progress bar, the stall usually
occurs at the end of the send, but I have seen it occur at begiining and middle
of the send.  Sometimes, killing Mozilla completely and restrating it clears the
problem, but not always.  The problem seems to occur randomly.  I have not
noticed a pattern as to return address, time of day, or phase of moon.

--jon

Comment 13

15 years ago
After playing around for a while I am starting to see a pattern (at least for my
specific flavor of this bug).  It seems the problem is related to quoting or
attaching the original text of a previously received email.  When it happens, it
doesn't make any difference whether I hit the reply button and include the
original email text automatically, or create a new message and manually copy the
text into the new email, or include it as an attachment.  I've also noticed that
sometimes if I start shaving off the copied text it sometimes will work.  If I
include none of the original text then it seems to always work (note: I have a
very limited sampling for this problem).

Based on the evidence I suspect that there is some screwy character in the
original text or its formatting (perhaps it contains some bad html) that is
causing Mozilla or SMTP to balk.  In any case, this bug is awfully annoying.

BTW: As with the original bug report, I am using Windows 2000, specifically with
SP3.

--jon 

Comment 14

15 years ago
This will be my last update (promise).

The light bulb finally went on.  I remembered that this problem has happened to
me before.  A couple years ago it got so serious that I called up the ISP,
thinking it was their problem.  They insisted that I create a new outgoing
server entry, even if the info was the same.   I did it and poof, outgoing email
stopped stalling.  Well, after having tried everything else this time, I created
a "new" outgoing server account.  Poof, the problem seems fixed, again.  Note
that the information in the "new" account is exactly the same as the old one,
and installing the latest Mozilla 1.4RC2 did not correct the problem.

--jon  

Comment 15

15 years ago
I'm going to take a shot in the dark and throw this over to Composition. 

Here's why: The bug only happens when replying to an email that contains HTML.
Perhaps composition is doing some I/O that doesn't fly well with IMAP. If it's
not Composition, feel free to shoot it over to Networking: IMAP.

Also, I don't think that this is critical -- no data loss or crash, here.
Downgrading to major, though normal might be more appropriate.

-M
Severity: critical → major
Component: Mail Window Front End → Composition

Comment 16

15 years ago
Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.5a) Gecko/20030619

Local-to-IMAP message copy performance has improved a bit in recent builds.

However, the problem of no status ticker is still the same--when you highlight,
say, 100 messages in your local "Sent" folder and drag them to your IMAP "Sent"
folder, nothing happens for about 5 seconds.  Then the barberpole starts moving
at the bottom for about 30 seconds (which is when the message moves are actually
taking place), then it disappears.  Then about 5 seconds later all the moved
messages finally disappear from the list.

This is a horrible way to inform the user that something useful is actually
happening.  We need the old Netscape 4.x behavior of IMMEDIATE status on the
bottom (i.e "Moving message 3 of 100...").

Comment 17

15 years ago
Re: comment 16

That would be a separate bug, asking for a dialog.

-M

Comment 18

15 years ago
I'm still having the problem and I've noticed that it sometimes happens even on
emails without HTML.  The only somewhat reliable workaround I have for this
problem is 1) create a new SMTP account with exactly the same info as the
previous one 2) set the new account as the default 3) restart mozilla.  Once the
problem occurs on a given email I have never had it go through until I execute
the workaround.  In the meantime, other email replies may or may not go through.  

Also, if others are having this problem then it is a major one, not a minor one.
 I agree that data is not being lost, it is just sitting in my Drafts folder for
a long time until I can shake the chicken bones and utter a chant.  Because the
problem is random and concerns something as fundamental as email, users may
loose confidence in the app's overall reliability.

--jon

Comment 19

15 years ago
The problem still occurs now and then, and can be pretty tenacious at times.  I
have even noticed it occuring on new email, not just replies.  I am beginning to
suspect that it has something to do with the address and perhaps the mood of the
mail server (timing issue?).  In the most recent example the email was quite
short and plain text -- nothing fancy.  I have used the email address many times
before.  
I even tried creating the message again and copying and pasting the text.  The
send mail still stalls.  I can send test emails to myself and others and it goes
through just fine.

BTW: I tried enabling logging but the log captures nothing when the problem occurs. 

This is really frustrating because it is so random.  Can anyone suggest a good
alternative mail app that is compatible with mozilla saved mail and address book?

Comment 20

15 years ago
As further evidence that there is possibly something related to the address, the
message length, and the sever timing (along with the phase of the moon)...

Regarding the short plain text email I created that refused to be sent, I tried
sending a really short message to the very same email address and it went
through.  I tried sending the "bad" message and it balked.  I then removed the
first two sentences and a blank line, and the message went through just fine.

Note: Although nothing shows up in the log when a message balks, the
send/receive lights go back and forth a few times as if the system is trying to
send the email, but can't.

Comment 21

14 years ago
Just wondering what's happening with this bug.  It's still there in both 1.5 and
1.6b, and it's a very annoying bug.

I lose my copy of about 30% of my sent emails every day, the emails go, but no
copy for me :(

Seems to me that there is just no retry or something similar, when it works (the
good 70% of the time), it happens fast. If it doesn't happen (copying to IMAP
sent folder) within 10 seconds, then it never seems to happen, and the timeout
isn't for another 5 or 10 minutes (perhaps longer).  Thus I think it's a retry
issue.  The IMAP server is remote (different ISP). 

Comment 22

14 years ago
Copying the mail to the Sent Items folder doesn't seem to work at all (inc.
v1.6), so I'm using the following workaround:

1. Edit -> Mail & News account settings -> Copies & Folders
2. Uncheck "Place a copy in".
3. Check "BCC these email addresses" and type your own address.

For #3, I use sent@myusername.fastmail.fm which puts the mail directly in my
Sent folder. You'd need a mail service that supports this configuration,
otherwise the mail will end in your inbox.

Prog.

Comment 23

14 years ago
I suspect I could use mozilla to move it to sent using filters as it comes back
into my inbox (from the BCC idea in comment #22), as the other filters work fine
with IMAP folders, but it's a pretty ugly workaround.  By the way, the bug is
marked as Windows 2000 and I'm using Linux, so it's not windows only.

Comment 24

14 years ago
As per comment #23, changing OS to ALL. If anyone encounters this bug on other
platforms (e.g. Mac), please note.

Prog.
OS: Windows 2000 → All

Comment 25

14 years ago
The real downside of the workaround (using BCC to save a copy via your inbox) is
that you loose a copy of all the people that were BCC'ed!  This is a pretty
fatal flaw in the work around I think, I've got to choose between a 30% chance
of loosing my copy of the email all together, or a 100% chance of loosing the
list of all the people I BCC'ed it to.  Not nice!  Does anyone know if
Thunderbird has the same problem?

Comment 26

14 years ago
Just some additional information.  I mainly use Mozilla with IMAP remotely (not
on a LAN).  I just got back on the Lan and tested it to see if this problem
occurs when I am local, and it doesn't!  This doesn't help me, but it might help
someone solve the bug.  If anyone has checked this bug in Thunderbird, can they
please let me know, because I can't find any Thunderbird Linux binaries (!?). 
This bug is very serious for me, if anyone knows of any other GUI IMAP client
that's as good as Mozilla for Linux, could they let me know, because I'm losing
too many emails due to this bug :(

Comment 27

14 years ago
OK, this bug is in Thunderbird too.  I just tested 0.4 for a while, and still
lost a couple of emails, although it didn't seem to lose as many as I'm losing
in 1.6 of Mozilla.  I fear this bug is not being monitored, I emailed the person
it was assigned to a few days ago and have heard nothing back.

Comment 28

14 years ago
I think an ethereal dump of the session might help, here.

sspitzer hasn't said anything on this bug -- let's see if anybody's awake over
in Networking: IMAP. :-)

-M
Assignee: sspitzer → bienvenu
Component: Composition → Networking: IMAP
QA Contact: stephend → grylchan
(Assignee)

Comment 29

14 years ago
An imap protocol log of a failure to copy to the sent folder would be helpful,
Craig. Here are the log instructions:

http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#imap

When connecting remotely, are you going through a firewall or vpn or something,
that might be dropping the connection?

Comment 30

14 years ago
Hi David,

Thanks for the input.  I'm running behind a NAT'ing firewall (Netgear dedicated
box) with no outbound restrictions, and the other end is using port forwarding,
so it's pretty direct.

Anyhow, here is the log, the gap is the long wait before the save gives up.

I don't have any other IMAP troubles.  I have junk auto moved to Junk,
everything works fine.  But a very significant number of sent email saves fail
(30-50%).

98310[8b809a8]: 8b7ff70:mail.ucw.com.au:S-INBOX:ProcessCurrentURL: entering
98310[8b809a8]:
8b7ff70:mail.ucw.com.au:S-INBOX:ProcessCurrentURL:imap://craig@mail.ucw.com.au:143/addmsgflags%3EUID%3E/INBOX%3E80554%3E2:
 = currentUrl
16384[810a860]: 95540e8:mail.ucw.com.au:NA:SetupWithUrl: clearing
IMAP_CONNECTION_IS_OPEN
98310[8b809a8]: 8b7ff70:mail.ucw.com.au:S-INBOX:SendData: 23 uid store 80554
+Flags (\Answered)
213000[9079060]: ImapThreadMainLoop entering [this=95540e8]
213000[9079060]: 95540e8:mail.ucw.com.au:NA:ProcessCurrentURL: entering
213000[9079060]:
95540e8:mail.ucw.com.au:NA:ProcessCurrentURL:imap://craig@mail.ucw.com.au:143/appendmsgfromfile%3E/Sent:
 = currentUrl
98310[8b809a8]: ReadNextLine [stream=8b80c38 nb=50 needmore=0]
98310[8b809a8]: 8b7ff70:mail.ucw.com.au:S-INBOX:CreateNewLineFromSocket: * 1445
FETCH (FLAGS (\Seen \Answered) UID 80554)
98310[8b809a8]: ReadNextLine [stream=8b80c38 nb=27 needmore=0]
98310[8b809a8]: 8b7ff70:mail.ucw.com.au:S-INBOX:CreateNewLineFromSocket: 23 OK
UID STORE completed
98310[8b809a8]: 8b7ff70:mail.ucw.com.au:S-INBOX:SendData: 24 check
98310[8b809a8]: ReadNextLine [stream=8b80c38 nb=28 needmore=0]
98310[8b809a8]: 8b7ff70:mail.ucw.com.au:S-INBOX:CreateNewLineFromSocket: 24 OK
Checkpoint completed





213000[9079060]: ReadNextLine [stream=93291b0 nb=0 needmore=1]

(Assignee)

Comment 31

14 years ago
Craig, what version of what client generated that log? If you're not running the
latest code, there's more logging in 1.7, and the latest nightly thunderbird
builds that might be helpful

Comment 32

14 years ago
OK, I'm now running build 2004020507.  Here is the new log.

98310[8e75ac0]: 8a9e060:mail.ucw.com.au:S-INBOX:ProcessCurrentURL: entering
98310[8e75ac0]:
8a9e060:mail.ucw.com.au:S-INBOX:ProcessCurrentURL:imap://craig@mail.ucw.com.au:143/addmsgflags%3EUID%3E/INBOX%3E80556%3E2:
 = currentUrl
16384[810a840]: 9bfa410:mail.ucw.com.au:NA:SetupWithUrl: clearing
IMAP_CONNECTION_IS_OPEN
98310[8e75ac0]: 8a9e060:mail.ucw.com.au:S-INBOX:SendData: 19 uid store 80556
+Flags (\Answered)
294919[894ae28]: ImapThreadMainLoop entering [this=9bfa410]
294919[894ae28]: 9bfa410:mail.ucw.com.au:NA:ProcessCurrentURL: entering
294919[894ae28]:
9bfa410:mail.ucw.com.au:NA:ProcessCurrentURL:imap://craig@mail.ucw.com.au:143/appendmsgfromfile%3E/Sent:
 = currentUrl
98310[8e75ac0]: ReadNextLine [stream=8e75de0 nb=50 needmore=0]
98310[8e75ac0]: 8a9e060:mail.ucw.com.au:S-INBOX:CreateNewLineFromSocket: * 1447
FETCH (FLAGS (\Seen \Answered) UID 80556)
98310[8e75ac0]: ReadNextLine [stream=8e75de0 nb=27 needmore=0]
98310[8e75ac0]: 8a9e060:mail.ucw.com.au:S-INBOX:CreateNewLineFromSocket: 19 OK
UID STORE completed

<30+ second pause>
<Message "connection to mail.ucw.com.au timed out">

294919[894ae28]: ReadNextLine [stream=894b0b8 nb=0 needmore=1]
294919[894ae28]: 9bfa410:mail.ucw.com.au:NA:CreateNewLineFromSocket: clearing
IMAP_CONNECTION_IS_OPEN - rv = 804b000e
294919[894ae28]: 9bfa410:mail.ucw.com.au:NA:TellThreadToDie: close socket connection
294919[894ae28]: 9bfa410:mail.ucw.com.au:NA:CreateNewLineFromSocket: (null)
294919[894ae28]: 9bfa410:mail.ucw.com.au:NA:ProcessCurrentURL: aborting queued urls
294919[894ae28]: ImapThreadMainLoop leaving [this=9bfa410]
(Assignee)

Comment 33

14 years ago
OK, at first glance, it just looks like the connection to the imap server is
taking a long time to establish. This is happening outside the imap code - I
wonder if something somewhere is limiting the number of connections you can make
to the server. You're right that we have no retry - you should get an error and
get prompted to return to the compose window. Does that not happen?

Comment 34

14 years ago
Yes this does happen, but it's not much use, because the message has already
been sent, so if I click send again, it will go twice.

I suppose this could be a server problem (I'm using university of washington
server, a fairly old one from RH 7.3).
(Assignee)

Comment 35

14 years ago
do you ever get this error when you try to open an imap folder? It seems like it
should be just as likely to get this error opening a folder (that you haven't
previously opened) as it does getting it copying to the sent folder.

Comment 36

14 years ago
Occasionally it hangs opening another folder, not very often, but it does
happen.  This doesn't really cause much of a problem though, I just re-open the
inbox, and then re-open the other folder.  This bug with not saving Sent emails
actually causes lost [copies of] emails, which is very nasty.

Comment 37

14 years ago
Can I get an idea if this bug seems solvable any time soon.  It's ruining my
email experience, and if it can't be solved soon I think I'm going to have to
find another email client (ouch!)  I'd donate some money towards this bug if it
helps, is there any way to do that?
(Assignee)

Comment 38

14 years ago
it's on my list of things to do. My idea is to add an option to try again, when
the first fcc fails. I hope to get this into 1.7b (but not 1.7a) Yes, you can
pledge money by adding a comment to the bug, and redeem your pledge by donating
on www.mozilla.org.

My personal opinion is that something is rather wrong somewhere along the way in
your setup, and that all I can do in Mozilla is do the retry I described above,
but it still going to be rather ugly when it spins for 30 seconds before failing
(that's rather out of my control, how long it takes to fail).

You might try turning on nsSocket logging and see if we get told anything
interesting about why the connection is failing.

on windows, it would be something like:
set NSPR_LOG_MODULES=nsSocketTransport:5
set NSPR_LOG_FILE=c:\socketlog.txt

Comment 39

14 years ago
Hey David,  I'll do a socket log now.  The 30 second thing is a bit of a red
herring I think, I've got one sending dialog up now thats been there for about
10 minutes, maybe more.  Sometimes it fails quite fast (10-20 seconds), usually
it's a minute or more.  If you have any idea's on what could be wrong with my
setup, let me know.  I'm a programmer (Java), and understand tcp/ip networking
pretty well.  Also, I run Linux on both the client and the server, so I can run
whatever tools you like.  Perhaps a TCP proxy would help, to see all the
traffic?  I've got one that I use to debug http stuff.

Also, I've run mtr (Matt's traceroute), and I'm gettting about 2% packet loss to
all sites (my ISP's experiencing a growth spurt and hasn't upgraded capacity yet
:( ).  This doesn't seem anywhere near high enough to explain the 50% failure
rate, but I spose it could be related?

I'll pledge US$100 to this bug.
(Assignee)

Comment 40

14 years ago
Craig, are these packets really lost, i.e., not received by the server? Or do
they get resent? If they're truly lost, IMAP Append is going to spin until you
cancel, because the way Append works is that we tell the server, here comes 1000
bytes, and then we send 1000 bytes, and then we wait until the server tells OK,
got the data. If the server only gets 998 bytes, it will never tell us OK. But
this pre-supposes that we've actually established a connection. From the log you
sent me, we never established a connection. It might be that in some situations
we do establish a connection and start sending data, and then run into a
problem, and spin. In that case, cancelling should work, though again, you
wouldn't be able to get your copy in the sent folder. An IMAP protocol log of
the latter case would also be helpful.

Re more logging, it sounds like you know more about the possibilities there. I
think you can turn on server-side UW imap logging, but I don't know the details
there...
(Assignee)

Comment 41

14 years ago
Created attachment 143239 [details] [diff] [review]
proposed fix

this patch makes it so we re-prompt when an fcc fails, and if the user says OK,
we redo the fcc.
(Assignee)

Updated

14 years ago
Attachment #143239 - Flags: superreview?(mscott)

Comment 42

14 years ago
Thanks David.  Sorry I didn't get back to you, I couldn't get the UW IMAP server
to do any logging.  Seems like all the debugging is only compile time settings
!?!  Anyhow, I couldn't really get the source and recompile the server (too busy
+ old server).  I'll check out your patch when it gets approved, I assume there
will be a post here about it.

Thanks again David.  My client is upgrading their mail server, and I'm putting
Cyrus IMAP server on it, so I'll let this bug know if that fixes the problem
(and if it doesn't, debugging should be easier).

Updated

14 years ago
Attachment #143239 - Flags: superreview?(mscott) → superreview+
(Assignee)

Comment 43

14 years ago
Craig, you can generate a client-side protocol log by following these instructions:

http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#imap

But I'll check in this fix, tonight, I hope, and you can see what happens.

Comment 44

14 years ago
I've had almost this exact problem for a long time (since pre Moz. 1.0).  
Here's what I see, figured you might be interested in more perspective:

IMAP Server: University of Washington IMAP version 4.7.
Mozilla mail clients: Thunderbird 0.6 (20040502) or Mozilla 1.7b
Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7b) Gecko/20040316

I can almost always reproduce this problem, just not very consistently.  I.E., 
I might go a whole day of using TBird or Mozilla mail and not have the sent 
mail problem happen, then the next day it'll happen almost every time I send a 
message.

The behavior I see is this:

1. Compose a message
2. Click Send
3. Send dialog comes up and if I have the problem, it will state one of the 
following:
  "Sending authenticate login information"
or "Copying message to Sent folder" (and the status bar is up to 100%)
4. Either of these messages will stay up with the Compose window behind it for 
who knows how long -- I don't generally wait for them to time out.
5. I generally will eventually get this message if I leave the dialog window up 
(paraphrasing): "Message has been sent, but copy to sent folder faild, Retry?"
6. Instead of waiting, I wait up to 30 seconds, then click the Cancel button, 
and it returns me to the compose window.  At that point, I close the compose 
window and answer "Don't Save" to the prompt.

Here's where this gets interesting:

1. The message is ALWAYS saved to the Sent folder; I don't think I've ever seen 
it not saved except as noted in #3 below.

2. If I attempt to display the contents of my "Sent" folder at this point 
(i.e., select it with a single-click), I get the hourglass mouse pointer, and I 
can see the list of cached messages in my sent folder in the message-list part 
of the screen, and I can select messages that I've previously sent and view 
their contents.  However, I do NOT see the message that I just sent where the 
Copy-to-Sent-Folder failed, or "stalls".  The only way for me to see that 
message is for me to close TBird/Mozilla Mail and reopen.  At that point, I can 
get into the Sent folder; no hourglass; can see the message I just sent.

3. One caveat to #1 is a scenario setup like this:
   a. Send a message; works just fine, everything as expected.
   b. Send another message and run into the problem -- the "Copying to Sent" 
dialog stalls on the screen.
   c. Now, cancel the Send dialog, close the Compose window, answer "Don't 
Save".
   d. Compose another message, click Send.
   e. Without a doubt, this message will stall on the Copying to Sent dialog -- 
every single time; never seen it behave differently.
   f. While this message does get sent to the intended recipient, it DOES NOT 
GET SAVED to the Sent folder.  I.E., if a previous message has stalled on the 
copy to sent, and you haven't restarted TBird/Mozilla mail, the next and any 
subsequent messages you send not only will stall at the copy to sent dialog, 
but also will never make it into the sent mail folder.

4. One way that I can almost ALWAYS reproduce, although there have been times 
it works just fine, is if I attach a file to the message.  I pretty much know 
I'm going to be restarting the mail client if I attach a message -- the 
messages get sent and copied to Sent, attachments and all, but I'll almost 
always have to cancel out of the Copying to Sent dialog, then close and reopen 
the mail client.

Other tidbits:

I've tested this on a multitude of Windows PCs.  Brand new installations of XP 
Pro with all patches etc., brand new installs of either Mozilla or TBird.  I've 
tried it on Windows 2000 with all service packs and patches.  I've tried 
deleting every possible tidbit of either the Mozilla or TBird install 
(including my profile), reinstalling, recreating the profiles etc., and still 
run into the same problem.

I clean up my Sent folder by archiving the messages fairly often (1x per 
month), but just to be sure, I've emptied my Sent folder to make sure it's not 
an issue with the size of the folder, or how many messages are in it etc.

I generally use SSL, but it fails equally with either.

I'm willing to do some kind of IMAP/SMTP logging if need be, just point me in 
the right direction.

Thanks for listening.

Comment 45

14 years ago
I'm pretty sure this is the same bug as bug 123063. And probably also bug 196095.

I noticed that someone recently implemented a change that allows a retry for the
save to the IMAP sent folder. Originally this appeared not to help in my case -
 the retry always failed if the original save failed.

I recently updated to Moz 1.7rc2 (on XP pro) and switched to the fastmail.fm
smtp server. I'm pretty sure the switch to the new smtp server has fixed this
problem for me - no matter where I send from, the faster smtp performance (or
maybe the different use of sockets, as I switched to TLS) seems to mean that the
"save to IMAP sent folder" activity succeeds.  Which seems odd.

The other thing I noticed is that on the odd occasion that my IMAP save fails,
the retry now generally works.

And finally, the IMAP retry seems screwy - if I try to save a message to draft
and it fails, the retry dumps it in the sent folder!

Cheers

Alan



Flags: blocking-aviary1.0?

Comment 46

14 years ago
need to minus for lack of test case and patch.  if we get closer to those before
tbird 1.0 renominate.
Flags: blocking-aviary1.0? → blocking-aviary1.0+

Comment 47

14 years ago
chofmann -- You said minus but gave it a plus.

Comment 48

14 years ago
minus'ing for now pending the log file from Craig. Note the fix in this bug was
already checked in and we think it fixed the problem for most folks. Need
Craig's log to learn more about remaining issues. 
Flags: blocking-aviary1.0+ → blocking-aviary1.0-
Product: MailNews → Core

Comment 49

13 years ago
(In reply to comment #48)
> minus'ing for now pending the log file from Craig.

That was from six months ago, and six months *after* the last comment here from 
Craig.  Presumably the log is not forthcoming.


(In reply to comment #45)
> I'm pretty sure this is the same bug as bug 123063.

A fix for bug 123063 was checked in this weekend; a fix for bug 287658 was 
checked in a month ago.

Alan Hart, volesky@colostate.edu, Craig O'Shannessy: please try a current 
nightly build of Mozilla or TB, whichever you're using -- be sure the build is 
dated May 1st or later before bothering to download, as the nightlies on 
ftp.mozilla.org are not getting regular updates.

Comment 50

13 years ago
Very encouraged to see the recent flurry of fixes forthcoming.

Would the following be an appropriate place to get the nightly build to try?

ftp://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/2005-05-03-05-trunk/

Thanks

Alan
(Assignee)

Comment 51

13 years ago
yes, that's the place. Bear in mind, if the problem is because of your imap
server,the stall won't go away, but the cancel behaviour should be improved.

Comment 52

13 years ago
Sorry I didn't get back with a client log.  I can confirm that it doesn't seem
to be server dependant.  I am running Mozilla 1.8a2 and am having the same
problem with both Cyrus and uw-imapd (losing sent mail, after long hangs).  The
retry behaviour helps, and saves most of the timeouts, but I still get infinate
hangs (e.g. > 10 hour), with no way to make the client retry.  Interestingly
this same behaviour has NEVER happened to me when connecting using a LAN, only
ever using the net.

I'll try to download a nightly in the next couple of days and let you know if
there is any improvement.

Thanks very much,

Craig

Comment 53

13 years ago
OK, I tried a quick test from the nightly whose URL I mentioned above in message 50.

Start Thunderbird - it seems to load all my folders and accounts correctly.

Compose new message, giving it a subject line and an addressee only. Hit Save
button (my drafts folder is on an IMAP server). The activity bar runs back and
forth but the save fails after a few seconds. A dialog appears asking if I want
to retry. I accept. The save succeeds this time but Thunderbird actually dumps
the message in my Sent folder.

I compose a new near-empty message, then send it immediately. Send succeeds,
though smtp seems to take a good few seconds. Save of the sent messge to the
correct IMAP folder proceeds according to plan.

I leave Tbird running for the time it takes to type this report so far - 5-10
minutes, then compose and send a new short message. Successful, though again,
smtp seems to take 20 seconds or so.

Now I leave it running for perhaps half an hour while I load up a game. When I
quit the game and compose a new short message, smtp is slow but successful, it
correctly attempts to reconnect to my IMAP to save the sent message; this then
spins for about a minute or so at the "sending login information" stage until
this times out, at which point I get an alert. When I agree to a retry, the
progress box now reads "Copy failed" throughout the retry. Again, it times out
with an alert. I agree to retry again and this time it appears to write to the
Sent folder successfully, and quickly. The sent folder, however, does not
display the message. At this point I decide that the server and client are no
longer communicating.

I quit thunderbird and reload it. I switch to the sent folder, and there still
seems to be some problem contacting the server, so the message just sent is
still not there. After clicking between a couple of different accounts, I get an
alert: Mail server mailserv.net is not an IMAP4 mail server. I dismiss the
alert. Finally, clicking back to my sent folder generates the message "opening
folder" in the status bar, which usually seems to indicate that the server is
talking to my client again. And hey presto, the sent message is there.

I would say that overall this is better than before but still not really usable
in "IMAP sent folder" mode (BCC for sent messages works much better).
Unfortunately it looks to me as though this bug still exists, though bug 123063
and its dupes may be cured. Thanks for all the effort thus far, David and others.

Comment 54

13 years ago
PS the infinite hang is apparently cured, though I don't think that's what this
bug was originally reporting. The problem in bug 196095 appears to persist too.
David, reading your earlier comments again, perhaps you expected this at this point.

Server details:
* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT
THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready.
Copyright 1998-2004 Double Precision, Inc.  See COPYING for distribution
information.

Running Thunderbird under WinXP Home.

Comment 55

13 years ago
I have to agree with:
[Additional Comments From craig@ucw.com.au  2005-05-03 07:32 PDT]

Craig explains the symptoms exactly.  I hadn't realized something until his
comment about LAN verses the Internet.  I'm a unix admin for a Postfix/Courier
IMAP server.  We are a company of 5 employees, that host mail for ourselves and
about 160 email clients on a beefy Dell PE 1750 server (dual 1.2GHz, 2GB RAM). 
The only IMAP clients are the 5 employees.  Of the 5 employees the following
MUA's are used:
2 x Thunderbird
1 x Outlook 2000
2 x Outlook 2003
We were hosting the email server in our office on a T1 line (155 POP3 clients)
and all 5 employees connected via 100Mb LAN, until three hurricanes whacked
Florida, so we moved the server to the Verio complex.  Since the move ONLY
myself and the other Thunderbird MUA have this problem of copying the sent
message to the sent folder.  It is not 100% reproducible, but seems to happen
80% of the time.  Sometimes I'll restart Thunderbird and it will work for a few
sent messages, then back to the problem.  I cannot pinpoint anything that seems
to give me a concrete understanding about the true nature of the problem.  I
actually agree with the developers about the problem possibly being the IMAP
server or network, but what concerns me is the fact that the other MUA's NEVER
experience the problem?  If the problem is the IMAP server or the network,
shouldn't they have experienced this at least once in the past 10 months?  I
also wanted to mention that this could be related to specific settings on the
IMAP server like connection time outs and other settings to protect DoS attacks
that Thunderbird isn't expecting?  I would like to offer some of the developers
an account on my mail server if it will help troubleshoot the problem, I could
also send the IMAP server logs to compare against the client logs.

Comment 56

13 years ago
Thought this might help from my previous comment:

sshed to my IMAP server and issued this command:
$ telnet localhost imap
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

* OK [CAPABILITY IMAP4rev1 CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT
THREAD=REFERENCES SORT QUOTA AUTH=CRAM-MD5 AUTH=CRAM-SHA1 IDLE STARTTLS]
Courier-IMAP ready.

 Copyright 1998-2003 Double Precision, Inc.  See COPYING for distribution
information.

$ rpm -qi courier-imap
Name        : courier-imap                 Relocations: (not relocateable)
Version     : 2.2.1                             Vendor: (none)
Release     : 1.8.0                         Build Date: Fri 05 Mar 2004 03:59:59
PM EST
Install Date: Fri 05 Mar 2004 04:05:17 PM EST      Build Host: localhost
Group       : Applications/Mail             Source RPM:
courier-imap-2.2.1-1.8.0.src.rpm
Size        : 861000                           License: GPL
Signature   : (none)
Packager    : %{PACKAGER}
Summary     : Courier-IMAP 2.2.1 IMAP server
Description :
Courier-IMAP is an IMAP server for Maildir mailboxes.  This package contains
the standalone version of the IMAP server that's included in the Courier
mail server package.  This package is a standalone version for use with
other mail servers.  Do not install this package if you intend to install the
full Courier mail server.  Install the Courier package instead.

Comment 57

13 years ago
I'm running the following nightly "Gecko/20050503" (1.8a2).  And the problem
SEEMS to be gone.  I can't be sure, because the problem only shows up every now
and then.

I'm working from home tomorrow, so should be sending heaps of emails.  This
should thrash it a bit, but so far it looks good.  Thanks!

Craig

Comment 58

13 years ago
OK, I would say this this bug is fixed for me (e.g. bug 123063 and 287658 which
I seemed to have are fixed).

I haven't lost a mail, or got a hang since I updated to the nightly (on the
5/5/05, three days ago).

Thanks to whoever fixed it!

Comment 59

13 years ago
(In reply to comment #49)
> Alan Hart, volesky@colostate.edu, Craig O'Shannessy: please try a current 
> nightly build of Mozilla or TB, whichever you're using

OK: Alan claimed this (or a similar symptom) was still a problem (comment 53 & 
54); Craig claims it appears to have been cleared up (comment 58); <volesky> 
never responded.  Tony Pecora chimed in and claimed that he sees/saw the symptom 
described by Craig, but neglected to identify which version of TB.

Alan, Tony: still seeing it with *current* (e.g. 1.1a) builds of TB?

xref bug 206408, where a similar problem is getting attention again.

Comment 60

13 years ago
Tested under XP on TB 1.1 alpha 2.

Imap server is still the following:

* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT
THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready.
Copyright 1998-2004 Double Precision, Inc.  See COPYING for distribution
information.

One 5-min test did not immediately replicate the problem. The problem cannot
always be immediately replicated however. Can you give me a couple of days to
try it?

Thanks

Alan

Comment 61

13 years ago
The bug has now been reproduced under 1.1a2 as follows:

1. Compose new message. Enter the contents and addressee but don't send.
2. Go and brush teeth. Electric toothbrush recommended.
3. Return to PC and hit send.
4. "Copy failed"
5. Dialog appears. There was an error copying the message to the Sent folder. retry?
6. 10+ retries (hitting the retry button) did not work.
(Assignee)

Comment 62

13 years ago
Alan, does your imap server limit you to four connections from a given ip
address? Have you tried going into advanced imap server settings and setting the
number of cached connections to 4?

Comment 63

13 years ago
Hi David,

I believe reducing the number of connections from 5 to 4 did improve reliability
greatly for me in earlier versions. I now use 4 cached connections as standard
(including during this test) and the problem still occurs.

I would be happy to make you a logfile later today, probably mid afternoon PDT.
Are these instructions (below) still good for TB logging?
http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#imap

Cheers

Alan

PS if anyone can put the instructions for making an IMAP log up a bit more
obviously on the mozilla.org website, I'd be grateful. Googling for Thunderbird
IMAP log or mozilla IMAP log did not seem to be delivering the goods last night.
(Assignee)

Comment 64

13 years ago
Alan, those instructions are still good. Re protocol logging, yeah, they should
be easier to find. Ultimately, we'd really like to build logging into the
product, so you could turn it on from the UI. TIA for generating a log. You can
e-mail it directly to me, if you like.

Comment 65

13 years ago
I've sent David Bienvenu an IMAP logfile as suggested (rather than carefully
censoring it and then posting here).

Cheers

Alan
(Assignee)

Comment 66

13 years ago
Created attachment 194511 [details] [diff] [review]
fix for problem in Alan's log

in Alan's log, what was happening when trying to run the append to sent folder
url was that all his connections had died - but he had reached his max
connections limit, so we didn't try to spin up a new connection. The fix is
simply to recalculate the number of alive connections before we decide if we
can spin up a new connection or not.
Attachment #194511 - Flags: superreview?(mscott)

Comment 67

13 years ago
Comment on attachment 194511 [details] [diff] [review]
fix for problem in Alan's log

this seems like a good 1.8b4  candidate as we have had others report similar
issues and the risk is very low.
Attachment #194511 - Flags: superreview?(mscott) → superreview+
(Assignee)

Comment 68

13 years ago
Comment on attachment 194511 [details] [diff] [review]
fix for problem in Alan's log

yes, this is a clear candidate for the branch - I'd like to see if it fixes
Alan's problem first on the trunk (I think it will). I'm sure it won't fix
everyone's problems.
Attachment #194511 - Flags: approval1.8b4?
(Assignee)

Comment 69

13 years ago
my recent patch has been checked into the trunk and will show up in tomorrow
morning's nightly trunk build.

Comment 70

13 years ago
please resolve this and get a trunk verification so we can approve for the
branch. thanks.
(Assignee)

Comment 71

13 years ago
Created attachment 194680 [details] [diff] [review]
put up error msg when courier server drops connection

Turns out that Alan's problem was that he had four accounts to the same server,
and it was dropping the fifth connection attempt. This patch restores the
errror message we used to put up in this situation (I think either a necko
change, or switching to blocking reads in IMAP changed the error code we were
getting for this situation, so I added a check for the error we're now getting
from necko).

Next, I need to figure out why we were spinning in this case. But putting up an
alert should help a lot.
Attachment #194680 - Flags: superreview?(mscott)

Comment 72

13 years ago
Comment on attachment 194680 [details] [diff] [review]
put up error msg when courier server drops connection

David, is this the right patch?
(Assignee)

Updated

13 years ago
Attachment #194680 - Attachment is obsolete: true
Attachment #194680 - Flags: superreview?(mscott)
(Assignee)

Comment 73

13 years ago
Created attachment 194702 [details] [diff] [review]
correct fix

wrong tree again...
Attachment #194702 - Flags: superreview?(mscott)
(Assignee)

Comment 74

13 years ago
second fix checked in, for courier problem...
Status: NEW → RESOLVED
Last Resolved: 15 years ago13 years ago
Resolution: --- → FIXED

Updated

13 years ago
Attachment #194702 - Flags: superreview?(mscott) → superreview+

Comment 75

13 years ago
Comment on attachment 194702 [details] [diff] [review]
correct fix

this is a no brainer for the branch but we should still let it sit for a day or
so on the trunk.
Attachment #194702 - Flags: approval1.8b5?

Updated

13 years ago
Flags: blocking1.4b- → blocking1.8b5+

Comment 76

13 years ago
I believe bug 196732 (not all junk labelled as junk is moved) is caused by the
same courier connection limit. At least in my case.

Alan

Updated

13 years ago
Attachment #194702 - Flags: approval1.8b5? → approval1.8b4+

Updated

13 years ago
Attachment #194511 - Flags: approval1.8b4? → approval1.8b4+

Comment 77

13 years ago
both patches are now on the 1.8 branch.
Keywords: fixed1.8
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.