780124 - After "4XX error on greeting from SMTP server" or "error response to SMTP command", mail is saved in Sent folder and composition window is closed, despite that SMTP send of mail fails

Status: RESOLVED WORKSFORME

(MailNews Core :: Networking: SMTP, defect)

Description

[Rob]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:14.0) Gecko/20100101 Firefox/14.0.1
Build ID: 20120713134347

Steps to reproduce:

When I send a mail using an incorrect smtp server (e.g. I have stmp.telenet.be configured but I'm at a customer whose ISP only accepts relay.skynet.be), and the server sends an error message instead of just timing out,


Actual results:

I get an error message (as I used to get), but instead of keeping the composer window open, the composer window is closed and the mail (which has not been sent succesfully) ends up in my SentMail.


Expected results:

Before the latest update, I would get an error message but the composer window would remain open so:
- I would know that sending failed
- I could try again with the correct smtp server

Comment 1

[Rob]

Attachment: An example of an SMTP error message which I refer to

Comment 2

[Luc DISCHERT]

Duplicate of bug 780086 ?

Comment 3

[WADA:World Anti-bad-Duping Agency]

(In reply to Rob from comment #1)
> An example of an SMTP error message which I refer to

"relaying denied" is seen in screen shot.
"relaying denied" is usually 550 or 5.7.1 from SMTP server o Tb, and mail is rejected by SMTP server. See bug 228198 for it.
In that bug, no one referred to phenomenon like "composition window is closed and mail is saved in Sent".
And, there similar bugs were opened consecutively.
  bug 775999, bug 780086, bug 780124(this bug)
Perhaps a regression by a Tb release.
Setting dependency of those bugs for ease of tracking and analysis.

Can you persistently reproduce "relaying denied" from SMTP server?

Comment 4

[WADA:World Anti-bad-Duping Agency]

4-th bug, Bug 780115 for "too many open connections error from SMTP server".
"Save to Sent folder" in such case may be a protection from loss of composing mail in case of abnormal termiation of mail composition window.

Comment 5

[WADA:World Anti-bad-Duping Agency]

Unable to produce problem in Tb 14.0.0 on Win-XP, by 553 to MAIL FROM: from Yahoo! SMTP server(Yahoo! rejects From: address other than assiged mail address).
When error occurred, dialog for the error was opened, and composition window was still kept after close of the dialog.
(SMTP log)
> 00000022	0.71450758	[6064] 0[210f140]: PLAIN auth	
> 00000023	0.71459109	[6064] 0[210f140]: Logging suppressed for this command (it probably contained authentication information)	
> 00000024	0.80845976	[6064] 0[210f140]: SMTP entering state: 0	
> 00000025	0.80857205	[6064] 0[210f140]: SMTP Response: 235 OK, go ahead	
> 00000026	0.80865669	[6064] 0[210f140]: SMTP entering state: 18	
> 00000027	0.80873466	[6064] 0[210f140]: SMTP Login response, code 235	
> 00000028	0.80880642	[6064] 0[210f140]: SMTP entering state: 3	
> 00000029	0.80890781	[6064] 0[210f140]: SMTP Send: MAIL FROM:<z-01@x.x.x> SIZE=350 	
> 00000030	0.91729367	[6064] 0[210f140]: SMTP entering state: 0	
> 00000031	0.91737914	[6064] 0[210f140]: SMTP Response: 553 From address not verified - see http://help.yahoo.com/l/us/yahoo/mail/original/manage/sendfrom-07.html	
> 00000032	0.91746324	[6064] 0[210f140]: SMTP entering state: 5	
> 00000033	6.38465023	[6064] 0[210f140]: SMTP Send: QUIT 	
> 00000034	6.38476753	[6064] 0[210f140]: SMTP entering state: 0	
> 00000035	6.57907629	[6064] 0[210f140]: SMTP entering state: 0	
> 00000036	6.57912779	[6064] 0[210f140]: SMTP Response: 221 Service Closing transmission	
> 00000037	6.57919836	[6064] 0[210f140]: SMTP entering state: 11	
> 00000038	6.58548069	[6064] 0[210f140]: SMTP entering state: 12	
> 00000039	6.58557463	[6064] 0[210f140]: SMTP connection error quitting 80004004, ignoring 	

As seen in log, "error to QUIT" takes 5 seconds. It was probably because I kept dialog open for 5 seconds. Tb looks to issue QUIT after dialog close. And, 221 is normally returned to QUIT after 200 msec from SMTP server.

Can you get SMTP log?  (see bug 402793 comment #28 for getting log)
QUIT sequence is normal when error occurs?

Auto-save may be relivant. Do you enable auto-save? If yes, was auto-save invoked before your send operation?
(Check "Show confirmation dialog when messages are saved" at Copis&Folders)

Comment 6

[WADA:World Anti-bad-Duping Agency]

Problem was reproduced by 553 to MAIL FROM:, with keep error dialog open for long time.
(1) Send, 553 to MAIL FROM: from smtp.mail.yahoo.com
(2) Error dialog is opened. Keep it open
(3) Wait for while, netstat /n /b /o
    Confirm "no connection between Tb and smtp.mail.yahoo.com"
(4) OK at error dialog
    => Mail was saved to Sent folder, and composition window was closed

Log with SET NSPR_LOG_MODULES=timestamp,smtp:5,MsgCopyService:5
> 00000022	0.50886804	[6064] 0[210f140]: PLAIN auth	
> 00000023	0.50891578	[6064] 0[210f140]: Logging suppressed for this command (it probably contained authentication information)	
> 00000024	0.60844296	[6064] 0[210f140]: SMTP entering state: 0	
> 00000025	0.60849607	[6064] 0[210f140]: SMTP Response: 235 OK, go ahead	
> 00000026	0.60854828	[6064] 0[210f140]: SMTP entering state: 18	
> 00000027	0.60859132	[6064] 0[210f140]: SMTP Login response, code 235	
> 00000028	0.60863435	[6064] 0[210f140]: SMTP entering state: 3	
> 00000029	0.60869664	[6064] 0[210f140]: SMTP Send: MAIL FROM:<z-01@x.x.x> SIZE=350 	
> 00000030	0.72535086	[6064] 0[210f140]: SMTP entering state: 0	
> 00000031	0.72543770	[6064] 0[210f140]: SMTP Response: 553 From address not verified - see http://help.yahoo.com/l/us/yahoo/mail/original/manage/sendfrom-07.html	
> 00000032	0.72552711	[6064] 0[210f140]: SMTP entering state: 5	
> 00000033	283.92575073	[6064] 0[210f140]: SMTP Send: QUIT 	
> 00000034	283.92584229	[6064] 0[210f140]: SMTP entering state: 0	
> 00000035	283.92593384	[6064] 0[210f140]: SMTP Send: QUIT 	
> 00000036	283.93383789	[6064] 0[210f140]: request 50b8b40 DoCopy - src  dest mailbox://x.x.x/Sent numItems 0 type=1	
> 00000037	284.00128174	[6064] 0[210f140]: NotifyCompletion - src  dest mailbox://x.x.x/Sent	
> 00000038	284.00137329	[6064] 0[210f140]: request 50b8b40 Clearing OK request - src  dest mailbox://x.x.x/Sent numItems 0 type=1

Confirming.

Comment 10

[Rob]

I have an smtp log:

0[110f140]: SMTP Connecting to: smtp.telenet.be
0[110f140]: SMTP entering state: 0
0[110f140]: SMTP Response: 421 gerard.telenet-ops.be bizsmtp 91.183.175.114 relaying denied
0[110f140]: SMTP entering state: 14
0[110f140]: SMTP Send: QUIT
0[110f140]: SMTP entering state: 0
0[110f140]: SMTP Send: QUIT

Comment 11

[Rob]

Re. auto-save: I don't know, I can't find the menu item you mention (I have a localized version of Thunderbird). Can you post a screenshot?

Comment 12

[Rob]

(In reply to Rob from comment #11)
> Re. auto-save: I don't know, I can't find the menu item you mention (I have
> a localized version of Thunderbird). Can you post a screenshot?

Found it. There's no check mark in the 'Show confirmation dialog' box.

Comment 13

[Francisco Leandro]

I have installed Thunderbird 15 Beta 3 in Windows 7 Professional x86, and the same problem still happens.

Comment 14

[Rob]

Not fixed in TB 15.0 release, and a very, very annoying bug for those of us who need to change smtp servers often when at other customers' sites.

Comment 15

[Ludovic Hirlimann [:Usul]]

Can someone affected follow the instrctions at http://www.rumblingedge.com/2009/02/24/howto-find-regression-windows-through-manual-binary-search/ , so we can figure out what broke this ?

Comment 16

[Rob]

Ludovic, I think the bug must be new since Thunderbird 14 (unless the particular mail server which gives the error messages has changed his behaviour recently).

Comment 19

[WADA:World Anti-bad-Duping Agency]

A relevant change : patch for bug 758878(status-thunderbird14: fixed)
Was problem exposed by the patch?
I can't think so, because the patch merely added following code, and because "SMTP connection error quitting ..." was not logged when this bug occurred although it was logged when this bug didn't occur...
> +  // ignore errors handling the QUIT command so fcc can continue.
> +  if (m_sendDone && NS_FAILED(aStatus))
> +  {
> +    PR_LOG(SMTPLogModule, PR_LOG_ALWAYS,
> +     ("SMTP connection error quitting %lx, ignoring ", aStatus));
> +    aStatus = NS_OK;
> +  }

Comment 20

[WADA:World Anti-bad-Duping Agency]

nsSmtpProtocol.cpp is only module changed by bug 758878, and file revisions is following.
> http://hg.mozilla.org/comm-central/log/45ccb6afce84/mailnews/compose/src/nsSmtpProtocol.cpp
Affected by change such as "Switch comm-central from using nsnull to nullptr", "Change types to nsresult", "Make more nsSmtpProtocol methods return nsresult"?

Comment 21

[WADA:World Anti-bad-Duping Agency]

Regression window: (checked by thunderbird-14.0a1.en-US.win32.zip)
(a) Good : 
    /pub/mozilla.org/thunderbird/nightly/2012/04/2012-03-19-03-00-43-comm-central
(b) Bad : 
    /pub/mozilla.org/thunderbird/nightly/2012/04/2012-03-20-03-01-27-comm-central

Changes pushed after 2012-04-19 00:00:00, before 2012-04-20 06:00:00
> http://hg.mozilla.org/comm-central/pushloghtml?startdate=2012-04-19+00%3A00&enddate=2012-04-20+06%3A00

Comment 26

[WADA:World Anti-bad-Duping Agency]

Correct Regression window: (checked by thunderbird-14.0a1.en-US.win32.zip)
(a) Good : 
    /pub/mozilla.org/thunderbird/nightly/2012/03/2012-03-19-03-00-43-comm-central
(b) Bad : 
    /pub/mozilla.org/thunderbird/nightly/2012/03/2012-03-20-03-01-27-comm-central

Correct Changes pushed after 2012-03-19 00:00:00, before 2012-03-20 06:00:00.
> http://hg.mozilla.org/comm-central/pushloghtml?startdate=2012-03-19+00%3A00&enddate=2012-03-20+06%3A00

Sorry for mistake.

Regression by patch for bug 554044(Target Milestone: Thunderbird 14.0)?

Comment 27

[WADA:World Anti-bad-Duping Agency]

To see "Copy to Sent" and "Socket close" by NSPR log, use following parameter.
  NSPR_LOG_MODULES=timestamp,smtp:5,MsgCopyService:5,nsSocketTransport:5
No connection between Tb and SMTP server can be observed by "netstat /n /b /o". It usually has "CLOSE_WAIT" status intead of "ESTABLISHED" after "Socket close" by Tb.

Comment 28

[WADA:World Anti-bad-Duping Agency]

cc-ing to Mark Banner who is ownwer of bug 554044.

Comment 29

[Rob]

Happy to see something move here. Sorry that I couldn't find the time to do the regression test myself.

Comment 30

[Rob]

If necessary, I can test nightly builds to see if this is solved. Just let us know.

Comment 31

[rheras]

I frequently receive this error in my laptop at work when I send an email.

However, at home: the same Exchange account (through IMAP), same operating system (Windows 7 x64) and same Seamonkey version 2.13, but this error doesn't happen here at home.

I don't understand the reason, what's happening it in my laptop at work and not in my pc at home, maybe the intranet environment?

Comment 35

[jay]

Smtp Yahoo: resources temporarily unavailable

Thunderbird moves to sent folder as if everything is fine.  Message did not go through.

Comment 36

[costelc]

I confirm I have this problem and is VERY annoying. Thunderbird 17.0.3, Yahoo smtp smtp.mail.yahoo.com port 465 SSL/TSL Normal password, but with a weird scenario: I get this "The mail server responded:  Resources temporarily unavailable" error only when trying to send to a yahoo mail. If I send to other servers it goes fine, if I send to yahoo mail accounts it fails every time with this error, message going into Sent folder but doesn't get actually sent.

Comment 37

[tataia]

I cna also confirm this behavior of thunderbird and I have found what is causing it.
I'm using qmail mail server and the message is copied to sent folder on error when the mail server responds with a 550 error and immediately close smtp connection.

This occurs when SMTP550DISCONNECT is activated in tcpserver.

SMTP550DISCONNECT

 Disconnect the SMTP session if a 5xx is produced by the sender
 Default: off
 Affects: qmail-smtpd
 Example: "" (any value will do)
 Note: This is useful if you have a spammer trying different senders or
       recipients in the same session separated by rset's. Be aware of
       the fact that this option "breaks" RFC 2821 and may cause problems
       with legitimate SMTP traffic.

When is not activated, Thunderbird acts as expected on such errors. I'm using 17ESR and I can confirm that 10ESR worked ok.

As far as I know yahoo also is using qmail

Comment 38

[Rob]

How can you tell which mail server is being used?

Anyway, since many smtp servers seem to use this or similar functionality, maybe it's worth programming around it in Thunderbird?

Comment 39

[tataia]

http://qmail.omnis.ch/top.html

A number of large Internet sites are using qmail: USA.net's outgoing email, Address.com, Rediffmail.com, Colonize.com, Yahoo! mail, Network Solutions,...

And I remember I received a message from yahoo's mailer-daemon saying 
..this is qmail-send at ...

I've been using TB from version 1 to 10esr with no issues with SMTP550DISCONNECT. Until now.
I strongly believe that Thunderbird guys must correct this, better said to revert to what was before.

Comment 42

[WADA:World Anti-bad-Duping Agency]

By bug 554044, which is a change seen in regression window, parameter was added to SendQuit().
> -PRInt32 nsSmtpProtocol::SendQuit()
> +PRInt32 nsSmtpProtocol::SendQuit(SmtpState aNextStateAfterResponse)
And, m_nextStateAfterResponse value was changed from solid SMTP_DONE to parameter value of aNextStateAfterResponse.
>    m_nextState = SMTP_RESPONSE;
> -  m_nextStateAfterResponse = SMTP_DONE;
> -  return(0);
> +  m_nextStateAfterResponse = aNextStateAfterResponse;
> +
> +  return SendData("QUIT" CRLF); // send a quit command to close the connection with the server.

However, not all call of SendQuit() is not changed to SendQuit(NextStateAfterResponse).
http://mxr.mozilla.org/comm-central/source/mailnews/compose/src/nsSmtpProtocol.cpp#1675
> 1675 nsresult nsSmtpProtocol::SendMessageResponse()
> 1689   return SendQuit();
> http://mxr.mozilla.org/comm-central/source/mailnews/compose/src/nsSmtpProtocol.cpp#562
> 562 nsresult nsSmtpProtocol::SendHeloResponse(nsIInputStream * inputStream, uint32_t length)
> 587     return SendQuit();

Is default of aNextStateAfterResponse SMTP_DONE?
Is this relevant to this bug's problem?

Comment 44

[Rob]

Oh boy, this bug is driving me crazy. I will be forever endebted to the one who solves it. Come to Belgium and I'll buy you beers until you drown. (sorry for spamming the list)

Comment 45

[Radim Kolar]

If SMTP server returns 4XX error on greeting (throttle limit) then Thunderbird still considers mail sent okay and moves it to Sent Mail folder.

Its really very annoying bug.

Comment 46

[WADA:World Anti-bad-Duping Agency]

As already written in comment #37, one of major causes is intentional RFC 2821 violation at SMTP server side. Such option means that the SMTP server considers "a login failure of ordinal SMTP client due to old password" as an "attack by spammer".

Because many ordinal/normal SMTP servers accept From: mail address of other provider, and because Tb's SMTP setting is independent from mail account setting, simplest/easiest available workaround is "use ordinal/normal SMTP server in your daily mail sending", "stop using SMTP server who considers you spammer when old password is used".
Gmail SMTP is an such ordinal/normal SMTP server, and Gmail provides free account anyone can use. So, "use Gmail's SMTP" can be a workaround.
  - Get free Gmail account.
  - Register your From: mail address as "mail address which can be used
    to send mail" at Gmail's setting.
  - Add Gmail's SMTP server to Tb's SMTP server definitios,
    Select Gmail's SMTP server at each Tb's Identity setting,
    Or set Gmail's SMTP server as "deafault SMTP" and use "always use
    defailt (SMTP) server" option for each Identity.
  - View sent mail copy via Web mail,
    or enable Gmail IMAP and define Gmail IMAP account in Tb.
Note: Yahoo!'s SMTP can not be called "normal/ordinal" in this context even thouh Yahoo! doesn't kill connectin immediately, because Yahoo! forces Yahoo!'s mail address in From:, as seen in commemt #6.

Comment 47

[Michael]

1) I can also confirm this bug -- it appears with a timeout or a typo in the password.  And as the reversion test indicated, older versions of thunderbird did not have this behavior.

2) re: WADA -- the gmail workaround is fine for people who need a single account, but those of us with multiple accounts who cannot change their account (e.g. a corporate or university account that is for work), the workaround is not a viable solution.

This bug hits me almost daily, and is quite annoying (and means I've missed responding to important emails as I have no way of knowing which got sent).  

In any case, regardless of what value is returned by the SMTP server, Thunderbird clearly knows that it hasn't sent the mail correctly; it gives an error message saying as much.  Why not change the logic of 'save to send/maintain open; save to draft' to occur concurrently with the dialog popup of the captured error rather than basing it on an error code that many servers to which many servers clearly do not conform.

Thanks for the help!

Comment 48

[WADA:World Anti-bad-Duping Agency]

(In reply to Michael from comment #47)
> Why not change the logic of 'save to send/maintain open;
> save to draft' to occur concurrently with the dialog popup of the captured error
> rather than basing it on an error code that many servers to which many servers clearly do not conform.

Current behavior of Tb is never by design, as seen in older Tb's behavior. It's different from design, and it's simply a bug of Tb, which is perhaps regression by unknown change.
Please note that adding complaints to bug of bugzilla.mozila.org won't help resolving Tb's bug by developer. As I wrote in comment #26, a regresion window in the past is already found. If you want to help developers, please add comment like next, instead of merely adding complaints. (Note: Here is bugzilla.mozilla.org.)
- report of duplication test result to know the regression widow
  written in comment #26 is correct or wrong.
- trace data such as Socket log which may help problem analysis
  by developer.

By the way, another workaround is available if this bug is so critical for you.
  - Send mail via Web mail using Browser.
Note: Even if it works well technically, it may be violaton of law in your organization.

Comment 49

[Michael]

My apologies, I appreciate the hard work of the developers and people trying to chase down any and all bugs.  I definitely didn't mean to be merely complaining with nothing positive to add.

(In reply to WADA from comment #48)
> window in the past is already found. If you want to help developers, please
> add comment like next, instead of merely adding complaints. (Note: Here is
> bugzilla.mozilla.org.)
> - report of duplication test result to know the regression widow
>   written in comment #26 is correct or wrong.
> - trace data such as Socket log which may help problem analysis
>   by developer.

Comment 50

[viric]

There is a lot of text about the SMTP server implementation. But this problem happens when the very SMTP server is *unreachable* (as with temporary network problems).

Someone mentioned that it happens "if the connection is closed before the error dialog is acknowledgedby the user". This is also the case when the server is *unreachable* (connection refused, ip unreachable, or whose DNS name can't be resolved, etc.).

I'm surprised how this hasn't blocked any release from 14 to 17!

Comment 51

[WADA:World Anti-bad-Duping Agency]

(In reply to viric from comment #50)
> This is also the case when the server is *unreachable* (connection refused, ip unreachable, or whose DNS name can't be resolved, etc.).

If problem occurs even in first step of connection, step of "DNS name can't be resolved", and if it's consistently reproducible, can you attach NSPR log?
See bug 402793 comment #28. Following parameter is perhaps useful for first log check.
> Win example :
> SET NSPR_LOG_MODULES=timestamp,smtp:5,nsHostResolver:5,,MsgCopyService:5
If timing of "connection failure or loss" is important, add 
",nsSocketTransport:5" and check socket level flow, please.

Comment 52

[viric]

hm I tried to reproduce it in linux, and I couldn't; I've seen the issue in Windows (coworker's computer), and I could reproduce it simply unplugging the ethernet cable and trying to send a letter. By this latter, I wrote as if it were trivial to reproduce.

I'll try again tomorrow specifically on the Windows machine, and grab the log if possible.

Comment 54

[WADA:World Anti-bad-Duping Agency]

(In reply to viric from comment #50)
> I'm surprised how this hasn't blocked any release from 14 to 17!

This bug was found in Tb 14 after release of Tb 14 and after released Tb 14 was used by many many users.
- We can't withdraw already released Tb 14 without finding cause and
  solution of this bug.
- Even newer release of Tb is shipped with this bug,
  nothing worse than this bug won't occur in any newer release.
  As for this bug, it's absolutely same as Tb 14, Tb 15, ..., Tb x
  which was already released and used for long time.

What is reason to block release of Tb(x+1), Tb 1(x+2), ... , Tb (x+n) because of existent of this bug, even though Tb 14 to Tb x was already released and many many users who never experience this bug?
What is reason to add such comment to bug of bugzilla.mozilla.org long the way? 
Do you say "Mozilla messaging company should have released Tb 13 as Tb 15 again due to this bug" and "Mozilla messaging company shouldn't have released newer Tb releases due to this bug"?

Comment 55

[WADA:World Anti-bad-Duping Agency]

(In reply to viric from comment #52)
> I'll try again tomorrow specifically on the Windows machine,
> and grab the log if possible.

If you get NSPR log, try following parameter.
> SET NSPR_LOG_MODULES=smtp:5,nsHostResolver:5,MsgCopyService:5,DOMLeak:5,DocumentLeak:5,nsDocShellLeak:5
"Dialog open/close" is also logged by DocumentLeak:5, so "when error dialog is opened and closed" is also seen in log.

Comment 56

[WADA:World Anti-bad-Duping Agency]

To Mark Banner who is ownwer of bug 554044:
Do you have answer to my question of comment #42?

Comment 57

[viric]

Attachment: Reproduced on 17.0.6 windows, disconnecting the ethernet cable

I started 17.0.6 with the log flags proposed, went to write a letter, disconnected the ethernet cable, and clicked "Send".

After a while trying to connect to the smtp server, an error message appear, the composer window closed, and the message got stored into the Sent folder.

Comment 58

[viric]

(In reply to WADA from comment #54)
Sorry for the confusion; my comment on how this issue wasn't a blocker was based on my experience in Windows, that I could reproduce this issue very easily all the time.

I didn't understand that it was any hard to reproduce, then.

Comment 59

[WADA:World Anti-bad-Duping Agency]

(In reply to viric from comment #57)
> Created attachment 753133 [details]
> Reproduced on 17.0.6 windows, disconnecting the ethernet cable

Log.
> 0[120f140]: SMTP Connecting to: mail.aqsense.com
> 9056[120f530]: Resolving host [mail.aqsense.com].
> 9056[120f530]: Using cached record for host [mail.aqsense.com].
> 9056[120f530]: Checking blacklist for host [mail.aqsense.com], host record [e1acbf0].
> 0[120f140]: SMTP entering state: 0
> 0[120f140]: SMTP Response: 421 Cannot connect to SMTP server 212.49.179.72 (212.49.179.72:25), NB connect error 1460
> 0[120f140]: SMTP entering state: 14
> 0[120f140]: SMTP Send: QUIT
> 0[120f140]: SMTP entering state: 0
> 0[120f140]: request 4b51940 DoCopy - src  dest mailbox://[e-mail]/Sent numItems 0 type=1
> 0[120f140]: NotifyCompletion - src  dest mailbox://[e-mail]/Sent
> 0[120f140]: request 4b51940 Clearing OK request - src  dest mailbox://[e-mail]/Sent numItems 0 type=1

For Tb, following Greeting is returned from "your SMTP server, mail.aqsense.com, 212.49.179.72:25), after normal connection establishment with your SMTP server.
  SMTP Response: 421 Cannot connect to SMTP server
This case is already reported by comment #45. 

Do you use proxy server for SMTP?

(In reply to viric from comment #58)
> I didn't understand that it was any hard to reproduce, then.

As known by commeny #6, reproducing of "error response to SMTP command" case is pretty easy, because Yahoo! IMAP always rejects From: mail address other than Yahoo!'s one.
However, "4XX error on greeting" case is hard to reproduce in ordinal environment.

Comment 60

[viric]

(In reply to WADA from comment #59)
> Do you use proxy server for SMTP?

No, I don't. But I understood your suspicion, and dig into this thing; I suspected it could be the fancy antivirus.

We are using Avast here. After tweaking all options, I tracked down this to the options "Anti-SPAM" and "Mail protection: Check SMTP connections". If I disable both (one of them is not enough), thunderbird behaves normally.

So, here Avast (and maybe other AVs) get into the SMTP connection causing this behaviour. No wonder that I couldn't reproduce this in Linux. :)

Comment 61

[WADA:World Anti-bad-Duping Agency]

Because no timestamp in your log, it's hard to know "time between events". However, because "44x error response greeting" case, it's simplest case.
It looks next.
See following for SMTP state in log, and Tb's action when a SMTP states.
> http://mxr.mozilla.org/comm-central/source/mailnews/compose/src/nsSmtpProtocol.h#21  
> http://mxr.mozilla.org/comm-central/source/mailnews/compose/src/nsSmtpProtocol.cpp#1861

(1) Greeting is error response.
    Because similar to "too many connections" or "server too busy",
    server uually kills connection immediately after this kind of
    response. 
    In your case, because Avast! is used, Avast! perhaps acts as if
    SMTP proxy to do virus check of outgoing message. Because actual
    server connection is impossible, "connection with SMTP server for
    Tb" is lost just after this response.
> 0[120f140]: SMTP Response: 421 Cannot connect to SMTP server 212.49.179.72 (212.49.179.72:25), NB connect error 1460
(2) Tb goes to 14 = SMTP_EXTN_LOGIN_RESPONSE (same as login failure)
> 0[120f140]: SMTP entering state: 14
(3) Because it's not "actual login failure", error dialog is simply shown.
> 0[120f140]: DOCUMENT 4b48800 created
> 0[120f140]: DOCUMENT 4b48800 ResetToURI about:blank
> 0[120f140]: DOCUMENT 6048800 created
(4) By OK reply, error dialog is closed.
> 0[120f140]: DOCUMENT 4b48800 destroyed
(5) As a step of "state: 14", "send of QUIT" is reuested,
   and waits for response to QUIT.
> 0[120f140]: SMTP Send: QUIT
> 0[120f140]: SMTP entering state: 0
(5) However, connection is already lost,　so "response to QUIT from
    server" never occurs. And, SMTP_PAUSE_FOR_READ perhaps terminates
    because connection is already lost.
(6) Because it's not expecting "Error in SMTP send process => QUIT
    => OK to QUIT => End of SMTP process" status,
    Tb wrongly considers "Normal end of SMTP send".
    Then "Save in Sent" is invoked, and composition windw is closed.
> 0[120f140]: request 4b51940 DoCopy - src  dest mailbox://[e-mail]/Sent numItems 0 type=1
> 0[120f140]: NotifyCompletion - src  dest mailbox://[e-mail]/Sent
> 0[120f140]: request 4b51940 Clearing OK request - src  dest mailbox://[e-mail]/Sent numItems 0 type=1

We now know an prety easy way to reproduce "4XX error on greeting" case in prety ordinal environment :
  Simply utilize "Outgoing mail scanner" of anti-virus software
  such as Avast!, and inrtefere virus scanner's SMTP server connection.

Comment 62

[jonwestburne]

Comment 61 accords with my experience
I use Avast
Sending progress bar on for prolonged time.
When it closes message goes to sent folder and error message shows for a short time.
If sending cancelled before the progress bar closes there is no problem.
Problem happens different times with different servers but only when I am in a location with a less reliable internet connection.

Comment 64

[Mark Banner (:standard8)]

(In reply to WADA from comment #42)
> Is default of aNextStateAfterResponse SMTP_DONE?

Yes, see the .h change in attachment 607175 [details] [diff] [review]

> Is this relevant to this bug's problem?

Without in-depth re-investigation to get all the context back in my head, I can't say. It is conceivable that it could be. Trying a build from before then may help if you're looking for a regression range.

Comment 66

[WADA:World Anti-bad-Duping Agency]

(In reply to Mark Banner (:standard8) from comment #64)
>  Trying a build from before then may help if you're looking for a regression range.

See comment #26 for regression window, please.
Do you see other change than bug 554044 which may be relevant to this bug in check-ins during the regression window?

Comment 67

[[e-mail]]

NOT using Avast and still having the same problem, for almost a year now. With good or poor internet connection there is no difference. Irregularly the message fails to send but ends up in the sent box anyhow.  Using System Mechanic Professional System Shield.

The only way to make sure that the message actually was sent is to keep watching the screen when the SEND button is pressed.

Comment 68

[Peter Ashford]

I always BCC my emails to myself now.  If they come back to me I know that sending actually succeeded.

Comment 69

[WADA:World Anti-bad-Duping Agency]

Workarounds of this bug:
(a) If you experienced this bug, do Comment #68 : BCC: to yourself always.
(b) If you experienced this bug with a SMTP server several times,
    never use unreliable mailer named Thunderbird
    with the problematic SMTP server.
(b-1) Use other reliable mailer when you mail send.
      There is no need to use unreliable mailer in your mail send.
(b-2) Use other SMTP server for used Identity(From: address),
      if and only if mail send using Tb is mandatory for you.

Comment 71

[viric]

WADA, I'm not sure I understood your comments in #61.

You write "error dialog is simply shown", and "By OK reply, error dialog is closed". In my situation, there is no visible error dialog, or any user clicking an OK button.

Thunderbird shows the SMTP sending dialog (with a bar), and then it disappears and the outgoing letter goes to the Sent folder. From the user, there is no difference from a properly sent letter.

Comment 72

[viric]

I would like to understand the status of the issue. Specifically:

- Is there any developer considering that the bug exists in Thunderbird and should be fixed?
- Is there any developer working on it?
- What is required next, to get a fix?
- 

In my opinion, the bug exists, it has very important consequences: the SMTP conversation clearly ended in *will not send the letter*, and thunderbird tells the user *letter sent*. Solutions of the kind "fix your server" do not look any appropiate.

WADA solutions of "use BCC" or "stop using thunderbird" look to me either as temporary solutions, or jokes. But it's broken since TB 10 or 14, so I think I don't understand how to proceed next to help in the solution.

Comment 73

[WADA:World Anti-bad-Duping Agency]

(In reply to viric from comment #71)
> WADA, I'm not sure I understood your comments in #61.
> You write "error dialog is simply shown", and "By OK reply, error dialog is
> closed". In my situation, there is no visible error dialog, or any user
> clicking an OK button.

It depends on "at which step error is returned" and "when SMTP server closes conection after error response to Tb.
Please read SMTP log in comments in #6 well.

Because Yahoo! SMTP server doesn't kill connection immediately after "SMTP Response: 553 From address not verified", dialog of "SMTP send error" is normally shown as expected.
If dialog is closed within short period, Tb coorectly process send error, and composition window is kept open due to SMTP send error(this is coment #5).
Comment #6 is to reproduce this bug.
If dialog of "SMTP send error" is keopt open for long time, Yahoo! SMTP server or Tb closes connection. If connection is lost while dialog of "SMTP send error" is kept open, this bug is reprduced.

If SMTP server closes connection immediately after "error response like 553" or "4xx greeting", "SMTP send error dialog" is perhaps not shown by Tb.
Have you gotton NSPR log with timestamp,smtp:5,MsgCopyService:5 and checked log content by Text Editor?

Comment 74

[viric]

I think that the other end of tcp closes the connection just after giving a "4xx greeting". I imagine that's how the antivirus proxy notifies the failing connection.

I'll have access to the machine of the user with Avast next week, for the log.

Comment 75

[viric]

Attachment: The logs with the issue, with timestamp

I finally could reproduce the issue; I had to reenable the SNMP check in Avast, and reboot the Windows OS to get it effective. Avast didn't tell me that it wasn't effective, but I saw that the analysis counter didn't increase.

Comment 76

[Rob]

For those with difficulty reproducing this with antivirus proxies, I think it's rather easier to reproduce when you're at an ISP which doesn't allow sending from different e-mail addresses.

Here in Belgium, there's Belgacom Skynet (the biggest ISP) who gives such an error message when you try to send using the smtp server of another service provider (we have to use relay.skynet.be and the bug happens if we use, e.g., smtp.telenet.be)

Comment 77

[viric]

Oops, now I see that I wrote SNMP where I meant SMTP, in #75. Notice that the attachment has two letter sends: one succesful, and at the end, one unsuccesful. No dialog appears telling about any error; the sending bar dialog simply stays for a while, until it disappears and the letter goes to the Sent box, as if all was successful.

Comment 78

[viric]

Is there anything else I can do to help solve this issue?

Comment 79

[:Irving Reid (No longer working on Firefox)]

The most valuable next step for this bug would be if someone can build a Mozmill test case (https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Thunderbird_MozMill_Testing) that uses the SMTP fake server (https://developer.mozilla.org/en-US/docs/MailNews_fakeserver) to reproduce the bug.

Comment 81

[WADA:World Anti-bad-Duping Agency]

(In reply to :Irving Reid from comment #79)
> The most valuable next step for this bug would be if someone can build a Mozmill test case
As written in comment #6, this bug is 100% reproducible with pretty simple test procedure, with pretty popular Yahoo! SMTP server. i.e. Any developer can see this bug by his hand.
"Mozmill test case" is step after code analysys/patch implementation, isn't it?

Comment 82

[david]

Hello,

Version 31.1.0 and still that problem. Send e-mail, get 421 smtp message to many concurrent... e-mail not sent and moved to sent folder. Any news???

thanks

David

Comment 84

[Rob]

Still exists in version 31.4.0

Comment 85

[Daniel Kahn Gillmor]

Still seeing this error in icedove (debian's unbranded version of thunderbird) version 36.0~b1-2.  log info (gathered with `NSPR_LOG_MODULES=smtp:5,timestamp`):

> 2015-04-09 16:23:41.422817 UTC - -313727168[7f53ec0445c0]: SMTP entering state: 16
> 2015-04-09 16:23:41.422848 UTC - -313727168[7f53ec0445c0]: SMTP AuthLoginStep1() for [e-mail]@P½êS
> 2015-04-09 16:23:41.422873 UTC - -313727168[7f53ec0445c0]: LOGIN auth
> 2015-04-09 16:23:41.422888 UTC - -313727168[7f53ec0445c0]: Logging suppressed for this command (it probably contained authentication information)
> 2015-04-09 16:23:41.477716 UTC - -313727168[7f53ec0445c0]: SMTP entering state: 0
> 2015-04-09 16:23:41.477770 UTC - -313727168[7f53ec0445c0]: SMTP Response: 334 UGFzc3dvcmQ6
> 2015-04-09 16:23:41.477799 UTC - -313727168[7f53ec0445c0]: SMTP entering state: 18
> 2015-04-09 16:23:41.477815 UTC - -313727168[7f53ec0445c0]: SMTP Login response, code 334
> 2015-04-09 16:23:41.477828 UTC - -313727168[7f53ec0445c0]: SMTP entering state: 17
> 2015-04-09 16:23:41.477844 UTC - -313727168[7f53ec0445c0]: SMTP AuthLoginStep2
> 2015-04-09 16:23:41.477855 UTC - -313727168[7f53ec0445c0]: PLAIN/LOGIN auth, step 2
> 2015-04-09 16:23:41.477871 UTC - -313727168[7f53ec0445c0]: Logging suppressed for this command (it probably contained authentication information)
> 2015-04-09 16:23:42.385595 UTC - -313727168[7f53ec0445c0]: SMTP entering state: 0
> 2015-04-09 16:23:42.385641 UTC - -313727168[7f53ec0445c0]: SMTP Response: 421 4.3.2 Service not active
> 2015-04-09 16:23:42.385662 UTC - -313727168[7f53ec0445c0]: SMTP entering state: 18
> 2015-04-09 16:23:42.385670 UTC - -313727168[7f53ec0445c0]: SMTP Login response, code 421
> 2015-04-09 16:23:42.385678 UTC - -313727168[7f53ec0445c0]: marking auth method 0x100 failed
> 2015-04-09 16:23:42.385685 UTC - -313727168[7f53ec0445c0]: SMTP auth: server caps 0x10114, pref 0x300, failed 0x100, avail caps 0x0
> 2015-04-09 16:23:42.385693 UTC - -313727168[7f53ec0445c0]: (GSSAPI = 0x800, CRAM = 0x2000, NTLM = 0x4000, MSN =  0x8000, PLAIN = 0x200, LOGIN = 0x100, EXTERNAL = 0x400)
> 2015-04-09 16:23:42.385701 UTC - -313727168[7f53ec0445c0]: no auth method remaining
> 2015-04-09 16:23:42.385707 UTC - -313727168[7f53ec0445c0]: SMTP: ask user what to do (after login failed): new password, retry or cancel
> 2015-04-09 16:25:20.023784 UTC - -313727168[7f53ec0445c0]: cancel button pressed
> 2015-04-09 16:25:20.023821 UTC - -313727168[7f53ec0445c0]: SMTP Send: QUIT
> 2015-04-09 16:25:20.023839 UTC - -313727168[7f53ec0445c0]: SMTP entering state: 0

After i press the "cancel" button, the mail composition window disappears, and the message gets placed in the Sent mail folder.

Comment 86

[Daniel Kahn Gillmor]

I note that the server's response of "421 4.3.2 Service not active", according to https://tools.ietf.org/html/rfc3463 is:

 * 4.XXX.XXX   Persistent Transient Failure
 * X.3.XXX Mail System Status
 * X.3.2   System not accepting network messages

         The host on which the mailbox is resident is not accepting
         messages.  Examples of such conditions include an immanent
         shutdown, excessive load, or system maintenance.  This is
         useful for both permanent and persistent transient errors.

Comment 87

[David Anderson]

Confirming the existence of this issue on Thunderbird 31.5.0 (Linux). This issue has been irking me for years! The two situations I see it in is when my SMTP server (exim) returns a 4xx due to excessive load or excessive SMTP connections. TB shows the server's error message, but files the message as sent and closes the message window.

I've also seen other uses of TB who had this, and assumed that their message was sent (because of it being filed in the 'sent' folder), and who have then asked me why their email didn't arrive at its destination.

Comment 88

[viric]

After more than one year waiting for a fix, we moved the Windows users of the office to Outlook because of this bug. It caused too many troubles in the office to be acceptable.

Comment 89

[Wayne Mery (:wsmwk)]

It seems like there is plenty of data here that this could be fixed by someone (comment 42, comment 46, etc).

wada, my reading of the comments is the cause is bug 554044. If this is incorrect please give us an update. THanks.

Comment 90

[Rob]

I also don't understand why the status of this bug is still 'New'. I reported this 3 years ago. 

I know, we should solve it ourselves if we're so bugged by it, but I can't do it. If I could I would have.

Comment 92

[Ludovic Hirlimann [:Usul]]

Removing myslef on all the bugs I'm cced on. Please NI me if you need something on MailNews Core bugs from me.

Comment 93

[Magnus Melin [:mkmelin]]

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #89)
> It seems like there is plenty of data here that this could be fixed by
> someone (comment 42, comment 46, etc).

Hard to say, but that would seem like a good place to start debugging.

Comment 94

[Wayne Mery (:wsmwk)]

Daniel, thanks for that great info.

Rob, Daniel, ..., do you agree with comment 26 (please test)?

p.s. (Rob) I know  you are trying to be helpful, but no need to be posting in the bug that the problem still exists - the bug is open, so it is obvious the problem continues, and everyone knows, so these comments do not add value

Comment 95

[Rob]

Hi,

- I won't 'bump' anymore, sorry 'bout that. 
- is it possible to install nightly builds without affecting the main install? Otherwise I'll have to find another pc to install the nightlies on.

Comment 96

[Wayne Mery (:wsmwk)]

https://support.mozilla.org/en-US/kb/using-multiple-profiles describes how to set up additional profiles.

Safest approach I think is to set up a test pop account on the same zimbra server.

Comment 97

[Rob]

Hi Wayne,

Comment 26 is the one about the regression window. Testing that will require me to have several nightlies installed, so I asked how to do that without getting into trouble with my own installation. I know how to reproduce the issue, but I'm leery about installing multiple instances of thunderbird together on my (productive) pc.

Comment 98

[Wayne Mery (:wsmwk)]

You will want to create a test pop account. Then, using your current version of Thunderbird create a new test  profile to use that test pop account. 

Next, install the builds mentioned in comment 26 one at a time, and using the TEST PROFILE and attempt to reproduce the problem. 

After testing is done, reinstall Thunderbird release from http://getthunderbird.com/ and return to your production profile.

Comment 99

[Reuben Thomas]

Thanks for the User Story update. Could someone edit it for language? I find the English pretty hard to follow at present.

Also, the Workaround only works for one particular error (relaying denied). There are other possible causes of this bug in which there's nothing wrong with the SMTP server. Perhaps this could be clarified?

Thanks again to all those working to fix this problem.

Comment 100

[WADA:World Anti-bad-Duping Agency]

(In reply to Reuben Thomas from comment #99)
> Could someone edit it for language?

Ask it to Wayne, please.

Comment 101

[Wayne Mery (:wsmwk)]

I don't see great room for improvement in terms of language - just many technical aspects listed.
If there is something specific you don't understand please point it and describe what is difficult.

Comment 102

[Reuben Thomas]

I think the best way is to show by example. Here's a rewritten version, according to my best understanding of the current version. I added some notes in square brackets where something seems to be missing.

The phenomenon itself is already analyzed and well understood, so additional reports or logs are not needed.

What is needed is data [what sort of data?] to create a correct patch for this bug, or such a patch.

Regression window: See comment #26
Possibly relevant change : bug 554044
  The problem may not be in the code that was changed; the changes may have merely exposed a problem elsewhere.

Workaround ["for users who are getting errors about relaying"; perhaps specify the exact codes?]:
  It may be that your ISP's SMTP server refuses From: addresses other than those of your ISP.
  This contravenes RFC [add number], so use an RFC-compliant SMTP server. (Thunderbird allows any SMTP server to be used by any identity.)
  1. Set "Outgoing (SMTP) server" [preferably give the full route to this preference] to an RFC-compliant SMTP server, and set it as "Default SMTP server".
  2. In the settings for the Identity which is having problems, choose "use default SMTP server".

Caveats:
1. Not all SMTP servers accept any From: address. Example: Yahoo! [Is there some way a user can tell whether an SMTP server will relay?]
2. Some servers require additional set-up to relay. For example, Gmail requires registration of addresses from which you want to send.
3. Some receiving servers may reject relayed mail the domain of whose From: address does not match the domain of the sending SMTP server.

If you cannot get a different SMTP server to work, and this bug is critical for you, you should consider using a different mail client.

Comment 103

[WADA:World Anti-bad-Duping Agency]

(In reply to Reuben Thomas from comment #99)
> Also, the Workaround only works for one particular error (relaying denied).
> There are other possible causes of this bug in which there's nothing wrong with the SMTP server.
> Perhaps this could be clarified?

"Relaying denied" by whom? Error code from who/when? 

Once all mail data is transfered to SMTP server and OK response(250) is returned from SMTP server, email client can do nothing. All is up to SMTP server.
This bug is for "any error from greeting by SMTP server to just before OK response from SMTP server", as easily known by already reported phenomena and SMTP logs in this bug.
And, cause of this bug is "connection loss after the error", instead of "the error itself", as easily known by already reported phenomena and SMTP logs in this bug.
So, error code itself is absolutely irrerevant to problem of this bug.

"Good SMTP server" in this context means "SMTP server who doesn't close connection after he returns error code, as defined by RFC".

"4xx in bug summary" is too misleading for you?

Comment 104

[WADA:World Anti-bad-Duping Agency]

[Off-Topic]

(In reply to Reuben Thomas from comment #102)

"Caveats" is too techinical for general users and non-English speakers like me.
So, I want to reject it :-)

By the way, this bug is mystery for any user, and is pretty annoying for almost all users as known by many dup'ed bugs and many reports in QA forum etc.
But I can do nothing, because I don't know Tb's SMTP code well...

Comment 105

[WADA:World Anti-bad-Duping Agency]

[Off-Topic]

> If you cannot get a different SMTP server to work, (snip)

Anyone can get free account of Gmail and use Gmail's SMTP server at any place except some places or some countries.
And, if corporate environment, "SMTP server who violates RFC" should be corrected first usually.
Therefore, this is rejected by me :-)

.

Comment 106

[WADA:World Anti-bad-Duping Agency]

(In reply to Reuben Thomas from comment #102)
> Workaround
>   It may be that your ISP's SMTP server refuses From: addresses other than those of your ISP.
>   This contravenes RFC, so use an RFC-compliant SMTP server.

This is never RFC violation by SMTP server such as SMTP server of Yahoo!.
This is based on Spam Policy of Yahoo!.
Gmail has different Spam Policy.
  If From: address other than Gmail' mail addr which is assigned to the used UserID of Gmail account,
  Gmail replaces mail address in From: by "mail addr which is assigned to the used UserID of Gmail account".
  This is design of Gmail.
  If you want to use other than "mail addr which is assigned to the used UserID of Gmail account" in From:,
  you need to register it to Gmail as "mail address which can be used as From:".
  This is same on "mail addr which is assigned to UserID of other Gmail account".
  In this case, Gmail doesn't alter From:, and adds "Sender: Gmail's mail addr" or something to mail.
  This is also design of Gmail.
"Relaying denied" is also based on Spam Policy of ISP. 

Please avoid selfish interpretation.

Comment 107

[Reuben Thomas]

I did not make a "selfish interpretation", I merely tried to make the original user story easier to understand.

This sort of issue is precisely why I asked for clarification in the first place; it would have been better done by someone who understands the issues better, but since no-one offered, I volunteered. Clearly, there is room for further improvement; I hope that will be made.

Comment 108

[WADA:World Anti-bad-Duping Agency]

(Off-Topic)

(In reply to Reuben Thomas from comment #107)
> I volunteered. Clearly, there is room for further improvement; I hope that will be made

I'm also a volunteer. Because "User Story" was not used by bug opener, I merely wrote a quick summary to User Story for bug reader's convenience, with incorrect description as less as possible.
"User Story" can be replaced if comment poster can change a field of this bug such as bug summary.
Please freely update "User Story" for bug readers, not for you, as far as your description is correct.

Comment 109

[viric]

There are plenty of ISP servers available, all with their peculiarities. The issue is very clear: the server answers with an error code (4XX, permanent error, it doesn't matter which one), and Thunderbird considers it "OK,Sent".

The suggestion regarding change of From address is just to help some people reproduce one of such sending errors. Another way is to use Avast (MITM-antivirus for SMTP), and set up a connection to a non-working SNMP server. Avast will translate the connection error to a SMTP 4XX error that Thunderbird will consider as "OK,Sent".

Comment 110

[Reuben Thomas]

(In reply to WADA from comment #108)
I've already updated the story to the best of my understanding. I don't understand what it means to update it "for bug readers, not for you, as far as your description is correct". I did not try to correct the story you wrote, merely make it easier to understand.

Comment 111

[Reuben Thomas]

(In reply to viric from comment #109)
> The suggestion regarding change of From address is just to help some people
> reproduce one of such sending errors.

I'm not sure what suggestion you are referring to. The user story suggests changing one's SMTP server, not one's From: address.

Again, if someone with a better technical grasp of the situation could update the user story, that would be very helpful. (My perspective is one of trying to advise users who encounter this problem what they should do.)

Comment 112

[WADA:World Anti-bad-Duping Agency]

 (In reply to Reuben Thomas from comment #111)
> > The suggestion regarding change of From address is just to help some people reproduce one of such sending errors.

It's because "change From:" is not essentially workaround of this bug.

It may be a workaroud of issue like "Relaying Denied" if From; is cause of the "Replayng denied".
Workaround of "Relaying denied" may be "use SMTP-AUTH(login with userID/password)" instead of "SMTP send without login" in some environment or situations. If such error is not returned from SMTP server, this bug doesn't occur.

In cntrast to it, error code such as "server is temporaary unavailable" can not be avoided.

(Off-Topic)

Update of User Story" for "better expression for bug readers" by you is wellcome, because User Story in this bug is for bug reder's convenience(bug opener didn't use it).
Rejecttion of "Caveats" is a kind of joke.
Please see "History" of this bug before your update of "User Story".
Because User Story is a field of a bug(similar to bug summary), if big change is done frequently, History is fiiled by log for User Story update. 
Log for User Story update seems diff. So, small change is safe. Please avoid frequent modification of entire User Story.

Comment 113

[viric]

Ah I see. I don't understand much from the User Story, neither how it is related to this bug. Along the comments there are different "user stories" that trigger server errors as the title of the bug says. The bug is so wide (interpreting any server error as OK) that it spans many "user stories".

In my situation, it was having the Avast antivirus installed that triggered all the problems.

The bug is one and clear, and so it says the 1st line of the User Story: server errors should be interpreted as errors in Thunderbird, and never consider a letter sent if the server said that it has not been sent.

Comment 114

[WADA:World Anti-bad-Duping Agency]

(In reply to viric from comment #113)
> In my situation, it was having the Avast antivirus installed that triggered all the problems.

Avast! case is best for problem analysis because Avast! immeditely closes connection.
Can you create debug build for analysis? (e.g. hook at many places which can berelevant to issue, and put trace log)

Comment 115

[Rob]

On thursday I'll be again at a customer where I can test this (most ISP's time out if there's a non permitted mail server, but I remember getting the error at customers which use Skynet).

By the way, I no longer experience this issue because all my ISPs now allow encrypted and password authenticated smtp sending (and I experienced this bug only because I was constantly switching between default smtp servers and when I forgot switching, I lost my mail)

Comment 116

[Daniel Kahn Gillmor]

I'm sorry, the SMTP server i had this problem on is no longer exhibiting the 4.3.2 response (there was a bug on their end), so i'm not able to test this easily.

I could test it if someone is willing to set up an SMTP server that always returns 4.3.2 responses to mail submission and let me know where it is, but i don't have the bandwidth to set that server up myself.

Comment 117

[Daniel Kahn Gillmor]

fwiw, the current user story seems to imply that this is only a problem with certain SMTP servers that are deliberately configured to reject mail.  while that's true, the scenario i ran into the problem with was a server that was one that was seeing what it perceived as a "persistent transient failure" -- so it was declining traffic intermittently.

So the issue is not just an issue for people with (arguably) misconfigured thunderbird instances -- it's a problem for any thunderbird instance that talks to any SMTP server that may occasionally fail in this way.

Comment 118

[Rob]

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #94)
> Daniel, thanks for that great info.
> 
> Rob, Daniel, ..., do you agree with comment 26 (please test)?


Hi all, I'll attach my test report. Suffice it to say that YES, I've found WADA's comment 26 to be correct and the bug was first there in the version from march 20.

Comment 119

[Rob]

Attachment: Thunderbird Bug 780124.pdf

The test report requested in comment 94

Comment 120

[WADA:World Anti-bad-Duping Agency]

(In reply to Rob from comment #118)
> I've found WADA's comment 26 to be correct (snip)

Does it mean that you were distrustful about my comment #26 until you did duplication test :-)
Thanks for your dupliction test. As known by comment #26, I did mistake in cooment #21.

Comment 121

[Rob]

(In reply to WADA from comment #120)
> (In reply to Rob from comment #118)
> > I've found WADA's comment 26 to be correct (snip)
> 
> Does it mean that you were distrustful about my comment #26 until you did
> duplication test :-)
> Thanks for your dupliction test. As known by comment #26, I did mistake in
> cooment #21.

*points at Wayne Mery* *points at comment 94* *winks suggestively*

Comment 122

[WADA:World Anti-bad-Duping Agency]

(Off-Topic) Oh, who were distrustful of it was Wayne :-)

Comment 123

[WADA:World Anti-bad-Duping Agency]

By the way, it looks for me;
- Before some changes :
  Ｗhen QUIT was issued due to error while Tb tries to send mail data to SMTP server,
  Tb went to "send QUIT step" again with keeping "error staus",
  in case of that some error occurs while sending QUIT after original error.
  This can be a Quirks for bypassing hard-to-resolve problem.
- After some changes :
  According to some comments arround change which was done in the regression window,
  it seems for me that a reason why such change was made is to resolve "QUIT loop" problem,
  which may be caused by the Quirks.

Comment 124

[WADA:World Anti-bad-Duping Agency]

Without co-operation of who did changes around SMTP code during the regression window, I can say/do nothing. But ...

Comment 125

[Alex Henrie]

I'm having the same problem. The SMTP server at my work only works when I'm at work or connected through a VPN. When I am not on the right network, the server refuses to accept any outgoing mail. Thunderbird displays an error message, but copies the unsent message to the Sent folder anyway. I poked around and discovered that this is what's happening:

1. Thunderbird connects to smtp.umail.utah.edu.

2. Before Thunderbird sends anything, the server sends "554 ipo4smtp.cc.utah.edu".

3. nsSmtpProtocol::ExtensionLoginResponse sees the error and alerts the user "An error occurred while sending mail: The mail server sent an incorrect greeting:  ipo0smtp.cc.utah.edu."

4. nsSmtpProtocol::ProcessProtocolState sees the error, sends "QUIT", and queues the SMTP_ERROR_DONE state.

5. The server hangs up without acknowledging the QUIT.

6. nsSmtpProtocol::OnStopRequest stops the SMTP state machine without ever running nsSmtpProtocol::ProcessProtocolState in the SMTP_ERROR_DONE state.

7. Thunderbird thinks the message was sent and copies it to the Sent folder.

This is very similar to bug 200647. The server (which is a Microsoft SMTP Server) is transgressing RFC 5321 section 3.1, but for the sake of the user, Thunderbird should gracefully handle the abrupt hangup.

I agree with WADA that https://hg.mozilla.org/comm-central/rev/6a883737262a from bug 554044 is to blame. That revision set m_sendDone to true in step 4. Previously m_sendDone was false if there was an error, which triggered the error handling code for bug 200647 in step 6.

Comment 126

[Alex Henrie]

Attachment: [PATCH] Gracefully handle SMTP servers that send a bad greeting and hang up.

This patch fixes the problem. It was really hard for me to wrap my head around this code, so feedback is appreciated. Right now m_sendDone == m_nextStateAfterResponse == SMTP_DONE || m_nextStateAfterResponse == SMTP_ERROR_DONE, and I decided to not change that. Keeping the existing m_sendDone semantics allowed me to fix (without duplicating code) the related case where there is a network failure (not a clean hangup) while sending QUIT after an error.

Instead of showing a second error dialog for NS_ERROR_NET_INTERRUPT like in old versions of Thunderbird, I chose to set NS_ERROR_BUT_DONT_SHOW_ALERT because one error dialog is enough, and the NS_ERROR_NET_INTERRUPT dialog was not helpful: "The message could not be sent because the connection to Outgoing server (SMTP) smtp.umail.utah.edu was lost in the middle of the transaction. Try again."

Users will still get an NS_ERROR_NET_INTERRUPT dialog as long as the interruption didn't happen while trying to gracefully shut down after a previous error.

https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=bf65cbeb6018

Comment 127

[Kent James (:rkent)]

Comment on attachment 8801912 [details] [diff] [review]
[PATCH] Gracefully handle SMTP servers that send a bad greeting and hang up.

If Magnus does not get to this, I'll look at it. It would be great to get this fixed.

Comment 128

[Kent James (:rkent)]

Comment on attachment 8801912 [details] [diff] [review]
[PATCH] Gracefully handle SMTP servers that send a bad greeting and hang up.

I spent a few hours looking at this today, getting to the point where I could also reproduce and view the problem and proposed solution. So I'll do the review of this.

Comment 129

[Kent James (:rkent)]

Attachment: Don't claim we are done when we failed

Here's my recommendation. When I looked at this, I asked myself "Why are we relying on complex tests of protocol state to see if we succeeded? There should be a variable that makes it clear!" Well actually there is such a variable, but it is set to succeeded in QUIT when QUIT is also used to close the connection after a failure. This patch just moves the test to the points where we know that we succeeded.

Comment 130

[Alex Henrie]

Comment on attachment 8805031 [details] [diff] [review]
Don't claim we are done when we failed

Review of attachment 8805031 [details] [diff] [review]:
-----------------------------------------------------------------

Hi Kent, thanks for looking into this. If we are going to change the semantics of m_sendDone, it would be better to pass an aSendDone parameter to SendQuit and set m_nextStateAfterResponse based on aSendDone. However, there is still the problem of popping up 2 dialogs when only 1 of them has useful information.

Comment 131

[Kent James (:rkent)]

Comment on attachment 8801912 [details] [diff] [review]
[PATCH] Gracefully handle SMTP servers that send a bad greeting and hang up.

Review of attachment 8801912 [details] [diff] [review]:
-----------------------------------------------------------------

As a general statement, you are trying to fix two problems here. One is a really important problem, which is the fact that TB claims it sent when it did not. The other is not so important, too many error messages. The solution to the first is being held up by discussions of how to handle the second. Let's not try to solve the second here but in a different bug.

So, where do we go from here? Unfortunately all of the core people are really frazzled at the moment getting ready for an impending release, so I'm going to have to short circuit this process to move it forward, as we would really like to get this fix landed. (It could also backport to TB 45). So I'm going to give additional comments on my approach, and bring in another reviewer to evaluate where we are at.

I'm happy to give you credit for this bug regardless of what approach that we take, but if you are not happy with my approach (as indicated by a feedback-), and we end up going that way, then I'll probably need to reassign the bug to myself rather than land something in your name that you do not approve of.

In any case, you've played a critical role in getting this bug to be fixed, so I want to sincerely thank you for being the catalyst to get this fixed.

::: mailnews/base/util/nsMsgProtocol.cpp
@@ +39,5 @@
>  #include "nsAlgorithm.h"
>  #include "mozilla/Services.h"
>  #include <algorithm>
>  #include "nsContentSecurityManager.h"
> +#include "nsComposeStrings.h"

You should not include a file from a specific protocol in this file which is used for a generic protocol.

::: mailnews/compose/src/nsSmtpProtocol.cpp
@@ +422,5 @@
>      MOZ_LOG(SMTPLogModule, mozilla::LogLevel::Info,
> +        ("SMTP connection error %lx while quitting", aStatus));
> +    if (m_nextStateAfterResponse == SMTP_ERROR_DONE)
> +      // if we were going to abort fcc even if the QUIT had succeeded, abort it
> +      aStatus = NS_ERROR_BUT_DONT_SHOW_ALERT;

On the issue of too many error messages, actually the protocol code is supposed to handle this using m_urlErrorState, which is set to NS_ERROR_BUT_DONT_SHOW_ALERT after the alert shows that is the primary error message. So the existing code already had a mechanism to try to handle this. Trying to also fix it here is really a kludge. If you want to fix this, you need to figure out why the existing method is not doing what you want.

I also really dislike having protocol state information being used outside of the tight protocol state machine. Yes they did it above, but I also dislike it there.

@@ +445,5 @@
> +    // for example, see bug 780124
> +    MOZ_LOG(SMTPLogModule, mozilla::LogLevel::Info,
> +        ("SMTP connection dropped while attempting graceful failure"));
> +    nsMsgAsyncWriteProtocol::OnStopRequest(nullptr, ctxt,
> +                                           NS_ERROR_BUT_DONT_SHOW_ALERT);

Again, this is not the right place to fix this.

Comment 132

[Kent James (:rkent)]

Comment on attachment 8805031 [details] [diff] [review]
Don't claim we are done when we failed

Jorg, could you look at this? I could make the changes suggested by Alex if you want, but I'm not sure that I see the point.

Comment 133

[Jorg K (CEST = GMT+2)]

(In reply to Kent James (:rkent) from comment #132)
> Jorg, could you look at this? I could make the changes suggested by Alex if
> you want, but I'm not sure that I see the point.
We're talking about this, right?
=== Quote from copmment #130?
If we are going to change the semantics of m_sendDone, it would be better to pass an aSendDone parameter to SendQuit and set m_nextStateAfterResponse based on aSendDone.
===
Would that be hard to do and then we've got two solutions to chose from and comment on?

Comment 134

[Jorg K (CEST = GMT+2)]

Comment on attachment 8805031 [details] [diff] [review]
Don't claim we are done when we failed

Review of attachment 8805031 [details] [diff] [review]:
-----------------------------------------------------------------

I've looked at this now and I think Kent's solution to separate the setting of m_sendDone from SendQuit() is simple and transparent. Alex, where did you plan to set 'm_nextStateAfterResponse'?

::: mailnews/compose/src/nsSmtpProtocol.cpp
@@ +606,5 @@
>    NS_ENSURE_SUCCESS(rv, rv);
>    bool verifyingLogon = false;
>    smtpUrl->GetVerifyLogon(&verifyingLogon);
> +  if (verifyingLogon) {
> +    m_sendDone = true;  // We did what we were attempting.

Please move the inline comment above the line like this:

// Just verifying the ability to logon was done/succeeded(?)
// so we pretend the send (which we didn't do) was
// a success.
m_sendDone = true;

Comment 135

[Alex Henrie]

Attachment: change-m_sendDone-semantics.diff

Kent, this is how I would improve upon your patch. But my fear is that if we change the m_sendDone semantics, it will require complicated code to fix the two-dialogs problem, and so the two-dialogs problem will never be fixed.

> You should not include a file from a specific protocol in this file which is
> used for a generic protocol.

We can avoid referencing NS_ERROR_BUT_DONT_SHOW_ALERT in nsMsgProtocol::OnStopRequest by simply deleting the NS_ASSERTION from nsMsgProtocol::OnStopRequest. We'd have to do this anyway if we change the m_sendDone semantics.

> On the issue of too many error messages, actually the protocol code is
> supposed to handle this using m_urlErrorState, which is set to
> NS_ERROR_BUT_DONT_SHOW_ALERT after the alert shows that is the primary error
> message. So the existing code already had a mechanism to try to handle this.
> Trying to also fix it here is really a kludge. If you want to fix this, you
> need to figure out why the existing method is not doing what you want.

The protocol code is okay except that it has no way to tell the rest of the mail-sending code that an alert should not be shown for a network error following another error. The only solution I can see is to make nsSmtpProtocol::OnStopRequest change the error to NS_ERROR_BUT_DONT_SHOW_ALERT if m_nextStateAfterResponse == SMTP_ERROR_DONE.

Don't get me wrong; I am really happy to see movement on this bug, and I agree that pretty much anything would be better than what we have now.

Comment 136

[Kent James (:rkent)]

Comment on attachment 8805631 [details] [diff] [review]
change-m_sendDone-semantics.diff

I don't want to lose track of the need for me to look at this.

Comment 137

[Jorg K (CEST = GMT+2)]

Comment on attachment 8805031 [details] [diff] [review]
Don't claim we are done when we failed

Cancelling this for now since there is an alternative patch.

Comment 140

[Luis Freitas]

This problem is happening to me with frequency. My company uses a Microsoft Exchange cloud based server and it frequently replies with a 4.4.2 error and the message is not sent, instead of being queued. People using other mail clients dont experience any problems.

Does setting mailnews.sendInBackground to true will have influence in this problem?

Comment 144

[jonwestburne]

Any progress on this?
I have not had this problem for a very long time until last Friday when TB 52.03.00 gave
"An error occurred while sending mail. The mail server responded:  5.2.1 :  AOL will not accept delivery of this message.. Please check the message and try again.
Delivering mail 100%"
The error message stayed open until cancelled (a major improvement on before), on closing the error message the unsent message closed and was moved to sent folder, the message was not sent.
The problem still ocurred with message content "test" but only ocurred whilst using a Hull Trains wifi connection.

Comment 145

[Jorg K (CEST = GMT+2)]

Comment on attachment 8805631 [details] [diff] [review]
change-m_sendDone-semantics.diff

All I can say is that the review request does not show up in Kent's review queue. Strange. So that's why this has fallen trough the cracks. So let me cancel the request and request it again.

Comment 146

[Jorg K (CEST = GMT+2)]

Now it worked:
https://bugzilla.mozilla.org/request.cgi?action=queue&type=review&requestee=rkent%40caspia.com&group=type

Comment 147

[gene smith]

Attachment: 780124-gds-review-changes0.patch

In conjunction with Bug 1390442 and Bug 1409678, which are closely related to this bug (maybe duplicates, at least in terms of the resulting fix), I have a proposed modified patch that fixes these problems:

The following activities have a similar result:
A network stream error occurs while sending email.
A fatal SMTP reply errors occur while sending email.
A password is requested when sending and is entered incorrectly. Cancel button is clicked on the 3-button login dialog before re-entering the correct password.
A password is requested when sending and Cancel button is clicked instead of OK.
A password is requested when sending and is entered incorrectly. Then on the password re-entry prompt, just click cancel instead of OK.
Account A configured to use, e.g., yahoo's smtp server. Yahoo doesn't like this. Send an email from account A and take some time to read the error message while the connection times-out. (This is the cause of this bug.)

The message is not sent but the message appears in the Sent folder.

This has a different result:
When sending, enter bad password on 1st prompt. On next password re-entry prompt enter bad password again. Then cancel on login 3-button screen. 

Leaves "Sending messages - ..." with Connected to smtp.mail..... box with progress bar moving left to right. Cancel "Sending Message--..."  and progress bar remains. Then cancel progress bar window. Write (compose) window still present but read only! Send button and menus gray (disabled). In this case, message is not moved to Sent and Write window can only be dismissed.

The attached patch does basically what the previous patch did in addition to fixing the read-only write window problem. I have also done a lot of testing to diagnose the problem and verify the fixes.

Comment 148

[Jorg K (CEST = GMT+2)]

Comment on attachment 8923289 [details] [diff] [review]
780124-gds-review-changes0.patch

I don't know how I inherited this. Let's see whether Kent responds within a month. Looking at comment #131, he knows a whole lot more about this than I do.

Kent: What's the story with [e-mail]?

Comment 149

[Jim]

This problem has really been throwing me for a loop.  CenturyLink for some reason is frequently issuing 421 errors.  Because of the way Tbird was acting, I thought it was just a warning and since it copied the message to the "sent" folder, I had a record of sending the message, though it never went anywhere.  Really awful to tell people you had replied to them and yet you actually had not.

Comment 150

[Alex Henrie]

Comment on attachment 8923289 [details] [diff] [review]
780124-gds-review-changes0.patch

Review of attachment 8923289 [details] [diff] [review]:
-----------------------------------------------------------------

Like Kent's patch, Gene's patch fixes the major problem and would be OK to commit to Thunderbird, although it pops up two error dialogs when one is enough.

Comment 156

[Jorg K (CEST = GMT+2)]

Comment on attachment 8805631 [details] [diff] [review]
change-m_sendDone-semantics.diff

Ben, here comes another hard one. As you can see, many votes, many duplicates, apparently two problems, the second one is some doubled-up message, different approaches where one party puts r-/f- on the patches of the other. Magnus and I decided to give you the honourable task to cut though it and favourise one patch or tweak it into shape.

Comment 157

[Jorg K (CEST = GMT+2)]

Comment on attachment 8923289 [details] [diff] [review]
780124-gds-review-changes0.patch

And this one. Most likely the patches have bitrot. Enjoy ;-)

Comment 158

[Ben Campbell]

Sure thing. The fixes don't look too complex, so I guess the trickiness is understanding the larger context and side effects. A good excuse to dive into another part of the code I've not looked at much yet!

Comment 159

[Jorg K (CEST = GMT+2)]

Looks like Kent's patch (attachment 8805031 [details] [diff] [review]) is the base upon every one agrees. Then we have to improved versions. Gene does a lot of testing, see comment #147 (!!) and even Axel agreed that it might be the way forward (comment #150). Maybe you can work out what the "two dialogue" problem is and move it to a new bug/patch.

Comment 160

[Ben Campbell]

Attachment: badsmtpd.py

A little script to implement a badly-behaving smtp server which responds to MAIL commands with "554 Go away!", then disconnects.

Usage:
Run it locally, then set your outgoing STMP server to localhost:6502

(with this script I can confirm the copy-to-sent-mail-after-an-error behaviour, so I'm looking at the fixes now).

Comment 161

[Leonardo]

Hello people, I'm using SeaMonkey in Windows 10, so command lines mean nothing to an end user like me, illiterate in Linux and programming. I wonder if I copy the command lines into a .txt file and save it as .bat or .exe and execute it afterwards. Please give me a clue at it.

Comment 162

[Jorg K (CEST = GMT+2)]

How does your comment #161 relate to this bug? Please don't post unrelated comments.

Comment 163

[Leonardo]

Jorg K, my post is related to the codes pasted here as instructions for command-line based OSs.

Comment 164

[Leonardo]

Jorg K, please tell me if there is something I can do with the aforementioned patches, because SM has the option Tools -> Web Development -> Programmer's toolbar.
Maybe there is something I can do in order to make it work in my Windows 10 OS, thank you.

Comment 165

[Jorg K (CEST = GMT+2)]

In order to make something work, you need to build/compile Thunderbird. There is no easy fix here.

Comment 166

[Leonardo]

After giving up Seamonkey, I've taken my backed up data and installed Thunderbird. But the same problem is still there. I thought the greater Thunderbird popularity would be enough to prompt the community to fix this problem.
I've alto tried to Google some possible solution and ended up in an interesting web page with some workarounds which did not work in my case, though:
http://kb.mozillazine.org/Cannot_send_mail

Comment 167

[Leonardo]

BTW I've followed the instructions therein to log the events and you can see below the results:

2019-08-08 21:30:15.478000 UTC - [5220:Main Thread]: I/SMTP SMTP Connecting to: xxx.xxx.xxx.xx:110
2019-08-08 21:30:15.525000 UTC - [5220:Main Thread]: I/SMTP SMTP entering state: 0
2019-08-08 21:30:15.525000 UTC - [5220:Main Thread]: I/SMTP SMTP Response: +OK Hello there.
2019-08-08 21:30:15.525000 UTC - [5220:Main Thread]: I/SMTP SMTP entering state: 14
2019-08-08 21:30:16.500000 UTC - [5220:Main Thread]: I/SMTP SMTP Send: QUIT

2019-08-08 21:30:16.500000 UTC - [5220:Main Thread]: I/SMTP SMTP entering state: 0
2019-08-08 21:30:16.519000 UTC - [5220:Main Thread]: I/SMTP SMTP entering state: 0
2019-08-08 21:30:16.519000 UTC - [5220:Main Thread]: I/SMTP SMTP Response: +OK Better luck next time.
2019-08-08 21:30:16.519000 UTC - [5220:Main Thread]: I/SMTP SMTP entering state: 11
2019-08-08 21:30:16.530000 UTC - [5220:Main Thread]: I/SMTP SMTP entering state: 12
2019-08-08 21:30:16.530000 UTC - [5220:Main Thread]: I/SMTP SMTP connection error quitting 80004004, ignoring
2019-08-08 21:30:17.789000 UTC - [5220:Main Thread]: I/SMTP SMTP Connecting to: xxx.xxx.xxx.xx:110
2019-08-08 21:30:17.839000 UTC - [5220:Main Thread]: I/SMTP SMTP entering state: 0
2019-08-08 21:30:17.839000 UTC - [5220:Main Thread]: I/SMTP SMTP Response: +OK Hello there.
2019-08-08 21:30:17.839000 UTC - [5220:Main Thread]: I/SMTP SMTP entering state: 14
2019-08-08 21:30:18.812000 UTC - [5220:Main Thread]: I/SMTP SMTP Send: QUIT

2019-08-08 21:30:18.812000 UTC - [5220:Main Thread]: I/SMTP SMTP entering state: 0
2019-08-08 21:30:18.832000 UTC - [5220:Main Thread]: I/SMTP SMTP entering state: 0
2019-08-08 21:30:18.832000 UTC - [5220:Main Thread]: I/SMTP SMTP Response: +OK Better luck next time.
2019-08-08 21:30:18.832000 UTC - [5220:Main Thread]: I/SMTP SMTP entering state: 11
2019-08-08 21:30:18.842000 UTC - [5220:Main Thread]: I/SMTP SMTP entering state: 12
2019-08-08 21:30:18.842000 UTC - [5220:Main Thread]: I/SMTP SMTP connection error quitting 80004004, ignoring
2019-08-08 21:30:19.852000 UTC - [5220:Main Thread]: I/SMTP SMTP Connecting to: xxx.xxx.xxx.xx:110
2019-08-08 21:30:19.912000 UTC - [5220:Main Thread]: I/SMTP SMTP entering state: 0
2019-08-08 21:30:19.912000 UTC - [5220:Main Thread]: I/SMTP SMTP Response: +OK Hello there.
2019-08-08 21:30:19.912000 UTC - [5220:Main Thread]: I/SMTP SMTP entering state: 14
2019-08-08 21:30:20.944000 UTC - [5220:Main Thread]: I/SMTP SMTP Send: QUIT

2019-08-08 21:30:20.944000 UTC - [5220:Main Thread]: I/SMTP SMTP entering state: 0
2019-08-08 21:30:20.964000 UTC - [5220:Main Thread]: I/SMTP SMTP entering state: 0
2019-08-08 21:30:20.964000 UTC - [5220:Main Thread]: I/SMTP SMTP Response: +OK Better luck next time.
2019-08-08 21:30:20.964000 UTC - [5220:Main Thread]: I/SMTP SMTP entering state: 11
2019-08-08 21:30:20.974000 UTC - [5220:Main Thread]: I/SMTP SMTP entering state: 12
2019-08-08 21:30:20.974000 UTC - [5220:Main Thread]: I/SMTP SMTP connection error quitting 80004004, ignoring

Comment 170

[Wayne Mery (:wsmwk)]

(In reply to Ben Campbell from comment #160)

Created attachment 9046887 [details]
badsmtpd.py

A little script to implement a badly-behaving smtp server which responds to MAIL commands with "554 Go away!", then disconnects.

Usage:
Run it locally, then set your outgoing STMP server to localhost:6502

(with this script I can confirm the copy-to-sent-mail-after-an-error behaviour, so I'm looking at the fixes now).

Ben, Did you come up with any further ideas or modified WIP?

Comment 173

[Bernhard Riegler]

I have the following environment: Europe, Austria, A1 Provider, sending with Thunderbird 78.7.1(64-bit) on GNU/Linux Ubuntu 20.04 LTS
the outgoing configuration is SMTP port 587 with STARTTLS

everything works fine, but sometimes the Provider SMTP Server is BUSY.
I get in thunderbird client a pop up "load too high (#4.3.0)" and then the email is in 'sent' folder.
as I always put myself a CC in every email, I can guarantee that the email was not processed by the SMTP server.
this matches the pop up message.
this is a transient SMTP server error.
thunderbird client knows this and shall not put this failed output into folder 'sent' , but into folder 'draft' for later retry.

Comment 174

[Bernhard Riegler]

after thinking about the possibilities for this server "transient failure"
I would recommend rescue in the local Filesystem on server status (#4.3.0) independent of config for folder 'sent' and folder 'draft'
make a subdirectory "rescue" in the local File System below local ARCHIVE folder and put the email in here.
this is independent of the server BUSY problem.
the file in 'rescue' folder can now easily retried later and is not 'lost on transit'

Comment 175

[jonwestburne]

The reason that this bug is critical is because a user does not know an email was not sent. The solution that is needed is for the user to know that it has not been sent. A workaround does not help if a user does not know there is a problem. I raised the issue many years ago and have kept an eye that the brief message that sending fails, before the email is put in the sent folder, does not appear. For me the circumstances that originally caused it seem no longer to apply.

Comment 176

[stephan]

In case the server responds with a 550 Thunderbird correctly shows an error message to the user and goes back to the edit window, so you can resend the message.

If it would do the same in the case of a 4XX all would be good.

Should be an easy fix.

Comment 177

[Ulf Zibis]

Same cause for bug 1656240 comment 11.

Comment 178

[Bernhard Riegler]

I completely agree with jonwestburne
CRITICAL because the user does not know the email was not sent.

I had again a server BUSY condition.
thunderbird version 78.7.1(64-bit) on GNU/Linux Ubuntu 20.04 LTS platform.
popup "load too high (#4.3.0)" and then the email is in 'sent' folder of my local Filesystem Archive.
my automatic "CC to sender" proves that it was not processed by server. I did not receive my CC.

Comment 180

[Wayne Mery (:wsmwk)]

Do we expect this to be gone with smtp-js?

Comment 181

[Ping Chen [:rnons] (away)]

(In reply to Wayne Mery (:wsmwk) from comment #180)

Do we expect this to be gone with smtp-js?

Yes, I think so. Compose window is still open in my local test.

Comment 182

[Wayne Mery (:wsmwk)]

I just realized Ben has a test script comment 160. Does smtp-js have a test library, and does it cover this?

Comment 183

[Ping Chen [:rnons] (away)]

We have a Smtpd.jsm fakeserver for testing, we don't have a test case for this bug though.

Comment 184

[Jonathan Kamens]

The other reason this is a critical bug is because if the user does not have Thunderbird configured to save messages in Sent Mail -- which the user shouldn't, e.g., if the user is using Gmail, since Gmail saves outbound messages automatically, then this bug causes DATA LOSS by closing the window without saving the message ANYWHERE.

Can believe this bug is still not fixed after nearly a decade. Glad to see it's about to be fixed, but as someone else pointed out in a recent comment, if this is just a matter of changing the status codes that cause Thunderbird to leave the compose window open, then maybe we should make that fix in TB78 rather than waiting until the next major release for it to be fixed?

Comment 185

[Magnus Melin [:mkmelin]]

Thunderbird 91 is just around the corner, and any change to 78 could no longer get beta testing, so even with a fix to the old code we would not take it at this point.