534158 - mail not send Thunderbird 3 STARTTLS+AUTH (SMTP server retruns AUTH to EHLO for connection from external, but doesn' return AUTH to EHLO for connection from internal)

Status: RESOLVED FIXED

(MailNews Core :: Networking: SMTP, defect)

Description

[Rafał]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.0; pl; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR 3.5.30729)
Build Identifier: 

I upgraded last night to TB 3 and now receive this message:

Sending of message failed.
An error occurred sending mail: Unable to authenticate to SMTP server smtp.blablabla. It does not support authentication (SMTP-

AUTH) but you have chosen to use authentication. Uncheck 'Use name and password' for that server or contact your service provider.

I had no problem until I upgraded on Thunderbird 2, Internet Explorer, Windows Live, Microsoft Outlok, Opera, TheBat etc...

[img]http://rpc.one.pl/opis_fsv6505/conf.jpg[/img]

Server Exim4
STARTTLS+AUTH - WAN required
STARTTLS+AUTH - LAN optional

method
LOGIN
PLAIN

port 587 or 25

220 mail.xxxxxxxxxxxx.com.pl ESMTP to Exim
EHLO mail.xxxxxxxxx.com.pl
250-mail.xxxxxxxxxxxxxx.com.pl Hello yyyyyyyyyyyyy.xxxxxxxxxxxx.com.pl [192.168.1.1]
250-SIZE 52428800
250-PIPELINING
250-STARTTLS
250 HELP

other
#openssl s_client -host mail.xxxxxxxxxxx.com.pl -port 587 -starttls smtp
250 HELP
ehlo test
250-mail.xxxxxxx.com.pl Hello test [xxxxxxxxxxxxx]
250-SIZE 52428800
250-PIPELINING
250-AUTH LOGIN PLAIN
250 HELP



dovecot_login:
  driver = dovecot
  public_name = LOGIN
  server_advertise_condition =${if def:tls_cipher }
#  server_advertise_hosts = ${if eq{$tls_cipher}{}{}{*}}
  server_socket = /var/run/dovecot/auth-client
  server_set_id = $auth1

dovecot_plain:
  driver = dovecot
  public_name = PLAIN
  server_advertise_condition =${if def:tls_cipher }
  server_socket = /var/run/dovecot/auth-client
  server_set_id = $auth1

auth_advertise_hosts = !+relay_from_hosts : +auth_relay_hosts

Dovecot IMAP+POP3+IMAPS+POP3S support

Problem is Thunderbird 3 - logs (TB3 and TB2)

=============== Thunderbird 3 ======Send is FAIL

917 host in auth_advertise_hosts? no (matched "!+relay_from_hosts")
917 host in tls_advertise_hosts? yes (matched "*")
917 SMTP>> 250-mail.xxxxxxxxxxxxx.com.pl Hello yyyyyy.xxxxxxxxxxxxx.com.pl [10.0.0.107]
917 250-SIZE 52428800
917 250-PIPELINING
917 250-STARTTLS
917 250 HELP
917 SMTP<< STARTTLS
917 initializing GnuTLS as a server
917 read D-H parameters from file
917 initialized D-H parameters
917 certificate file = /etc/exim4/mail.xxxxxxxxxxxxx.com.pl.crt
917 key file = /etc/exim4/mail.xxxxxxxxxxxxx.com.pl.key
917 initialized certificate stuff
917 host in tls_verify_hosts? no (option unset)
917 host in tls_try_verify_hosts? no (option unset)
917 initialized GnuTLS session
917 SMTP>> 220 TLS go ahead
917 gnutls_handshake was successful
917 cipher: TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32
917 sender_fullhost = yyyyyy.xxxxxxxxxxxxx.com.pl [10.0.0.107]
917 sender_rcvhost = yyyyyy.xxxxxxxxxxxxx.com.pl ([10.0.0.107])
917 set_process_info: 917 handling incoming TLS connection from yyyyyy.xxxxxxxxxxxxx.com.pl [10.0.0.107]
917 TLS active
917 Calling gnutls_record_recv(8318bf8, 831b330, 4096)
917 SMTP<< EHLO [10.0.0.107]
917 sender_fullhost = yyyyyy.xxxxxxxxxxxxx.com.pl [10.0.0.107]
917 sender_rcvhost = yyyyyy.xxxxxxxxxxxxx.com.pl ([10.0.0.107])
917 set_process_info: 917 handling TLS incoming connection from yyyyyy.xxxxxxxxxxxxx.com.pl [10.0.0.107]
917 host in pipelining_advertise_hosts? yes (matched "*")
917 cached yes match for +relay_from_hosts
917 host in auth_advertise_hosts? no (matched "!+relay_from_hosts" - cached)
917 tls_do_write(82f3ac0, 125)
917 gnutls_record_send(SSL, 82f3ac0, 125)
917 outbytes=125
917 SMTP>> 250-mail.xxxxxxxxxxxxx.com.pl Hello yyyyyy.xxxxxxxxxxxxx.com.pl [10.0.0.107]
917 250-SIZE 52428800
917 250-PIPELINING
917 250 HELP
917 Calling gnutls_record_recv(8318bf8, 831b330, 4096)
917 SMTP<< QUIT
917 SMTP>> 221 mail.xxxxxxxxxxxxx.com.pl closing connection



=============== Thunderbird 2 =================Send is PASS

1230 SMTP>> 250-mail.xxxxxxxxxxxxx.com.pl Hello host2.xxxxxxxxxxxxx.com.pl [10.0.0.134]
1230 250-SIZE 52428800
1230 250-PIPELINING
1230 250-STARTTLS
1230 250 HELP
1230 SMTP<< STARTTLS
1230 initializing GnuTLS as a server
1230 read D-H parameters from file
1230 initialized D-H parameters
1230 certificate file = /etc/exim4/mail.xxxxxxxxxxxxx.com.pl.crt
1230 key file = /etc/exim4/mail.xxxxxxxxxxxxx.com.pl.key
1230 initialized certificate stuff
1230 host in tls_verify_hosts? no (option unset)
1230 host in tls_try_verify_hosts? no (option unset)
1230 initialized GnuTLS session
1230 SMTP>> 220 TLS go ahead
1230 gnutls_handshake was successful
1230 cipher: TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32
1230 sender_fullhost = host2.xxxxxxxxxxxxx.com.pl [10.0.0.134]
1230 sender_rcvhost = host2.xxxxxxxxxxxxx.com.pl ([10.0.0.134])
1230 set_process_info: 1230 handling incoming TLS connection from host2.xxxxxxxxxxxxx.com.pl [10.0.0.134]
1230 TLS active
1230 Calling gnutls_record_recv(8f09bf8, 8f0c330, 4096)
1230 SMTP<< EHLO [10.0.0.134]
1230 sender_fullhost = host2.xxxxxxxxxxxxx.com.pl [10.0.0.134]
1230 sender_rcvhost = host2.xxxxxxxxxxxxx.com.pl ([10.0.0.134])
1230 set_process_info: 1230 handling TLS incoming connection from host2.xxxxxxxxxxxxx.com.pl [10.0.0.134]
1230 host in pipelining_advertise_hosts? yes (matched "*")
1230 cached yes match for +relay_from_hosts
1230 host in auth_advertise_hosts? no (matched "!+relay_from_hosts" - cached)
1230 tls_do_write(8ee4ac0, 124)
1230 gnutls_record_send(SSL, 8ee4ac0, 124)
1230 outbytes=124
1230 SMTP>> 250-mail.xxxxxxxxxxxxx.com.pl Hello host2.xxxxxxxxxxxxx.com.pl [10.0.0.134]
1230 250-SIZE 52428800
1230 250-PIPELINING
1230 250 HELP
1230 Calling gnutls_record_recv(8f09bf8, 8f0c330, 4096)
1230 SMTP<< MAIL FROM:<[e-mail]> SIZE=1344
1230 spool directory space = 10268720K inodes = 16055705 check_space = 0K inodes = 0 msg_size = 6344
1230 SMTP>> 250 OK


Best Regards
Rafał

Reproducible: Always

Comment 1

[rsx11m]

The trigger for this seems to be the fact that the original greeting only advertises STARTTLS but no AUTH response is given. After TLS has been entered, AUTH mechanisms are provided in the second EHLO response. This should be a valid approach, e.g., if the server wants to enforce STARTTLS usage. I'd assume that the error message is triggered by the first response, though the TLS phase is evidently entered in the log and only then apparently the message shown. Thus, server capabilities would need to be updated from the 2nd EHLO reponse.

Comment 2

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #1)
> The trigger for this seems to be the fact that the original greeting only
> advertises STARTTLS but no AUTH response is given. After TLS has been entered,
> AUTH mechanisms are provided in the second EHLO response. This should be a
> valid approach, e.g., if the server wants to enforce STARTTLS usage.

Following is SMTP log with Gmail's SMTP server who requests port=587, StartTLS, and Use usename and password.
> SMTP Connecting to: smtp.gmail.com	
> SMTP Response: 220 mx.google.com ESMTP j34sm8506981waf.62	
> SMTP Send: EHLO [192.168.0.10] 	
> SMTP Response: 250-mx.google.com at your service, [222.9.106.185]	
> SMTP Response: 250-SIZE 35651584	
> SMTP Response: 250-8BITMIME	
> SMTP Response: 250-STARTTLS	
> SMTP Response: 250-ENHANCEDSTATUSCODES	
> SMTP Response: 250 PIPELINING	
> SMTP Send: STARTTLS 	
> SMTP Response: 220 2.0.0 Ready to start TLS	
> SMTP Send: EHLO [192.168.0.10] 	
> SMTP Response: 250-mx.google.com at your service, [222.9.106.185]	
> SMTP Response: 250-SIZE 35651584	
> SMTP Response: 250-8BITMIME	
> SMTP Response: 250-AUTH LOGIN PLAIN	
> SMTP Response: 250-ENHANCEDSTATUSCODES	
> SMTP Response: 250 PIPELINING	
> Logging suppressed for this command (it probably contained authentication information)	
> SMTP Response: 235 2.7.0 Accepted	
> SMTP Send: MAIL FROM:<[e-mail]> SIZE=407 	
Tb works as you say, as expected.

> I'd assume that the error message is triggered by the first response,
> though the TLS phase is evidently entered in the log
> and only then apparently the message shown.
> Thus, server capabilities would need to be updated from the 2nd EHLO reponse.

As seen in SMTP log with Gmail's SMTP, Tb updates capability from the 2nd EHLO response as you say and as expected. 
"No AUTH in EHLO responce to EHLO after StartTLS" case, isn't it?
(i.e. dup of bug 534111, isn't it?)

Comment 3

[WADA:World Anti-bad-Duping Agency]

> #openssl s_client -host mail.xxxxxxxxxxx.com.pl -port 587 -starttls smtp
> 250 HELP
> ehlo test
> 250-mail.xxxxxxx.com.pl Hello test [xxxxxxxxxxxxx]
> 250-SIZE 52428800
> 250-PIPELINING
> 250-AUTH LOGIN PLAIN
> 250 HELP

If port=587(message submission port), your server looks to return AUTH to 2nd EHLO after StartTLS, as "message submission port" requests.
Do you set port=587? Or port=25?
Next is "no AUTH to 2nd EHLO after StartTLS if port=25", isn't it?
> STARTTLS+AUTH - LAN optional

Comment 4

[Rafał]

The assumption is this: From the WAN authentication forced the first STARTTLS before AUTH. 
From the LAN is not required to use STARTTLS or AUTH. Is optional. 

So you can see that from the LAN can send mail without authorization, or if someone has set such a laptop with authorization STARTTLS + AUTH. And here comes the problem. In this case, the server does not provide the STARTTLS SMTP-AUTH because that would be required and I do not want to. I want to from the LAN SMTP-AUTH was optional. 

Thunderbird 2 and all previous versions including other mail clients when the server does not require a user name simply ignore it by sending an e-mail and look at some logs from Thunderbirda2. Thunderbird3 immediately assumed that I made an error in setting up and screams to turn off this option. But it is a laptop. Yet still I will not configuring it just to attach or disable a user name. In addition, As I wrote previously Thunderbird 2, Outlook Express, Windows Live, Microsoft Outlook, The Bat, Opera, IncreditMail behave the same or is all about I hope that this will be corrected. In this system for me is a big problem, and while it is not corrected rather not bring this to use in the company.

Comment 5

[Rafał]

Proposes to make the following assumption. If the mail client is enabled STARTTLS or/and SMTP-AUTH and the server does not require (or not present at EHLO) is a mail client should send an e-mail without authorization. In other words, even though the client option is enabled the client ignores the username option or TLS when sending mail.

Comment 6

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #5)
> In other words, even though the client option is enabled
> the client ignores the username option or TLS when sending mail.

Have you read bug 534111 I pointed? 
Implementation around StratTLS and "use username and password"(SMTP-AUTH) is changed by Tb3 mainly for security reason.
Read bug 534111 I already pointed for "use username and password", and read bug 532413 for StartTLS(no "TLS, if avail" in Tb3), please.
Your required action on Tb's SMTP setting is very simple;
  If server doesn't support SMTP-AUTH, uncheck "use username and password.
  If server requests SMTP-AUTH, check "use username and password.
  If server doesn't support StartTLS, select None or SSL/TLS.
  If server requests StartTLS, select StartTLS.

Your server looks to support next settings of Tb3:
  (a) Port=25
        "use username and password" is unchecked
        StartTLS
  (b) Port=587
        "use username and password" is checked
        StartTLS
If ISP started "outbound port 25 blocking", (b) is mandatory.
If ISP didn't start "outbound port 25 blocking", any of (a) and (b) is usable.
"Use (a) or (b)" is up to you, and "(a) works and/or (b) works" depends on your environment(ISP you use for internet access, SMTP server you use).

Comment 7

[rsx11m]

WADA, please see bug 534111 comment #10, this is yet another case of having different requirements for access from "trusted" vs. "untrusted" networks:

> (1) From the WAN authentication forced the first STARTTLS before AUTH.
> (2) From the LAN is not required to use STARTTLS or AUTH. Is optional.

In this case, on top of that STARTTLS is only supported for WAN access, which was accommodated by "TLS, if available" in TB 2.0 (no longer supported in 3.0).

It appears that there is more fall-out from bug 311657 by TB 2.0 use cases than was expected...

Comment 8

[Rafał]

In fact it can only send mail to port 587 Port 25 is blocked in the new year. But this is not a problem. Server supports the SMTP-AUTH just do not announce on the LAN. Is this a bug?. If the server supports the AUTH TLS and I said to you that he wants to use the STARTTLS AUTH Thunderbird3 it expects that it will send an e-mail according to the settings they declared. He checks the EHLO and displays an error message that the server does not support the AUTH which is not true. Exim simply does not present this in the EHLO. This does not mean, however, that if I want to send mail through STARTTLS AUTH is a server that does not handle. Although the server from the LAN does not provide SMTP-AUTH as possible send a message, according to my settings. These settings are the only option. It Thunderbird3 not seeing the SMTP-AUTH immediately assumes that it is wrong. Mail program should first try to send a message as it fails display appropriate message sent by the server (exim).

You can obviously see the error of the configuration, but only when the account setup in your mail client. During normal operation of the e-mail program should no longer detect whether the client is properly configured to give the server and sends a message to the user and the program displays this message if it is a configuration error.

Server supports:
1. From the WAN (Internet) forced STARTTLS + AUTH in the same order on port 587 (25 will be turned off for members). To send mail in your mail client I have set the port 587, STARTTLS, and username (AUTH). Other options will not work. So you want to.
2. From the LAN (extranet) to send mail without authorization or options STARTTLS, AUTH on port 587 (25 will be turned off for members). To send a message I have in your mail client set up (there are three possibilities / options):
a) port 587 (just enough)
b) port 587 + STARTTLS (just enough)
c) the port 587 + STARTTLS + username (AUTH) - (the server is not present in the EHLO SMTP-AUTH because he would not allow sending mail from the point of a, b)

From the outset, it was not possible from the WAN to send mail without authorization. The server is not set as openrelay.

However, for LAN users are made available to the ability to send mail without authorization from the options to send mail through STARTTLS, AUTH what the server from the LAN is not already present in the EHLO.

This method has been tested with most email clients. It was never the problem. Only now Thunderbird3 not want to send mail.

Comment 9

[Rafał]

This is not a problem the option "TLS, if available". In Thunderbird 2 I could only set the TLS and it worked correctly.

Comment 10

[rsx11m]

Based on your last comment, I don't quite understand which impact connection security has in your setup. Are you in fact applying the following settings:

1) LAN: STARTTLS without user name and password
2) WAN: STARTTLS *with* user name and password

If that's the only difference, this indeed is bug 534111/bug 524868.

Comment 11

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #8)
> In fact it can only send mail to port 587 Port 25 is blocked in the new year.
> But this is not a problem. Server supports the SMTP-AUTH just do not announce
> on the LAN. Is this a bug?.

In your log in comment #0, next is seen.
> #openssl s_client -host mail.xxxxxxxxxxx.com.pl -port 587 -starttls smtp
> ehlo test
> 250-mail.xxxxxxx.com.pl Hello test [xxxxxxxxxxxxx]
> 250-SIZE 52428800
> 250-PIPELINING
> 250-AUTH LOGIN PLAIN
> 250 HELP
This log is obtained via LAN you call instead of via WAN you call, isn't it?
Or this log is obtained via WAN you call, and no "250-AUTH LOGIN PLAIN" if via LAN you call?

> You can obviously see the error of the configuration, but only when the account setup in your mail client.

Not "error in your configuration". It's simply next;
  Tb's implementation is changed from Tb2 by Tb3,
  so change your configuration, please.

Comment 12

[WADA:World Anti-bad-Duping Agency]

Next log in comment #0(no 250 AUTH) is obtained by connection to 587?
If so, via LAN you call? If so, why different from log with manual connection to 587+StartTLS using openssl?   
 
> =============== Thunderbird 3 ======Send is FAIL
>(snip)
> 917 SMTP>> 220 TLS go ahead
>(snip)
> 917 SMTP<< EHLO [10.0.0.107]
> 917 SMTP>> 250-mail.xxxxxxxxxxxxx.com.pl Hello yyyyyy.xxxxxxxxxxxxx.com.pl
[10.0.0.107]
> 917 250-SIZE 52428800
> 917 250-PIPELINING
> 917 250 HELP
>(next)
> 917 SMTP<< QUIT
> 917 SMTP>> 221 mail.xxxxxxxxxxxxx.com.pl closing connection

Comment 13

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #8)
> a) port 587 (just enough)
> b) port 587 + STARTTLS (just enough)
> c) the port 587 + STARTTLS + username (AUTH) - (the server is not present in
> the EHLO SMTP-AUTH because he would not allow sending mail from the point of a,

AFAIK, message submission port(587) requests SMTP-AUTH.
Is your SMTP server properly set up "message submission port" support?
Or your SMTP server behaves as if port=25 when access via LAN you call?
(performane reason, server workload reduction, etc.)

Comment 14

[Rafał]

I've included logs again. I admit at the beginning of it too simplified
Again, the call logs from the WAN and LAN

================== Connection from the WAN WAN port 587 =================================

220 mail.xxxxxxxxxxxx.com.pl ESMTP to Exim
EHLO mail.xxxxxxxxx.com.pl
250-mail.xxxxxxxxxxxxxx.com.pl Hello yyyyyyyyyyyyy.xxxxxxxxxxxx.com.pl
[10.0.0.1]
250-SIZE 52428800
250-PIPELINING
250-STARTTLS
250 HELP

#openssl s_client -host mail.xxxxxxxxxxx.com.pl -port 587 -starttls smtp
250 HELP
ehlo mail.xxxxxxxxxx.com.pl
250-mail.xxxxxxx.com.pl Hello mail.xxxxxxxxxxx.com.pl [xxxxxxxxxxxxx]
250-SIZE 52428800
250-PIPELINING
250-AUTH LOGIN PLAIN
250 HELP

Here everything is clear and requires no comment

===================================================================================
================== Connection from the WAN LAN port 587 adress ip WAN=================================

220 mail.xxxxxxxxxxxx.com.pl ESMTP to Exim
EHLO mail.xxxxxxxxx.com.pl
250-mail.xxxxxxxxxxxxxx.com.pl Hello yyyyyyyyyyyyy.xxxxxxxxxxxx.com.pl [10.0.0.1]
250-SIZE 52428800
250-PIPELINING
250-STARTTLS
250 HELP


#openssl s_client -host mail.xxxxxxxxxxx.com.pl -port 587 -starttls smtp
250 HELP
ehlo mail.xxxxxxxxxxxx.com.pl
250-mail.xxxxxxx.com.pl Hello mail.xxxxxxxxxxx.com.pl [xxxxxxxxxxxxx]
250-SIZE 52428800
250-PIPELINING
250 HELP

This combination, although not present in the SMTP-AUTH still supports email clients who want to use this method. EHLO is that nothing means nothing prints. TB3 is assumed on the basis of the EHLO message that the server does not require AUTH which is not true. Server as the authentication method that best supports and authorizes no problem with the mail client.

===================================================================================
================== Connection from the WAN LAN port 587 adress ip LAN=================================

220 10.0.0.1 ESMTP to Exim
EHLO mail.xxxxxxxxx.com.pl
250-mail.xxxxxxxxxxxxxx.com.pl Hello yyyyyyyyyyyyy.xxxxxxxxxxxx.com.pl [10.0.0.1]
250-SIZE 52428800
250-PIPELINING
250-STARTTLS
250 HELP

#openssl s_client -host 10.0.0.1 -port 587 -starttls smtp
250 HELP
ehlo mail.xxxxxxxxxxxx.com.pl
250-mail.xxxxxxx.com.pl Hello mail.xxxxxxxxxxx.com.pl [xxxxxxxxxxxxxx]
250-SIZE 52428800
250-PIPELINING
250 HELP

This combination, although not present in the SMTP-AUTH still supports email clients who want to use this method. EHLO is that nothing means nothing prints. TB3 is assumed on the basis of the EHLO message that the server does not require AUTH which is not true. Server as the authentication method that best supports and authorizes no problem with the mail client.

===================================================================================
===================================================================================

The problem is that until now were set parameters TB permanently or port 587 + STARTTLS + AUTH. And it worked without any problem with the WAN and LAN. There was no need to constantly reorganize your mail client parameters. I can not imagine that a user on a laptop which is running either on the Internet or a corporate network is still przestawiał parameters depending on where it is located. There has never been any such problems with any mail client. TB3 first is unable to send a message with the adjusted parameters. It is not about the user to constantly change the configuration depending on whether work on the internet or extranet.

This e-mail server configuration is specially made so that clients mail from the LAN does not have to authorize (whether the desktop) to send the mail. The problem arises with laptops where the mail client is always set parameters.

Summing up
Mail should be sent wherever they plug into a network user (internet / extranet) without the constant changes in the parameters.

You rsx11m wrote in message
>>1) LAN: STARTTLS without user name and password
>>2) WAN: STARTTLS *with* user name and password

You WADA wrote in message
>>>>>>>>Not "error in your configuration". It's simply next;
>>>>>>>>  Tb's implementation is changed from Tb2 by Tb3,
>>>>>>>>  so change your configuration, please.

This forces the user to continuously change the parameters in your mail client. For me this error TB3

Comment 15

[Rafał]

I made a mistake in the description of the log sending again

================== Connection from the WAN port 587 =================================

220 mail.xxxxxxxxxxxx.com.pl ESMTP to Exim
EHLO mail.xxxxxxxxx.com.pl
250-mail.xxxxxxxxxxxxxx.com.pl Hello yyyyyyyyyyyyy.xxxxxxxxxxxx.com.pl
[10.0.0.1]
250-SIZE 52428800
250-PIPELINING
250-STARTTLS
250 HELP

#openssl s_client -host mail.xxxxxxxxxxx.com.pl -port 587 -starttls smtp
250 HELP
ehlo mail.xxxxxxxxxx.com.pl
250-mail.xxxxxxx.com.pl Hello mail.xxxxxxxxxxx.com.pl [xxxxxxxxxxxxx]
250-SIZE 52428800
250-PIPELINING
250-AUTH LOGIN PLAIN
250 HELP

Here everything is clear and requires no comment

===================================================================================
================== Connection from the LAN port 587 adress ip WAN=================================

220 mail.xxxxxxxxxxxx.com.pl ESMTP to Exim
EHLO mail.xxxxxxxxx.com.pl
250-mail.xxxxxxxxxxxxxx.com.pl Hello yyyyyyyyyyyyy.xxxxxxxxxxxx.com.pl [10.0.0.1]
250-SIZE 52428800
250-PIPELINING
250-STARTTLS
250 HELP


#openssl s_client -host mail.xxxxxxxxxxx.com.pl -port 587 -starttls smtp
250 HELP
ehlo mail.xxxxxxxxxxxx.com.pl
250-mail.xxxxxxx.com.pl Hello mail.xxxxxxxxxxx.com.pl [xxxxxxxxxxxxx]
250-SIZE 52428800
250-PIPELINING
250 HELP

This combination, although not present in the SMTP-AUTH still supports email clients who want to use this method. EHLO is that nothing means nothing prints. TB3 is assumed on the basis of the EHLO message that the server does not require AUTH which is not true. Server as the authentication method that best supports and authorizes no problem with the mail client.

===================================================================================
================== Connection from the LAN port 587 adress ip LAN=================================

220 10.0.0.1 ESMTP to Exim
EHLO mail.xxxxxxxxx.com.pl
250-mail.xxxxxxxxxxxxxx.com.pl Hello yyyyyyyyyyyyy.xxxxxxxxxxxx.com.pl [10.0.0.1]
250-SIZE 52428800
250-PIPELINING
250-STARTTLS
250 HELP

#openssl s_client -host 10.0.0.1 -port 587 -starttls smtp
250 HELP
ehlo mail.xxxxxxxxxxxx.com.pl
250-mail.xxxxxxx.com.pl Hello mail.xxxxxxxxxxx.com.pl [xxxxxxxxxxxxxx]
250-SIZE 52428800
250-PIPELINING
250 HELP

This combination, although not present in the SMTP-AUTH still supports email clients who want to use this method. EHLO is that nothing means nothing prints. TB3 is assumed on the basis of the EHLO message that the server does not require AUTH which is not true. Server as the authentication method that best supports and authorizes no problem with the mail client.

===================================================================================
The problem is that until now were set parameters TB permanently or port 587 +
STARTTLS + AUTH. And it worked without any problem with the WAN and LAN. There
was no need to constantly reorganize your mail client parameters. I can not
imagine that a user on a laptop which is running either on the Internet or a
corporate network is still przestawiał parameters depending on where it is
located. There has never been any such problems with any mail client. TB3 first
is unable to send a message with the adjusted parameters. It is not about the
user to constantly change the configuration depending on whether work on the
internet or extranet.

This e-mail server configuration is specially made so that clients mail from
the LAN does not have to authorize (whether the desktop) to send the mail. The
problem arises with laptops where the mail client is always set parameters.

Summing up
Mail should be sent wherever they plug into a network user (internet /
extranet) without the constant changes in the parameters.

You rsx11m wrote in message
>>1) LAN: STARTTLS without user name and password
>>2) WAN: STARTTLS *with* user name and password

You WADA wrote in message
>>>>>>>>Not "error in your configuration". It's simply next;
>>>>>>>>  Tb's implementation is changed from Tb2 by Tb3,
>>>>>>>>  so change your configuration, please.

This forces the user to continuously change the parameters in your mail client.
For me this error TB3

Comment 16

[Rafał]

Please refer to the 15 not 14 which is wrongly described.

Comment 17

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #15)
(Log-1)
> ================== Connection from the WAN port 587
> =================================
(Log-2)
> ================== Connection from the LAN port 587 adress ip
> WAN=================================
(Log-3)
> ================== Connection from the LAN port 587 adress ip
> LAN=================================

What is difference of environment among three logs.
What is "LAN port 587" and "WAN port 587"?

Port=587 was defined by RFC 2476. And SMTP-AUTH didn't look mandatory for 587, although many SMTP servers force SMTP-AUTH for 587.
> http://www.ietf.org/rfc/rfc2476.txt
Sorry for my misunderstanding.

As rsx11m says, I also think your case is port=587 version of bug 534111 comment #7.
At Campas or LAN environment, some SMTP servers seem to say "there is no need of SMTP-AUTH" by no AUTH response to EHLO, but "no AUTH response to EHLO" is "SMTP-AUTH is not supported yet" for Tb3.
What is meaning of "no AUTH response to EHLO"?

A possible/currently available solution is add-on like "SMTP switcher".
  (a) When WAN, use SMTP1 for which "use username and password" is checked.
  (b) When LAN, use SMTP2 for which "use username and password" is unchecked.
  (c) Switch between (a) and (b) is done by a few clicks.

"Fall back to no SMTP-AUTH(login), if no AUTH response but 'use username and password' is checked" should be enabled again?
If so, it should be optional?

Comment 18

[WADA:World Anti-bad-Duping Agency]

Quick summay of this bug's problem.
(A) SMTP's behaviar is special.
(A1) SMTP server doesn't return AUTH to EHLO, if secure connection or
     connection from internal. (via LAN in this bug's case)
(A2) SMTP server returns AUTH to EHLO if non-secure connection or
     connection from external. (via WAN in this bug's case)
(B) Tb's behaviour is changed by Tb3;
    No fall back to no SMTP-AUTH when "use username and password" is checked.
(C) Due to mismatch between (A) and (B), next happens.
    If "use username and password" is checked,   unable to send mail when (A).
    If "use username and password" is unchecked, unable to send mail when (B).

Similar issue can happen by change of Tb 3(no 'TLS, if avail' any more) on StartTLS/no-StartTLS by SMTP server according to non-secure/secure or external/internal connection. SMTP server which bug opener uses looks to behave specially on STARTTLS too;
  When connection from internal, SMTP server returns STARTTLS,
  but doesn't force StartTLS command use if internal connection(LAN). 
But it's not problem, because the SMTP server always returns STARTTLS and the SMTP server always accepts STARTTLS command properly. Then "Connection Security: StartTLS" of Tb3's setting always works well.

Comment 19

[Rafał]

You WADA wrote in message comment 17
>>What is difference of environment among three logs.
>>What is "LAN port 587" and "WAN port 587"?

Wanted me here just for the telnet port 587 from the LAN or WAN and communications sent by exim. Such shorthand. Sorry.

This is something very similar to bug 534111 comment 7

You WADA wrote in message comment 17
>>>A possible/currently available solution is add-on like "SMTP switcher".
>>>  (a) When WAN, use SMTP1 for which "use username and password" is checked.
>>>  (b) When LAN, use SMTP2 for which "use username and password" is unchecked.
>>>  (c) Switch between (a) and (b) is done by a few clicks.
That just would like to avoid switching. But in the current version of the TB3 can not be otherwise.

You WADA wrote in message comment 17
>>>"Fall back to no SMTP-AUTH(login), if no AUTH response but 'use username and
>>>password' is checked" should be enabled again?
>>>If so, it should be optional?
I understand it that way. If the option is selected username is TB should try to send a message with just such a setting. If someone would set a wrong turning it gets an error message from the mail server and message behind.

You WADA wrote in message comment 18
Yes, I agree with the conclusions. About something like it.

Comment 20

[rsx11m]

(In reply to comment #17)
> Port=587 was defined by RFC 2476. And SMTP-AUTH didn't look mandatory for 587,
> although many SMTP servers force SMTP-AUTH for 587.
> > http://www.ietf.org/rfc/rfc2476.txt

Actually, that RFC was obsoleted by 4409, which is more specific on SMTP-AUTH:

(Quoting http://tools.ietf.org/html/rfc4409#section-4.3 )
>  The MSA MUST by default issue an error response to the MAIL command
>  if the session has not been authenticated using [SMTP-AUTH], unless
>  it has already independently established authentication or
>  authorization (such as being within a protected subnetwork).

Thus, the case of requiring AUTH from an unprotected/untrusted network on 587 whereas not requiring it for a protected/trusted subnetwork on the same port is covered by the standard.

Comment 21

[WADA:World Anti-bad-Duping Agency]

Confirming.

Comment 22

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #0)
> Server Exim4
> STARTTLS+AUTH - WAN required
> STARTTLS+AUTH - LAN optional

"optional for AUTH" is misleading or wrong for LAN case.
Your server settup looks;
 (a) connection via WAN(connection from external network, from home):
     (a-1) Both Port=587 and Port=25 are supported
     (a-2) STARTTLS  : forces STARTTLS command use
     (a-3) SMTP-AUTH : forces SMTP-AUTH
 (b) connection via LAN(connection from internal network, at office building):
     (a-1) Both Port=587 and Port=25 are supported
     (a-2) STARTTLS  : returns STARTTLS for first EHLO.
                       forces STARTTLS command use
                       (STARTTLS is optional?)
     (a-3) SMTP-AUTH : no AUTH response to EHLO after STARTTLS.
                       - merely no need of SMTP-AUTH(as secure connection), 
                       - prohibits login by mail client(prohibits SMTP-AUTH),
                       - unable to enable SMTP-AUTH,
                       - cannot support SMTP-AUTH yet
(a-3) may be for server workload reason, server configuration reason, or server management reason.
As rsx11m says, RFC never prohibits (a-3).

For workaround in you case.
Rafał(bug opener), does your ITS/SMTP sever support SSL/TLS(port=465)?
If yes, can SSL/TLS(port=465) be a workaround of your case?

Comment 23

[Rafał]

(In reply to comment #22)
> (In reply to comment #0)
> > Server Exim4
> > STARTTLS+AUTH - WAN required
> > STARTTLS+AUTH - LAN optional
> 
> "optional for AUTH" is misleading or wrong for LAN case.
> Your server settup looks;
>  (a) connection via WAN(connection from external network, from home):
>      (a-1) Both Port=587 and Port=25 are supported
>      (a-2) STARTTLS  : forces STARTTLS command use
>      (a-3) SMTP-AUTH : forces SMTP-AUTH
>  (b) connection via LAN(connection from internal network, at office building):
>      (a-1) Both Port=587 and Port=25 are supported
>      (a-2) STARTTLS  : returns STARTTLS for first EHLO.
>                        forces STARTTLS command use
>                        (STARTTLS is optional?)
That is an option. But this is not a problem in this case.


> For workaround in you case.
> Rafał(bug opener), does your ITS/SMTP sever support SSL/TLS(port=465)?
> If yes, can SSL/TLS(port=465) be a workaround of your case?

No support port 465 on serwer

Comment 24

[Ken Smith]

I would think this is important. If people have upgraded to TBird 3, then they have a problem with the STARTTLS.  That means all upgrades are flawed and mail cannot be sent.... and the program cannot be used. 

My upgrade worked for about 4 days then all of a sudden I got the message that I could not connect.

I am not a "techy" I am one of the million users of Thunderbird who have opted for an alternative to Microsoft or the variety of web based programs available.

I also have problems with Lightening. I only mention this because this release, as innovative as it is, if it doesn't work, it is damaging the credibility of Thunderbird.

My work around has been to change the encryption to SLS/TLS.  For others it may be hotmail or Outlook Express.

Comment 25

[Dan Mosedale (:dmosedale, :dmose)]

I still need to finish triaging the results of the development meeting about the various SMTP issues, hoping to get to that in the next week or so.  I suspect that something to address this problem blocks, marking as needed to be sure it doesn't fall off our radar.

Comment 26

[Rafał]

Withdrew completely from the implementation of Tunderbird3. I have a couple of sites and some computers and I can not afford such problems not to mention the fact that users that tormented me. As long as I advise against its other users (still some phones) to install this version until you correct the error (other I had not noticed). At the moment I am at the stage of searching for an alternative because there's no sign of a change and I do not want to install outloka and similar products from the Microsoft stable.

Comment 27

[Dan Mosedale (:dmosedale, :dmose)]

We're resetting the blocking flag for 3.1 on this bug.  We have too many blocking-3.1 bugs, to the point where it doesn't mean much, and managing the list is making it hard to actually work on closing bugs, which helps no one.

Thunderbird 3.1's primary purpose is to allow us to offer a prompted major update to Thunderbird 2 users, to ensure their continued ability to safely use Thunderbird.  Thunderbird 2 is built on an outdated version of Gecko, and our long-term ability to maintain the users' safety for Thunderbird 2 users is limited.

If you think this bug meets the requirements below, please renominate with a detailed explanation of how it meets the following two criteria, and we will reconsider.  To qualify, this bug must either:

a) make the upgrade experience from TB2 very painful for a large number of users

or

b) be a new, reproducible, severe quality issue (eg dataloss, frequent crashes)

Just because this bug doesn't block TB3.1 doesn't mean it can't or won't make the release.  Once they're done with their blockers (if any), we encourage developers to keep working on non-blocking bugs, and to try to land them as early in the cycle as possible, as non-blocking bugs will become increasingly difficult to land in the later stages of the cycle.

Comment 28

[Rafał]

Testing Thunderbird version "Lanikai 3.1 Beta 1" (http://www.mozillamessaging.com/en-US/thunderbird/early_releases/downloads/)

working properly

Comment 29

[Rafał]

Applicable version of Firefox 3.1 beta 1
Sorry everyone. However, it is still bad. Does not work as expected.

Comment 30

[rsx11m]

I would have been surprised if it actually did - to the extent of my
knowledge, other than auto-sensing of secure authentication (bug 522633),
nothing really was checked in yet regarding SMTP authentication.

Comment 31

[Rafał]

Just updated the thunderbird, and for the first time I sent correctly. Then again I ran the program and no longer work properly. Just a little hurried and unnecessary confusion.
Still waiting for the version that will operate correctly + SMTP AUTH.

Comment 32

[Rafał]

Is there maybe something done on this bug? Have you forgiven repair this?

Comment 33

[Trapper]

Unfortunately TBird has had TLS issues almost since inception. Early TBird 1.x had a major TLS flaw in regards to POP3 checking. That same difficulty has also reappeared in 3.x. Checking POP3 with TLS always fails. Check again immediately thereafter and you can connect and will continue to be able to connect until you close TBird. Once you reopen TBird you again have to make a 2nd attempt before you get a successful connection.

Quite frankly I consider TBird development in the area of TLS to be inept. Others do not experience these issues. TBird developers need to design their application to meet and comply to the standards of the various email protocol RCF's. It would eliminate these issues that shouldn't exist in the first place.

Claiming to create safety and security by simply eliminating necessary functions is an unacceptable no-no. Figure things of this nature out prior to releasing your apps. They are critical for end users usage. It creates havoc for us when you don't. It's forcing users to use a less safe earlier version of TBird because your new "safer" version doesn't do what's required. When we are required to use TLS to get/send email we have no option but to remain unsafely in the past.

Sadly, this difficulty is easily resolved with a couple of easy code hacks. Unfortunately, as in 1.x we are going to be stuck with this TBird usability flaw again for an extended period. It should have been fixed immediately when discovered.

Comment 34

[David :Bienvenu]

If anyone has a test account that exhibits any of these problems, I'd be happy to try to fix it.

Comment 35

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #33)
> Unfortunately TBird has had TLS issues almost since inception. Early TBird 1.x
> had a major TLS flaw in regards to POP3 checking. That same difficulty has also
> reappeared in 3.x. Checking POP3 with TLS always fails.

To Trapper, who is comment poster of Comment #33:

This bug is for problem due to:
  - Server advertises SMTP-AUTH is usable if access from outside,
    and requets SMTP-AUTH if access from outside.
  - Server advertises SMTP-AUTH is not usable if access from inside,
    and requets no-SMTP-AUTH if access from inside.

Why your TLS related phenomenon can be relevant to this bug?
Do you understand bug summary of this bug?
Do you really read thru this bug before adding your comment to this bug?

Trapper, please open separate bug for your absolutely different problem from this bug, after searching well at B.M.O. for already opened bugs relevant to your problem via "Advanced Search".
Please note that here is B.M.O., never support forum.

Comment 36

[David :Bienvenu]

Has anyone tried TB 3.1? Someone was kind enough to give me a test account on an smtp server that only advertised auth=plain after STARTTLS and TB 3.1 tried auth plain after that.

Comment 37

[Rafał]

TB 3.1 does not work correctly. Have already lost hope that it will ever be repaired. I gave up and recommended a change to other mail programs, or staying with an old version of TB to all our customers.

Comment 38

[David :Bienvenu]

(In reply to comment #37)
> TB 3.1 does not work correctly. Have already lost hope that it will ever be
> repaired. I gave up and recommended a change to other mail programs, or staying
> with an old version of TB to all our customers.

Ah, well, thanks anyway. If anyone else is interesting in helping me diagnose these problems, please let me know.

Comment 39

[Rafał]

It seems to me that quite accurately described the problem of assuming that Bug. Is something unclear? As far as their abilities can help.

Comment 40

[David :Bienvenu]

(In reply to comment #39)
> It seems to me that quite accurately described the problem of assuming that
> Bug. Is something unclear? As far as their abilities can help.

Sorry, I can't really parse that. But what I need is a test account on a server that has the issue, so I can reproduce it and debug a fix.

Comment 41

[David :Bienvenu]

Attachment: proposed fix

Ben, what do you think of this? If the server doesn't advertise any auth, then we just try to send the message, even if the user has told us to use a username.

Comment 42

[David :Bienvenu]

I can't really think of an other way of fixing this without adding UI to set a pref that says "use username unless the server doesn't want one", which seems like silly UI. This won't fix the case where the server still advertises auth, but I don't think that worked before.

Comment 43

[David :Bienvenu]

Linux 3.2 pre try server build with patch here - http://s3.mozillamessaging.com/build/try-server/2010-06-23_17:[e-mail]-1277339956/[e-mail]2

(warning, there are a few known regression bugs with trunk builds, so they're not for everyday use)

Comment 44

[Ben Bucksch (:BenB)]

bienvenu, I basically think this is a WONTFIX. The server is misconfigured, it MUST accept authentication, per RFC 4409.

Even more so if the server requires authentication in some cases, it must accept it in the other cases, too.

If you take your stance that we shouldn't even try to auth when the server doesn't advertise auth, then the patch looks reasonable.
Just change the comment to "We don't need to auth, per pref, or the server doesn't advertise AUTH, so skip auth and send old HELO." I haven't tested the patch.

Comment 45

[Ben Bucksch (:BenB)]

Note that while RFC4409 allows to not *require* authentication when authorization was been established by other means like IP address, as in this case here, it still requires that SMTP-AUTH be *supported*:

Section 7:
> AUTH           Authentication                MUST      [SMTP-AUTH]
...
>   [SMTP-AUTH] allows the MSA to validate the authority and determine
>    the identity of the submitting user and MUST be supported by the MSA.

Comment 46

[Ben Bucksch (:BenB)]

Rafał, I was about to write to postmaster of one.pl, when I realized that it's some other server. Unfortunately, you have hidden the mail server that shows the problem, with "mail.xxxxxxxxxxxx.com.pl". I respect your privacy, but please tell which kind of server this is and who is responsible for this email server. If this is you company's server, please tell your administrator or the company that provides your email server that their server is broken and they should fix it to allow SMTP-AUTH in all cases.

Comment 47

[Ben Bucksch (:BenB)]

> I can't really think of an other way of fixing this without adding UI to set a
> pref that says "use username unless the server doesn't want one", which seems
> like silly UI.

Another option would be a UI-less pref. This would help until all the servers are fixed, but still encourage server admins to fix the servers.

Comment 48

[Rafał]

Probably not read from the beginning to what I wrote.

> Server Exim4
> STARTTLS+AUTH - WAN required
> STARTTLS+AUTH - LAN optional


Server supports:
1. From the WAN (Internet) forced STARTTLS + AUTH in the same order on port 587
(25 will be turned off for members). To send mail in your mail client I have
set the port 587, STARTTLS, and username (AUTH). Other options will not work.
So you want to.
2. From the LAN (extranet) to send mail without authorization or options
STARTTLS, AUTH on port 587 (25 will be turned off for members). To send a
message I have in your mail client set up (there are three possibilities /
options):
a) port 587 (just enough)
b) port 587 + STARTTLS (just enough)
c) the port 587 + STARTTLS + username (AUTH) - (the server is not present in
the EHLO SMTP-AUTH because he would not allow sending mail from the point of a,
b)


WADA fully described, my problem
> Your server settup looks;
>  (a) connection via WAN(connection from external network, from home):
>      (a-1) Both Port=587 and Port=25 are supported
>      (a-2) STARTTLS  : forces STARTTLS command use
>      (a-3) SMTP-AUTH : forces SMTP-AUTH
>  (b) connection via LAN(connection from internal network, at office building):
>      (a-1) Both Port=587 and Port=25 are supported
>      (a-2) STARTTLS  : returns STARTTLS for first EHLO.
>                        forces STARTTLS command use
>                        (STARTTLS is optional?)
>      (a-3) SMTP-AUTH : no AUTH response to EHLO after STARTTLS.
>                        - merely no need of SMTP-AUTH(as secure connection), 
>                        - prohibits login by mail client(prohibits SMTP-AUTH),
>                        - unable to enable SMTP-AUTH,
>                        - cannot support SMTP-AUTH yet


Besides strange that the error occurs only TB3 +.
I tested many other email programs and work perfectly.

rsx11m fully described, my problem
> Actually, that RFC was obsoleted by 4409, which is more specific on SMTP-AUTH:
> 
> (Quoting http://tools.ietf.org/html/rfc4409#section-4.3 )
> >  The MSA MUST by default issue an error response to the MAIL command
> >  if the session has not been authenticated using [SMTP-AUTH], unless
> >  it has already independently established authentication or
> >  authorization (such as being within a protected subnetwork).
> 
> Thus, the case of requiring AUTH from an unprotected/untrusted network on 587
> whereas not requiring it for a protected/trusted subnetwork on the same port > is covered by the standard.

Comment 49

[Ben Bucksch (:BenB)]

Rafal, I fully understood that. For RFC4409, see comment 45.
Rafal, please see comment 46.

Comment 50

[David :Bienvenu]

Attachment: proposed fix

Ben, this seems to be causing enough people issues that we might as well do what TB 2 and all other mail apps seem to be doing. It's not like it's some ancient mail server - dovecot is pretty common. So I would like to go forward - I've adjusted the comment a bit.

Comment 51

[Ben Bucksch (:BenB)]

Comment on attachment 453738 [details] [diff] [review]
proposed fix

bienvenu, the SMTP server in question is Exim (see logs above). Dovecot is an IMAP server.

(Nit: please take the extra space out at the start of the comment, before you commit)

r=BenB

I still have reservations about the approach, I'd rather use a hidden pref that people who have problems can enable. But given that you're sure it's the right approach, I concur.

Comment 52

[David :Bienvenu]

thx, fixed on trunk. I have slight reservations about this, but I think it's the best course available.

Comment 53

[David :Bienvenu]

Comment on attachment 453738 [details] [diff] [review]
proposed fix

getting on radar for 3.1.1

Comment 54

[Mark Banner (:standard8)]

Comment on attachment 453738 [details] [diff] [review]
proposed fix

a=Standard8. Please land on just comm-1.9.2 default and I'll sort out relbranch later

Comment 55

[David :Bienvenu]

pushed to comm-1.9.2 tip/default (http://hg.mozilla.org/releases/comm-1.9.2/rev/b2d0dbdd6e9d) - hope that was right...

Comment 56

[Mark Banner (:standard8)]

Checked into relbranch as well:

http://hg.mozilla.org/releases/comm-1.9.2/rev/ecea2f814778