Closed Bug 368611 Opened 18 years ago Closed 16 years ago

SMTP-TLS problems with sslio/MatrixSSL server

Categories

(Thunderbird :: General, defect)

Product:
Thunderbird
Component:
General
Platform:
x86
Windows XP
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED DUPLICATE of bug 392846
Milestone:
Thunderbird 3

People

(Reporter: mscott, Assigned: mscott)

References

Details

(Keywords: regression)

Attachments

(3 files, 1 obsolete file)

ssltap output: first send attempt (works)
3.49 KB, text/plain
Details
ssltap output: third send attempt (works)
2.25 KB, text/plain
Details
ssltap output: second send attempt (fails)
2.50 KB, text/plain
Details 
I've had several folks in the forums report problems sending mail in thunderbird 2. But 1.5.0.9 works just fine on the same profile. These users are all using SMTP over TLS. 

Soren is one of these users (cc'ed on the bug). He reported this problem with beta 1 and has confirmed it still exists in beta 2.

Soren got the following server side log:

2006-12-13 12:18:14 TLS error on connection from cpe.atm2-0-73126.0x535be69e.banxx4.customer.tele.dk ([172.16.0.216]) [83.91.230.158] (gnutls_handshake): Decryption has failed.

He also sent me a client SMTP log, but it doesn't show much:

-1223289168[8ce0eb0]: SMTP entering state: 0
-1223289168[8ce0eb0]: SMTP Response: 250-yoda.jido.dk Hello cpe.atm2-0-73126.0x535be69e.banxx4.customer.tele.dk [83.91.230.158]
-1223289168[8ce0eb0]: SMTP entering state: 0
-1223289168[8ce0eb0]: SMTP Response: 250-SIZE 52428800
-1223289168[8ce0eb0]: SMTP entering state: 0
-1223289168[8ce0eb0]: SMTP Response: 250-PIPELINING
-1223289168[8ce0eb0]: SMTP entering state: 0
-1223289168[8ce0eb0]: SMTP Response: 250-AUTH PLAIN LOGIN
-1223289168[8ce0eb0]: SMTP entering state: 0
-1223289168[8ce0eb0]: SMTP Response: 250-STARTTLS
-1223289168[8ce0eb0]: SMTP entering state: 0
-1223289168[8ce0eb0]: SMTP Response: 250 HELP
-1223289168[8ce0eb0]: SMTP entering state: 4
-1223289168[8ce0eb0]: SMTP entering state: 22
-1223289168[8ce0eb0]: SMTP Send: STARTTLS
-1223289168[8ce0eb0]: SMTP entering state: 0
-1223289168[8ce0eb0]: SMTP Response: 220 TLS go ahead
-1223289168[8ce0eb0]: SMTP entering state: 20
-1223289168[8ce0eb0]: SMTP entering state: 15
-1223289168[8ce0eb0]: SMTP Send: EHLO [172.16.0.216]
   

Nelson, is there a way to turn on client side logging for watching the handshake ?
Flags: blocking-thunderbird2+
Scott,  NSS supports both SSL 3.0 and SSL 3.1 (better known as TLS).
By default, both are enabled.  There are numerous servers out there that
work OK with SSL 3.0 but have troubles with TLS, especially with TLS 
hello extensions or TLS ECC cipher suites.  We call those servers 
"TLS intolerant".  In general, the solution for TLS intolerant servers
is to tell NSS to disable support for SSL 3.1, and only support SSL 3.0.

Note well that TLS (SSL 3.1) and "STARTTLS" are very different things.
TLS is a version of the SSL protocol.  STARTTLS is a feature of SMTP and
IMAP that allows SSL 3.x (that is, SSL 3.0 or 3.1) to be negotiated in the 
middle of an SMTP or IMAP connection.  TLS intolerant servers are intolerant
of SSL version 3.1, not of the "STARTTLS" feature of SMTP + IMAP.

PSM has a feature, used in FireFox, that detects failure to complete a 
handshake when TLS is enabled, and then disables TLS and tries again.  
I suspect that works for connections that use TLS from the beginning of the 
connection and not for those that use STARTTLS to negotate SSL/TLS in 
mid-connection.

Does TBird have any way to disable the use of SSL 3.1 (TLS), and hence to
force the use of SSL 3.0?  Perhaps by editing a preferences file?

If so, I'd suggest that the afflicted folks should try disabling SSL 3.1.
If that works, it is both diagnostic and curative.  It tells them that 
they're dealing with a TLS intolerant server, and it stops trying to ask
that server to use TLS, working around its intolerance.

If disabling TLS doesn't fix it, then I'd suggest using the ssltap program
next.  -DDEBUG builds of libSSL do have a tracing feature built in that can
be activated through environment variables, but ssltap output is likely to
be a faster route to the cause.
Nelson, thanks for the great explanation about SSL, TLS and STARTTLS.

I've e-mailed Soren (cc'ed on this bug) asking him to try turning off:

security.enable_tls

in Thunderbird 2's config editor. That should turn off SSL 3.1 if I'm understanding things correctly.
Dang, Soren set enable_tls to false and he still couldn't send to the SMTP server over STARTLS in thunderbird 2.


"I've done as you asked and it makes no difference: still can't send mail. (but as stated earlier: if I choose to not use TLS and go un-encrypted there's no problem) As an added bonus, the option prevented me to contact the IMAP servers I have via TLS."


comment 3 makes we wonder if perhaps that pref actually does something other
than what its name suggests, such as perhaps disabling both SSL 3.0 and 3.1,
or disabling STARTTLS instead of disabling versino 3.1.  
I wonder if this is another manifestation of the problem described in 
bug 352530 comment 23 though 25.  That bug describes an IMAP problem.
This bug is about SMTP.  Might both problems have the same cause?
(In reply to comment #4)
> comment 3 makes we wonder if perhaps that pref actually does something other
> than what its name suggests, such as perhaps disabling both SSL 3.0 and 3.1,
> or disabling STARTTLS instead of disabling versino 3.1.  

Nelson, preference value security.enable_tls directly corresponds to 
      SSL_OptionSetDefault(SSL_ENABLE_TLS, enabled);

I seem to be unable to reproduce the problem.
Scott, could you please point me to the forum post?
Maybe I can comment there directly.

From the log in comment 0, I conclude you are using a connection to yoda.jido.dk port 25, setting TLS (the third checkbox in smtp prefs).

That server is using an untrusted cert, so we get cert warnings. Because of bug 368126 I would like to ask that you try to dismiss both cert warning dialogs in < 5 seconds, to ensure we don't disconnect. Hope to fix bug 368126 soon.

If I do the above, I get a reasonable response from the mail server: relay denied.

Here is my set up in detail:
- latest 1.8 branch
- pop mail account, details don't matter, mail fetching disabled
- smtp server settings:
    server name: yoda.jido.dk
    port: 25
    use name and password: NOT checked
    secure connection: TLS (3rd button)

my identity:
- your name: Thunderbird Tester
- email: kaie@kuix.de

composing email:
- to: test@test.com
- subject test, body test

hitting send

I'm getting this answer:
- an error occurred
- mail server responded: relay not permitted.

In order to ensure we are really going through SSL/TLS, I changed the source in nsSSLThread.cpp to dump the application data being transfered passed on to libssl and received from it.

I get this:

SSLDATA fd 0x17e1b40 RECEIVE < 66 bytes
SSLDATA 220 yoda.jido.dk ESMTP Exim 4.50 Fri, 02 Feb 2007 21:35:58 +0100..
SSLDATA fd 0x17e1b40 SEND > 21 bytes
SSLDATA EHLO [192.168.2.21]..
SSLDATA fd 0x17e1b40 RECEIVE < 148 bytes
SSLDATA 250-yoda.jido.dk Hello p579b6f35.dip.t-dialin.net [87.155.111.53]..250-SIZE 52428800..250-PIPELINING..250-AUTH PLAIN LOGIN..250-STARTTLS..250 HELP..
SSLDATA fd 0x17e1b40 SEND > 10 bytes
SSLDATA STARTTLS..
SSLDATA fd 0x17e1b40 RECEIVE < 18 bytes
SSLDATA 220 TLS go ahead..
SSLDATA fd 0x17e1b40 SEND > 21 bytes
SSLDATA EHLO [192.168.2.21]..
SSLDATA fd 0x17e1b40 RECEIVE < 134 bytes
SSLDATA 250-yoda.jido.dk Hello p579b6f35.dip.t-dialin.net [87.155.111.53]..250-SIZE 52428800..250-PIPELINING..250-AUTH PLAIN LOGIN..250 HELP..
SSLDATA fd 0x17e1b40 SEND > 35 bytes
SSLDATA MAIL FROM:<kaie@kuix.de> SIZE=375..
SSLDATA fd 0x17e1b40 RECEIVE < 8 bytes
SSLDATA 250 OK..
SSLDATA fd 0x17e1b40 SEND > 25 bytes
SSLDATA RCPT TO:<test@test.com>..
SSLDATA fd 0x17e1b40 RECEIVE < 25 bytes
SSLDATA 550 relay not permitted..


composing 
Thanks for helping us look into this Kai/Nelson.

FYI, the original forum post came from here:

http://forums.mozillazine.org/viewtopic.php?p=2647759#2647759


Kai,

I have told thunderbird I trust the cert long time ago so I'm not asked about it after having upgraded to tb2.

The mail server is set up so you need to use username and password to send mail. If you want, I can give a username and password to test. Just write me at line at tempadd01@jido.dk within the next few days (after that the address is deleted) and I'll send them to you.

Cheers,
Soren
Søren gave me a test account.
In all my tests it works perfectly!

Søren, could you please try a fresh profile?

Maybe there is something in your old profile that causes a problem?
If a fresh profile works, we'll have to compare them.
I have isolated the problem! It is concerned with digital signing.

For each of the following steps I checked if SMTP with TLS worked, and then I restart TB2b2 and checked again.

1. I used a fresh profile (and SMTP worked)
2. I imported my certificate for digital signing (worked)
3. I imported my old certificate which has another name but *hasn't expired* - and now SMTP *doesn't* work!
4. I delete the old certificate where I have a different name - and now SMTP works again!
5. I import an even older certificate of mine with my old name but which *is expired* - and SMTP still works!

The culprit seems to be another certificate that is still valid but where I have a different name (marriage). This is apparently not a problem in TB 1.5.0.x.

Can this be fixed?
(In reply to comment #11)
> I have isolated the problem! It is concerned with digital signing.
> [...]
> 1. I used a fresh profile (and SMTP worked)

Just tested it with my old profile, and it still works if I delete the still valid certificate of mine where I have a different name.
Nelson, how does the security code figure out which cert to use for TLS? Or is that even relevant?
Time is running out for TB 2.0 - TB doesn't even know about different certs when doing TLS, so this would have to be at a lower level. Are you sure this worked in 1.5.0.x?
Yes, I'm completely sure that it works in 1.5.0.x, and also that it doesn't work in 2.0.
And I should add that the behaviour is the same on both Windows and Linux.
Is this fixed by the checkin for bug 370136?
It's still there (version 4 March)
Bill, it might be - cc'ing Kaie. If so, we might really want that patch for TB 2.0
kaie == kengert
I were cc'ed already ;-)

It's unfortunate that I'm still not able to reproduce this reliably :-/

We have some other reports about strange SSL behaviour.
I hope I will be able to get some traction soon.

d'uh, sorry. Anyway, our code freeze is today - by traction, do you mean the patch on the trunk isn't just going to land on the branch?
IMHO, you should really really take the fix for bug 370136 for a TB 2 release, even if that fix is unrelated to this bug.

Bug 370136 is wanted for FF 2.0.0.x anyway, so as soon as I get approval I'm ready to land it.

By "traction" I mean, "what is the reason some users experience trouble with SSL".

I know that bug 370136 only affects users who own at least 2 personal certificates, but we have SSL failure reports from users, who do not own personal certs, and I don't have yet traction on those. Can't reproduce yet.

I wonder if the cases without multiple personal certificates are running into the  issue reported in bug 365898.
Moving this off the blocker list. We'd be interested in fixing this in a point release if we figure out what's going on and the patch looks safe.
Flags: blocking-thunderbird2+ → blocking-thunderbird2-
Target Milestone: --- → Thunderbird 3
I am having some problems with STARTTLS too, as reported in bug 372354. Kai Engert said in comment #7 that he tweaked some source file in order to see at what point the conversation was failing. I was wondering whether I can do something similar or look into some other place in order to see if my problem somewhat related to this bug.
(In reply to comment #26)
> I am having some problems with STARTTLS too, as reported in bug 372354.

It would seem that my problems are not tied to STARTTLS. They might be related to the interpretation of a wrong anser to an EHLO command, but this is not the proper bug in which to discuss this.

Sorry for the intrusion.
Blocks: 377481
Bug 377481 might be a duplicate of this one.

But I don't know yet. The two bugs are only similar.
For now I've added a dependency only.
I can offer access to a server where the problem is reproducible with thunderbird 1.5.0.10 or 2.0.0.0.
With tb 1.5.0.9 everything works.

I get the error every second mail I send.
What I do to reproduce:
start tb
send an email via smtp over ssl
no error and the mail is delivered
send another mail
connection error
resend and it works

In tb log I get the first complete successful log and then only one line:
-1222989056[8d2feb0]: SMTP Connecting to: smtp.mydomain.it

In the server log:
2007-07-05 16:17:35.405412500 sslio[26939]: fatal: ssl decode error: illegal par
ameter
2007-07-05 16:17:35.405585500 sslio[26939]: info: bytes in: 188
2007-07-05 16:17:35.405713500 sslio[26939]: info: bytes ou: 0

If I switch from sslio to stunnel-tls it always works.

Kaie, any chance you can take Filipo up on his offer and try this problem out using his SMTP server?
For me this problem also exist but I don't see any clue how reproduce it it's just randomly can't send message if TLS enabled on SMTP
Filippo, thank you very much.

I am finally able to reproduce this bug. On both Windows XP and on Linux 64 bit, using the latest Thunderbird 2.0.0.6

What I did is:
- fresh profile
- setup access to my own imap account
  (IMAP/SSL, do not check on startup)
- sent messages will go to the sent folder on the imap server by default
- setup smtp/ssl to the account that Filippo provided
- permanently trust the invalid cert used by that server
- allowed thunderbird to stored the passwords for both 
  test imap and test smtp accounts
- set up imap account to store its templates in local folders
- create a message, sent to my own account, subject "test", body "test",
  saved this message in the local templates folder

With this preparation, I can always reproduce using the following steps:
- start thunderbird
- click local templates folder, click message, edit as new
- send message, works
- click local templates folder, click message, edit as new
- send message, failure!

I disabled the "copy to sent folder" option, this should ensure that the SMTP/SSL is the only kind of SSL used during this application session.
I can still reproduce that way.

Comment on attachment 276876 [details]
ssltap output: second send attempt (fails)

sigh, attached wrong file for second attempt...
Attachment #276876 - Attachment is obsolete: true
The correct sequence of attachments is:

1st attempt: attachment 276875 [details]
2nd attempt: attachment 276879 [details]
3rd attempt: attachment 276877 [details]

In the 1st attempt, client sends a SSL hello v3.1 
                and server responds with a hello v3.0 (works)

In the 2nd attempt, client sends a SSL hello v3.0 
                and server responds with a SSL protocol error (fails)

In the 3rd attempt, client sends a SSL hello v2
                and server responds with a hello v3.0 (works)


If I understand correctly, this server claims it wants to speak SSL v3.0, but is unable to accept a client hello of the same version?

Nelson?

FYI, this is NSS version 3.11.5.

I tested using the latest 3.11.7+ snapshot, same bug.

Interesting.  
Here are some observations, a question and some suggested experiments.

This table summarizes the 3 connection attempts:

#  hello   hello    Sess ID  ECC      Extensions
   format  version  length   present  Present
-  ------  -------  -------  -------  ----------
1   3.x      3.1       0       yes       yes
2   3.x      3.0      32       yes       yes
3     2      3.0       0        no       N/A

It is possible that the server is failing the second attempt because
the server believes that extensions should not be present when the 
hello version is 3.0.  This would be a server error, IMO, but if this is
very common, we could work around it by suppressing ECC cipher suites
and hello extensions when we're not attempting to negotiate 3.1, like
NSS does when the client app has disabled TLS.  

In some sense, the behavior of the second attempt is similar to what
we do for the TLS-intolerant fall-back case in the browser.  We attempt 
to negotiate only 3.0 on the second try, not because of failure of the 
first attempt, but rather because the first attempt negotiated only 3.0.  
In the browser, if the first attempt had failed, the browser would then
disable TLS and try again.  Disabling TLS would cause us NOT to send the
ECC cipher suites nor the hello extensions in the client hello record.

I wonder what would happen if we 
a) sent 3.1 in the second client hello record, so that it was the same
   as the first hello record except for the non-zero Session ID.
b) did not send ECC cipher suites nor hello extensions in the second 
   attempt.   We can try that through preference settings (I think).

Q) How did TB come to try this progression of connections?  
Did it decide that the server was TLS intolerant after the second failure?
or did you change the prefs before the third try?

I ask because, last I knew, TB did not do automated TLS-intolerance recovery.
Only the browser did.  Has that changed?  

We probably should not declare TLS intolerance if connection attempts fail
that were not attempting to negotiate TLS. I don't know if the application
can determine that NSS tried TLS or not.  

Suggested experiments:
a) repeat the above 3 tests, and then do a 4th attempt and capture it with
SSLtap also.  I wonder if it will fail in the same way as the second 
attempt.

b) Disable all the ECC cipher suites, and repeat the 3 (or 4) tests, 
capturing with ssltap again.  I wonder if that will cure it. 
It should be possible to disable them through preferences.

c) enable all the ECC cipher suites, but disable TLS, and repeat the 
tests with SSLtap.  
Nelson, I have not yet fully read your comment, I will do so next.

But I wanted to let you know the results of an experiment I just completed.

I hacked libssl to never send hello extensions:

Index: ssl3ecc.c
===================================================================
RCS file: /cvsroot/mozilla/security/nss/lib/ssl/ssl3ecc.c,v
retrieving revision 1.3.2.11
diff -u -5 -r1.3.2.11 ssl3ecc.c
--- ssl3ecc.c   4 Jan 2007 17:48:31 -0000       1.3.2.11
+++ ssl3ecc.c   16 Aug 2007 01:58:13 -0000
@@ -1369,11 +1369,11 @@
 ssl3_CallHelloExtensionSenders(sslSocket *ss, PRBool append, PRUint32 maxBytes,
                                const ssl3HelloExtensionSender *sender)
 {
     PRInt32 total_exten_len = 0;
     int i;
-
+return 0;
     if (!sender)
        sender = &clientHelloSenders[0];

     for (i = 0; i < MAX_EXTENSION_SENDERS; ++i, ++sender) {
        if (sender->ex_sender) {


With this change, I confirmed using ssltap, the extensions do not get sent, neither with v3.1 nor with v3.0

In the second connection, the client sends the v3.0 hello without extensions, and the server accepts it, no failure.

In the third connection attempt, the client still uses the v3.0 hello.

(In reply to comment #41)
> 
> It is possible that the server is failing the second attempt because
> the server believes that extensions should not be present when the 
> hello version is 3.0.  This would be a server error, IMO,

This seems to be the case, as my experiment showed.


> but if this is
> very common, we could work around it by suppressing ECC cipher suites
> and hello extensions when we're not attempting to negotiate 3.1, like
> NSS does when the client app has disabled TLS.  

This sounds like a reasonable strategy.


> I wonder what would happen if we 
> a) sent 3.1 in the second client hello record, so that it was the same
>    as the first hello record except for the non-zero Session ID.

If you're still interested in the results of this experiment, please let me know how I should do it. I could hack the handshake code to unconditionally send 3.1


> b) did not send ECC cipher suites nor hello extensions in the second 
>    attempt.   We can try that through preference settings (I think).

I'm not aware of a preference that can suppress hello extensions separately.

We can enable SSL v2, but this will force us to use a v2 hello (we don't want that).

I could disable all the individual ECC cipher prefs, but would that automatically disable the sending of hello extensions? I guess it will not. I guess it will still send the server-name-indication-hello?

But I guess the experiment I described in the previous comment already answers your question.


> Q) How did TB come to try this progression of connections?  
> Did it decide that the server was TLS intolerant after the second failure?
> or did you change the prefs before the third try?

No.
I manually told TB to send a message 3 times.
First message got sent correctly.
Second send attempt failed.
Third attempt worked again.


> I ask because, last I knew, TB did not do automated TLS-intolerance recovery.
> Only the browser did.  Has that changed?  

In this scenario we are not seeing an automatic TLS intolerance retry.

But in general I think we do that retry for any SSL connection attempts, not limited to https. I can look that up if it's still relevant for this bug, but I think it's not.


> We probably should not declare TLS intolerance if connection attempts fail
> that were not attempting to negotiate TLS. I don't know if the application
> can determine that NSS tried TLS or not.  

Agreed, and we don't do that. PSM knows that it had TLS enabled when it constructed the failing socket. But this is not the case here.

Note: Of course I reverted my hack before trying these tests, I'm testing with the original code.


(In reply to comment #41)
> Suggested experiments:
> a) repeat the above 3 tests, and then do a 4th attempt and capture it with
> SSLtap also.  I wonder if it will fail in the same way as the second 
> attempt.

I had previously tested that the 4th and 5th attempt to send work, no more failures.
Now I looked what happens with ssltap.

To my surprise, on the 4th and 5th attempt, NSS is sending a v3.0 client hello, no ECC ciphers, no hello extensions!

So libssl seems to be smart and is learning that ecc and hello extensions should not be used on that server.

But I'm surprised that libssl uses a v3.0 hello in the 4th and 5th connections, even though it had use a v2 hello in the 3rd connection. Just want to make sure you are aware of this, it might make sense and, well, it's working.

> b) Disable all the ECC cipher suites, and repeat the 3 (or 4) tests, 
> capturing with ssltap again.  I wonder if that will cure it. 
> It should be possible to disable them through preferences.

I disabled all cipher prefs whose names contain "ecdh" (includes "ecdhe").

Yes, indeed, that works, too.
1st: v3.1 hello, no extensions
2nd, 3rd, 4th: v3.0 hello, no extensions



> c) enable all the ECC cipher suites, but disable TLS, and repeat the 
> tests with SSLtap.  

I had already tested this earlier, having the ECC ciphers enabled (default setting), disabled TLS. All attempts worked.

ssltap tells me: On all attempts, we always use v3.0 hello (including in the initial attempt), never send ecc ciphers, never send hello extensions

The final (last) SSL 3.0 internet draft specification, 
<draft-freier-ssl-version3-02.txt> says, in 5.6.1.2 Client hello, page 24:

   Forward compatibility note:
                  In the interests of forward compatibility, it is
                  permitted for a client hello message to include
                  extra data after the compression methods.  This data
                  must be included in the handshake hashes, but must
                  otherwise be ignored.

It MUST be included in the handshake hashes, but MUST otherwise be ignored.
Not used to cause a fatal error, but IGNORED.

The server with which this "bug" is reproduced clearly does NOT ignore
the "extra data after the compression methods", which is now formally known
as client hello extensions.  Thus it is not compliant with the spec.
> The server with which this "bug" is reproduced clearly does NOT ignore
> the "extra data after the compression methods", which is now formally known
> as client hello extensions.  Thus it is not compliant with the spec.

There are no client hello extensions in SSL 3.0, so it seems to me that the client is not compliant either.

Note also that there are two different server implementations involved. Soren refers in the forum thread to a gnu-tls based server. The server provided by Filippo uses matrixssl. I don't think it's safe to assume that both server SSL implementations have the same client hello parsing bug.

Nelson in comment #41 concentrates on "ECC cipher suites" (which I take to be "compression_methods" from the draft) and hello extensions, but the non-zero session ID might deserve more scrutiny - the only differences between the first (untimately successful) 3.1 client hello and the unsuccessful 3.0 client hello are the version number and the session ID:

--- /home/charlieb/ch-3.0       2007-08-17 12:12:16.000000000 -0400
+++ /home/charlieb/ch-3.1       2007-08-17 12:12:01.000000000 -0400
@@ -1,19 +1,19 @@
-Connection #2 [Thu Aug 16 02:26:21 2007]
+Connection #1 [Thu Aug 16 02:25:54 2007]
 Connected to nethservice.nethesis.it:465
 --> [
-(156 bytes of 151)
-SSLRecord { [Thu Aug 16 02:26:21 2007]
+(124 bytes of 119)
+SSLRecord { [Thu Aug 16 02:25:54 2007]
    type    = 22 (handshake)
-   version = { 3,0 }
-   length  = 151 (0x97)
+   version = { 3,1 }
+   length  = 119 (0x77)
    handshake {
       type = 1 (client_hello)
-      length = 147 (0x000093)
+      length = 115 (0x000073)
          ClientHelloV3 {
-            client_version = {3, 0}
+            client_version = {3, 1}
             random = {...}
             session ID = {
-                length = 32
+                length = 0
                 contents = {..}
             }
             cipher_suites[28] = {
@@ -60,11 +60,51 @@
 }
 ]
 <-- [
-(7 bytes of 2)
-SSLRecord { [Thu Aug 16 02:26:21 2007]
-   type    = 21 (alert)
+(994 bytes of 74, with 915 left over)
+SSLRecord { [Thu Aug 16 02:25:54 2007]
+   type    = 22 (handshake)
    version = { 3,0 }
-   length  = 2 (0x2)
-   fatal: illegal_parameter
+   length  = 74 (0x4a)
+   handshake {
+      type = 2 (server_hello)
...

Kai> To my surprise, on the 4th and 5th attempt, NSS is sending a v3.0 client
Kai> hello, no ECC ciphers, no hello extensions!
Kai>
Kai> So libssl seems to be smart and is learning that ecc and hello extensions
Kai> should not be used on that server.

Should they *ever* be used with a v3.0 client hello? 

The SSL 3.0 specification clearly says what it says.
There are MANY flawed implementations in the wild, and the newer and 
less well known ones tend to have more flaws than the older ones.  
The existece of two flawed implementations (out of MANY implementations
of SSL) does not mean that the specficiation is to be ignored.  

BTW, any correct SSL 3.x parser of client hello messages will not 
confuse cipher suites numbers with compression method numbers because
both have explicit lengths.  A parser that treats unrecognized cipher 
suite numbers as compression methods needs repair.  
Kai's experiments, comments 32-40 show clearly and unequivocally that the
server he was testing is intolerant of TLS extensions when the version
number in the header is 3.0.  Disabling TLS extensions (by disabling TLS)
will completely cure that, because NSS sends no TLS extensions when TLS is
disabled.  In comment 54, Kai confirmed that disabling TLS eliminated the
problem for this TLS-intolerant server.

But, back in comments 2-6, it was reported that Soren set the preference 
to disable TLS, but was still unable to connect to his server.  This tells 
me that either (a) he was having a different problem than the one Kai 
observed in comments 32-40, or (b) despite Soren's attempts, TLS was not
actually disabled.  This makes me wonder: Did Kai test with the same 
server (and same version of that server) and Soren did in comments 2-6 ?  

If not, then there are probably multiple separate issues being conflated 
in this bug (ignoring comment 11, which is clearly a separate issue).
> The SSL 3.0 specification clearly says what it says.

It does. It says there that a 3.0 client hello does not contain hello extensions, so I don't know why you send them. 

> There are MANY flawed implementations in the wild, and the newer and 
> less well known ones tend to have more flaws than the older ones.  
> The existece of two flawed implementations (out of MANY implementations
> of SSL) does not mean that the specficiation is to be ignored. 

Sure. I'm not sure what you're saying here though. Is there any specification which says TB *should* send hello extensions in a v3.0 hello?

> BTW, any correct SSL 3.x parser of client hello messages will not 
> confuse cipher suites numbers with compression method numbers because
> both have explicit lengths. 

Agreed. I am actually the one confused by "ECC ciphers", not any parser. They're just included in the vector of cipher_suites, and the server will ignore them, because it won't support them, right? I don't see any evidence here that TB2 proposing them is provoking the loss of communication.


> This makes me wonder: Did Kai test with the same server (and same
> version of that server) and Soren did in comments 2-6 ?  

See comment #47. Soren's server log (quoted in Scott's first post here) shows that gnutls was used in his server. Filippo's server, which Kai tested against, uses a matrixssl based server process.
In reply to comment 51:
The observations in comment 51 make it clear that there are two separate
sets of problems here, with two different implementations of SSL/TLS,
and both have been conflated into this bug.  They should be separated
into separate bugs.  

In this bug, we now have a detailed analysis of the problem seen with 
Filippo's matrixssl server.  I think we have no comparable analysis for 
Soren's server, but we know their problems are different, and the 
workaround for Filippo's server will not help Soren's.  

I leave it to mscott or Kai to decide how to split this bug.
I have a patch for Kai to try with Filippo's server, but I will wait 
until this bug is split before attaching it.
Kai wrote to me that Filippo's server uses "sslio".  
I don't know if that's the same thing as "matrixssl" or not.  

I propose splitting this bug into two bugs, one of which is named
"Problems with SMTP over TLS and GnuTLS servers" and the other is named
"Problems with SMTP over TLS and MatrixSSL servers".  (or sslio servers).
(In reply to comment #53)
> Kai wrote to me that Filippo's server uses "sslio".  
> I don't know if that's the same thing as "matrixssl" or not.  
From sslio man page (http://smarden.org/ipsvd/sslio.8.html):
"The sslio program uses the SSLv3 implementation of the matrixssl library."
I've identified the bug in matrixssl, and forwarded a patch to its maintainers. matrixssl was indeed failing to ignore hello extensions if the client hello requested v3.0.

FTR, note that matrixssl is available under two different licenses. There is a GPL version which implements SSL 3.0, but not TLS (SSL 3.1). That's the one used by sslio, which exhibited the communication problems with TB2. matrixssl is also available under a commercial license, and the commercial version implements TLS. The communication problem with TB2 probably isn't seen with that version, as there would be no fallback to v3.0.

I don't know how widely deployed either version of matrixssl is, but it's not particularly obscure.


(In reply to comment #52)
> In reply to comment 51:
> The observations in comment 51 make it clear that there are two separate
> sets of problems here, with two different implementations of SSL/TLS,
> and both have been conflated into this bug.  They should be separated
> into separate bugs.  

Well, I think it should be fine to keep a single Thunderbird tracking bug, since the symptoms of both problems were the same.

Since the issue that we found will be fixed in NSS, I propose we file a new NSS bug for the issue we have clearly identified. We can later make a note in here, that Filippo's problem is fixed, while Soren's original report is still not fixed. I guess that's simpler than cloning this bug.


> In this bug, we now have a detailed analysis of the problem seen with 
> Filippo's matrixssl server.  I think we have no comparable analysis for 
> Soren's server, but we know their problems are different, and the 
> workaround for Filippo's server will not help Soren's.  
> 
> I leave it to mscott or Kai to decide how to split this bug.
> I have a patch for Kai to try with Filippo's server, but I will wait 
> until this bug is split before attaching it.


I'd think we should have a separate NSS bug and attach your patch there.
Depends on: 392846
I filed bug 392846 for the proposal to never send hello extensions with SSL v3.0 which should fix the issue with the matrixssl server.

I propose all future discussion around that issue shall happen in bug 392846.

I propose we keep this bug open to continue the issue that Soren initially reported. In the hope that we will get new evidence.

Current status is: Soren gave me access to his server a while ago. I was never able to reproduce his problem. With the new knowledge around the matrixssl behavior, I tried to repeat my tests, but unfortunately I can no longer reach Soren's server.

Thanks to Soren, who just gave me back access to his test server.

I still have no problems sending mail using that server. Thunderbird uses SMTP with STARTTLS on port 25, it sends a v3.1 hello with extensions, and the server responds with a v3.1 response.

Second and third attempt to send mail work fine, too.

One thing that is different from "usual" handshakes, I think the server asks for a client auth cert, and it sends a big list of CA cert issuers (about 100) that it will accept.

So I'm really sorry I still can't reproduce.

Soren, I guess you still see this bug and are still unable to send mail using your server?


I wonder why we did not try that before, but let's try to debug your SSL/TLS connection.

Please obtain the "ssltap" program that comes with binary distributions of NSS. Let me know if you need help to get it.

Then start ssltap with the following command line:

ssltap -s -p 1925 -l YOURSERVERHOST:25

In Thunderbird, be sure to configure your smtp server to use host "127.0.0.1" as the mail server hostname and 1925 as the port number.

When you send mail, you'll get a hostname mismatch, this is of course expected. 

Repeat sending mail until you run into an error message. Now look at the output created by ssltap. We are interested in the trace of the most recent connection, that section should start with a header like:

Connection #2 [Mon Aug 20 09:40:41 2007]
Connected to ...:25

Please attach a file with that log.

kaie> Soren, I guess you still see this bug and are still unable to send mail kaie> using your server?

 *No, the problem's gone!*

I haven't used it for a while so I didn't check regularly, but I've just checked and it works flawlessly for me now (also upgraded to 2.0.0.6 from some earlier 2.0.0.x version). It used to be that it didn't work when I had an older but valid certificate installed with a different surname but same e-mail address. No longer so.

Soren, it's great news that the bug is fixed for you.

As your problem was clearly related to your installed personal certificates, I think this must have been fixed by our patch for bug 370136.

Although you had talked about that bug already in comment 17 and comment 18, I believe the version you tested with wasn't new enough. In comment 18 you stated that you tested a version from March 04, but I checked in the fix to Thunderbird 2 on March 05, so the earliest version with that bug fixed would have been a nightly build of March 06.


So, the problem that Filippo reported is the only one remaining.

I propose we keep this bug open until a fix for bug 392846 has been picked up by Thunderbird 2.0.0.x
It might take a while until that happens, because it will require Thunderbird to pick up a newer release of NSS.

Hi Nelson/Kai.  

smtp-TLS problems are a big deal for us (a few thousand PCs).  Our server is sendmail 8.14 using openssl.  Everything here requires secure connections.

Our people who are on v2 are encountering intermittent send problems - some have absolutely no problem at all, but some consistently have trouble. I can offer to assist in the identification and resolution of any SSL/TLS problems, including this bug.  In particular I should like to test a build of what you've got from bug 392846 to seen if solves some of our existing problems.

Note - I am greatly concerned about timely delivery of SSL related fixes  given that: a) thunderbird v2 is now in full automatic distribution and b) the gentleman asked to review bug 392846 has a review (bug 95829) as old as 2007-06-16 which has not received a response and his last comment in any bug is 2007-07-26

I am eager to help.
I am changing the summary of this bug to narrow the scope to just include 
Filippo's specific server bug.  I don't want this bug to become the ultimate 
catch-all bug for all complaints about TLS in TBird.  

Wayne,  Are you also running a server with sslio/MatrixSSL ?  If not,
your users are having a problem that may or may not have anything to do 
with the specific problem found in this bug.  IMO, the way to diagnose it 
is not to try various patches for other problems, but rather is to capture 
connection attempts that fail using ssltap, and analyze them.  Please do 
that in another bug.  thanks.
Summary: Problems with SMTP over TLS in Thunderbird 2 → SMTP-TLS problems with sslio/MatrixSSL server
This problem (SMTP over SSL) is solved with the patch Charlie Brady added to MatrixSSL.

(In reply to comment #61)
> In particular I should like to test a build of what you've
> got from bug 392846 to seen if solves some of our existing problems.

Wayne, if you want to test whether the problems in your environment might be related to bug 392846, you can do so using configuration.

Use the config editor in Thunderbird preferences and search for "security.enable_tls". Switch it to "false" (a double click should be sufficient). Then restart Thunderbird.

If you're still having your problems after this change, then you are experiencing a different problem.

If this change fixes your problems, then you can use it as a workaround until the fix gets officially released.


Wayne, please also answer Nelson's question from comment 62.

Thanks.

(In reply to comment #62)
> Wayne,  Are you also running a server with sslio/MatrixSSL ?  If not,
> your users are having a problem that may or may not have anything to do 
> with the specific problem found in this bug.  IMO, the way to diagnose it 
> is not to try various patches for other problems, but rather is to capture 
> connection attempts that fail using ssltap, and analyze them.  

in answer to your question - no.  agree with your comments - I was a hasty in hitching my wagon to this bug.

filed bug 394152. I forgot to mention in that bug, sending works fine using thunderbird v1.5
Marking as a duplicate of bug 392846 since it sounds like the actual problem was a bug in MatrixSSL, but bug 392846 implements a work-around for the MatrixSSL bug in NSS.  If I somehow misunderstood the situation, please re-open.
Status: NEW → RESOLVED
Closed: 16 years ago
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: