311939 - starttls mishandled

Status: RESOLVED DUPLICATE of bug 325379

(MailNews Core :: Networking: IMAP, defect)

Description

[Per Steinar Iversen]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b5) Gecko/20051006 Firefox/1.4.1
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b5) Gecko/20051006 Firefox/1.4.1

If security settings for an IMAP server is set to "TLS, if available" then this
does not work as expected, thunderbird is unable to make the transition to TLS.
A trace done with ethereal shows this:

* OK imap.hio.no Cyrus IMAP4 v2.2.12-Invoca-RPM-2.2.12-1 server ready
1 capability
* CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ MAILBOX-REFERRALS NAMESPACE
UIDPLUS ID NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY SORT
THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE IDLE STARTTLS LOGINDISABLED
AUTH=DIGEST-MD5 AUTH=CRAM-MD5 SASL-IR LISTEXT LIST-SUBSCRIBED X-NETSCAPE
1 OK Completed
2 STARTTLS
2 OK Begin TLS negotiation now
3 capability
2 NO Starttls negotiation failed
* BAD Invalid tag
4 authenticate CRAM-MD5
...the rest deleted


Reproducible: Always

Steps to Reproduce:
1. Install thunderbird 1.5beta2
2. Set thunderbird to use TLS, if available
3. The connection fail to upgrade to TLS.

Actual Results:  
Connection fails to use TLS, but works ok.

Expected Results:  
Switches to TLS.

Comment 1

[David :Bienvenu]

StartTLS seems to be failing in NSS, at least in Thunderbird 1.5 beta2. Any idea
when this started failing, Per? I'm pretty sure it worked at one point.

Comment 2

[David :Bienvenu]

I get error -12195 back from NSS, and an alert popped up to that effect.

Comment 3

[Nelson Bolyard (seldom reads bugmail)]

According to
http://www.mozilla.org/projects/security/pki/nss/ref/ssl/sslerr.html#1042994
error -12195 means:
"Peer does not recognize and trust the CA that issued your certificate."

That suggests to me that the server requested SSL client authentication
from the (email?) client during the SSL/TLS handshake, and the client 
did attempt to authenticate using a certificate (and private key) that
it had.  But the server did not recognize and trust the issuer of that
SSL client certificate, and so terminated the handshake after sending 
to the client an error alert record which means "I don't like your 
cert's issuer."

It is also possible that the server requested SSL client authentication,
but the client had no cert, and so responded saying "I have no cert",
and the server then terminated the handshake after sending the client 
an error alert record which means "I don't like your cert's issuer."
In that case, it may be that the server is sending the wrong error alert 
code for that situation.  This error code is specifically about an 
untrusted issuer, and IMO should not be used in the case where the 
client sent no client authentication certificate to the server.  

Perhaps this is yet another place where mozilla's error dialog needs to 
be smarter about telling the user what went wrong, e.g. something like:
"The server rejected your certificate because it does not recognize and 
trust the issuer of your certificate." 

In any case, given the error code, I think it unlikely that any bug in
NSS is indicated, and more likely that the client simply attempted to 
authenticate with a certificate that the server was unwilling to accept.
If that is indeed the case, then no mozilla bug is confirmed here.

Comment 4

[Per Steinar Iversen]

(In reply to comment #3)
> According to
> http://www.mozilla.org/projects/security/pki/nss/ref/ssl/sslerr.html#1042994
> error -12195 means:
> "Peer does not recognize and trust the CA that issued your certificate."
> 
> That suggests to me that the server requested SSL client authentication
> from the (email?) client during the SSL/TLS handshake, and the client 
> did attempt to authenticate using a certificate (and private key) that
> it had.  But the server did not recognize and trust the issuer of that
> SSL client certificate, and so terminated the handshake after sending 
> to the client an error alert record which means "I don't like your 
> cert's issuer."
> 
> It is also possible that the server requested SSL client authentication,
> but the client had no cert, and so responded saying "I have no cert",
> and the server then terminated the handshake after sending the client 
> an error alert record which means "I don't like your cert's issuer."
> In that case, it may be that the server is sending the wrong error alert 
> code for that situation.  This error code is specifically about an 
> untrusted issuer, and IMO should not be used in the case where the 
> client sent no client authentication certificate to the server.  
> 
> Perhaps this is yet another place where mozilla's error dialog needs to 
> be smarter about telling the user what went wrong, e.g. something like:
> "The server rejected your certificate because it does not recognize and 
> trust the issuer of your certificate." 
> 
> In any case, given the error code, I think it unlikely that any bug in
> NSS is indicated, and more likely that the client simply attempted to 
> authenticate with a certificate that the server was unwilling to accept.
> If that is indeed the case, then no mozilla bug is confirmed here.

In my case Thunderbird was not set up to use any kind of client certificate. The
ethereal output show no trace of any such activity, it would have been very
different if that was the case.

Comment 5

[Nelson Bolyard (seldom reads bugmail)]

I suggest you use the program SSLTAP to monitor the connections
between client and server.  It will tell you conclusively whether
any client authentication was requested, or not, and what response
was given to such a request.  That output would be a good attachment
to this bug.  For more info, see 
http://www.mozilla.org/projects/security/pki/nss/tools/ssltap.html

The error number -12195 has only one cause in NSS, and that is the 
receipt of an alert record from the peer (the server in this case)
that specifically means "I don't trust your cert's issuer".  
Perhaps you and David Bienvenu are experiencing different errors.
Perhaps you are actually experiencing an error other than -12195.

Comment 6

[Per Steinar Iversen]

Attachment: Output from ssltap

Comment 7

[Per Steinar Iversen]

(In reply to comment #5)

ssltap gives the same kind of output as ethereal, see new attachment.

Comment 8

[David :Bienvenu]

I'm pretty sure we're seeing different problems. My cert is from thawte; perhaps
the CMU server doesn't trust thawte, or doesn't really support TLS. But at least
for me, it's trying. It sounds like for the reporter, it's not trying at all,
which makes me suspect something's failing early on the client side in a attempt
to start negotiations. Does SSL work for you, if that server supports SSL?

Comment 9

[Per Steinar Iversen]

(In reply to comment #8)
> I'm pretty sure we're seeing different problems. My cert is from thawte; perhaps
> the CMU server doesn't trust thawte, or doesn't really support TLS. But at least
> for me, it's trying. It sounds like for the reporter, it's not trying at all,
> which makes me suspect something's failing early on the client side in a attempt
> to start negotiations. Does SSL work for you, if that server supports SSL?

I know our server handles STARTSSL ok, it works fine with pine for example.

Comment 10

[David :Bienvenu]

I didn't mean to imply you were having a server problem - it seems more like a
client configuration issue, since, if your logs are right, we're never even
starting the negotiation.

Comment 11

[Per Steinar Iversen]

(In reply to comment #10)
> I didn't mean to imply you were having a server problem - it seems more like a
> client configuration issue, since, if your logs are right, we're never even
> starting the negotiation.

Some more experiments with the IMAP server security settings show that "TLS, if
available" always fails. "TLS" performs STARTTLS correctly! "SSL" works but over
port 993. The problem must be with the conditional then?

Comment 12

[David :Bienvenu]

I think the very first time you connect using try tls, it wouldn't work, because
we wouldn't know the server supported STARTTLS - we have to set the connection
type before we connect to the server. But once you've connected and we've
determined that the server does support STARTTLS, from then on, we should always
try it, because we remember the server capability persistently. And since we're
sending STARTTLS, we know the server supports it. That very first connection
should be the only difference. I wonder if we're failing trying to create the
transport - but that wouldn't explain why the "always use TLS" setting works. In
short, I have no idea why always use TLS would work and try tls wouldn't. The
only difference is that we check the capability flag...

Comment 13

[Nelson Bolyard (seldom reads bugmail)]

The attachment contains this:
--> [ 1 capability^M ]
<-- [ * CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ MAILBOX-REFERRALS 
      NAMESPACE UIDPLUS ID NO_ATOMIC_RENAME UNSELECT CHIL DREN MULTIAPPEND 
      BINARY SORT THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE IDLE 
      STARTTLS LOGINDISABLED AUTH=DI GEST-MD5 AUTH=CRAM-MD5 SASL-IR LISTEXT 
      LIST-SUBSCRIBED X-NETSCAPE^M
      1 OK Completed^M ]
--> [ 2 STARTTLS^M ]
<-- [ 2 OK Begin TLS negotiation now^M ]
--> [ 3 capability^M ]
<-- [ 2 NO Starttls negotiation failed^M 
      * BAD Invalid tag^M]
--> [ 4 authenticate CRAM-MD5^M ]

David, it appears to me that after the server responded positively to the 
client's STARTTLS command, the client resent the capability command, rather
than starting the TLS handshake by sending a TLS client_hello message.  
Then the server complained that the client failed to start TLS.  

So my question is: 
why would the client resend the capability command instead of starting TLS?  

Comment 14

[Nelson Bolyard (seldom reads bugmail)]

David, here's an additional question (to the one I asked in comment 13).

After the client succesfully starts TLS, does the client then attempt
to resend the capability command as the first IMAP command over the 
encrypted channel?  

In other words, is it possible that the command that was supposed to have
been sent encrypted was sent unencrypted?  If not, then what (if anything)
can we deduce from the presence of that capability command at the point
where the start of the SSL/TLS handshake was expected?

Comment 15

[David :Bienvenu]

We do send the capability command after we think the channel has been encrypted.
So we can deduce that we think we've tried to encrypt the channel. But looking
at the code, we issue the capability no matter what the return codes are from
the various methods called to encrypt the channel. I assume one of those calls
is failing, but I don't know why always use TLS would actually work.

Comment 16

[AJ]

Attachment: TLS alternate port error

Comment 17

[AJ]

Hi Guys,

I have been testing TLS with our server with Tunderbird.  I have set the option to if TLS available and to TLS.  With TLS available it does not negotiate a TLS connection even though TLS is enabled an working just fine with clients like Outlook, Outlook Express and Eudora.  If I enable it to connect with the option TLS I am able to send a message and see that it did use TLS.

I then tested with the alternate port number for TLS 993 and I was not able to connect to the server and I get this error - see attachment.

As we are the development team for our mail server we might be able to give you more data if you need it to be able to resolve this issue.

-AJ

Comment 18

[Nelson Bolyard (seldom reads bugmail)]

(In reply to comment #15)
> We do send the capability command after we think the channel has been
> encrypted.
> So we can deduce that we think we've tried to encrypt the channel. But looking
> at the code, we issue the capability no matter what the return codes are from
> the various methods called to encrypt the channel. I assume one of those calls
> is failing, but I don't know why always use TLS would actually work.

David,
If the client continues to behave as if the channel was encrypted, when it 
isn't, isn't that a P1 security bug?  
What if the authentication to the server succeeded?  
Would the user have ANY way (short of running a sniffer) of knowing that the
traffic was actually unencrypted?  Surely the password would be potentially
compromised at that point (unless it was using CRAM-MD5).

Comment 19

[David :Bienvenu]

Nelson, if you tell us we have to use TLS, we'll use it. The problem seems to arise if you tell us to use it if we can, and for some reason, for this user, we fail to use it, and fall back...your only guarantee of getting TLS is to tell us to always use it.

However, I have no idea why it's not working for this user...and I'm not sure what to try. My guess is that this line is failing:

http://lxr.mozilla.org/seamonkey/source/mailnews/imap/src/nsImapProtocol.cpp#787

But I don't know why it would...I can try to add logging to see if that indeed is happening.

Comment 20

[David :Bienvenu]

Attachment: add logging, and handle errors from ssl layer

this adds some logging, and aborts starttls connections if they fail - I'd like to try this to diagnose what's going on, and then go from there. I'd like to put up an appropriate ssl error, but I don't know why the ssl layer's not doing that already.

Comment 21

[David :Bienvenu]

I've checked in the last patch - if you want to try a trunk build from tomorrow or later and let me know what shows up in our imap protocol log in this situation, that would be very helpful.

http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#imap

Comment 22

[Daniel Veditz [:dveditz]]

Is this something we need to fix on the 1.8 and 1.8.0 branches? Fully fix on the trunk?

Comment 23

[David :Bienvenu]

Personally, it seems reasonable to expect that changing connect options would not affect connections that have already been opened, so I don't think we need a fix for this on the older branches.

Comment 24

[David :Bienvenu]

sorry, that last comment was meant for the bug about closing cached imap connections when the user changes their new connection prefs. I'm going to mark this as a dup of bug 325379 - I did end up landing the extra error checking in this patch since it made the patch in bug 325379 easier to apply.

*** This bug has been marked as a duplicate of 325379 ***

Comment 25

[David :Bienvenu]

clearing blocking nominations