116168 - TLS server name indication extension support in NSS

Status: RESOLVED FIXED

(NSS :: Libraries, enhancement, P2)

Description

[Julien Pierre]

This is a tracking defect for new DNSName support in NSS. DNSName has not been
finalized yet as a standard but should be within the next few months. It would
be a good goal to support it in NSS 4.0, as it will allow a new level of virtual
server hosting especially for HTTP, resolving the need for a new TCP endpoint
for an SSL virtual server.

The idea behind DNSName is to allow an extra string to be sent by the client in
the Client Helo SSL/TLS message. The server is then free to interpret it.
This bug is to vouch for ways to programmatically do the following in an NSS
application :
a) for a client, to be able to set the string that will be sent. This may
implemented as the same as is done in the seturl option already
b) for a server, to retrieve the string halfway through the handshake. The
server needs to be able to do this right after accept of the NSPR TCP socket,
before it has selected its certificate, cipher, or even the version of SSL (2, 3
or TLS) that it wants to use . Nelson suggested that this should be done through
a callback function.

Comment 1

[Nelson Bolyard (seldom reads bugmail)]

This feature necessitates a substantial change in the SSL/TLS API for 
servers, and a very minor change for clients.  For clients there is 
simply one more value, the requested server name, to be set on the 
socket through a new optional function call prior to initiating the 
handshake.  

But for servers, the change is larger.  Today, a server application
preconfigures the server socket with one certificate for each of the 
key exchange and/or authentication key types that it wishes to support,
e.g. a cert with an RSA key and another with a DSA key, and the SSL
code chooses from among those according to the key type needed to do 
the handshake.  In the future, the server may have many certs of the
same key type (e.g. certs with RSA keys), each having a different 
host name.  The choice of which cert to use must be based on both key
type and host name required.  

From SSL's perspective, the challenge is simply to choose the right cert, 
but the server application (e.g. https) may have other things that have 
to be associate with it, such as the choice of a document root and security 
policies associated with that virtual server.  

There are at least two ways we can attempt to add this functionality to 
the SSL API, including:

1. The server registers multiple cert/key pairs with the SSL socket prior
to the handshake, and SSL chooses the right pair from among them using
the (optional) DNS name and the key type.  THe server learns which cert
was chosen by calling SSL_LocalCertificate (which is now in NSS 3.4)
after the handshake completes.  

2. The server registers a callback function, rather than any cert/key pairs 
with SSL prior to the handshake.  When the SSL code receives the client
hello, it calls the callback asking for the cert(s) and matching key(s) to 
use.  This could be done a couple of ways:

2a. When the application gets called back with the server's name, it then
registers all the certs for that name (one for each supported kea key type) 
with the socket, using SSL_ConfigSecureServer, much as it does now.  Then
SSL completes the cipher suite selection and cert selection much as it does
now.  This amouns to simply delaying the SSL_ConfigSecureServer call to a
later point in the socket's lifetime.

2b. Alternatively, when the application gets called back, it actually does
the TLS cipher suite negotiation, choosing the cipher suite and key type,
and returns just one cert/key pair, the right pair for the negotiated 
cipher suite.  This scheme gives the server maximum flexibility for 
choosing the certificate, but burdens it with a significant part of the 
TLS protocol processing.

Comments are invited.

Comment 2

[Julien Pierre]

Nelson,

Option 1 limits the flexibility of server applications to do what they want
based on the DNSName field value. For example some servers may want to support
dynamic DNSNames as opposed to static. A preconfigured list of DNSNames on the
SSL socket would not allow complex things like regular expression matching,
unless NSS explicitly supported the exact algorithm that the server wants to use
to choose the security policy. I feel that it's up to the server application to
make the security policy choice and that it needs to be esented with the name
sent by the client as it happens.

Not all NSS SSL servers preconfigure the server socket with a single cert.
The requirement exists to do hardware virtual servers today, based on IP
endpoint. For example, a server with multiple IP addresses can listen on address
0.0.0.0 on port 443 . Today iWS/NES 6 will create an NSPR socket for this,
without importing it to SSL, and accept the connection on that socket. They will
then look at the local IP address of the connection, in order to determine which
security policy/indentity to use. Then and only then will they import the
*accepted* connection socket to SSL, using a model socket. This is how web
server 6 works today. It maintains a pool of model SSL sockets that are
preconfigured with certs, ciphers, and selected security protocol. Then the
accepted connection socket gets imported with that model SSL socket, and a
handshake is forced.

I think the application - in this case the web server - should control the list
of identities and how they will be selected. I don't think it can be handled by
NSS. Say for example you have a 0.0.0.0:443 listen socket. You want to support
two DNSName identities for IP address 1.1.1.1 on port 443, and two other DNSName
identities for IP address 1.1.1.2 . If you require a listen socket to be created
through NSS with SSL, then NSS has to support not only the selection of
identities based on DNSName but also based on IP address - which is really of
the application's responsibility. This is an example of why I don't like your
option 1.

You might object to that that you can have a sort of intermediate approach,
where for the hardware virtual server case with multiple IPs, the socket would
still be accepted as NSPR socket like today, then the import to SSL would
happen, with a model socket containing multiple DNSName identities, and further
down after the handshake the selected identity would be retrieved by the
application. I don't like that idea either however as I think it's unnecessary
complication to do this within NSS.

Ideally, I believe the server should maintain a model SSL socket for each of his
security policies, whether they are to be chosen based on an IP:port TCP
endpoint (hardware virtual server case) or on a different DNSName identity. This
will greatly simplify things for the application. The question remains about how
the server retrieves the DNSName identity sent by the client in order to select
its security policy.

This means I like option 2 with the callback approach.

I don't think option 2b is desirable because it requires too much knowledge of
SSL inside details by the application and I don't believe that it would add much
value to it. Basically you would be able to select a different identity for each
cipher suite, but I'm not sure that much granularity is necessary. I think one
identity per key type as is the case today is sufficient.

So I like option 2a.
I think the prototype for the callback should look like this :

PRFileDesc* GetIdentity(char* DNSName, void* appdata);

Basically, the callback would provide the server application with DNSName, as
well as some application specific data (the web server would store the IP:port
the connection was accepted on for example, to know where to branch in its
security policy tree) - and then the server would select a policy and return a
model SSL socket back to the SSL layer. The SSL layer would use the model socket
to select the certificate, private key, and cipher suites for the connection
socket, basically doing the equivalent of an SSL_ImportFD(modelsocket,
connectionsocket). This would allow the SSL handshake to proceed (serverhelo, etc).

My arguments for the above prototype is that we don't want servers to call
things like SSL_ConfigSecureServer and other SSL socket configuration calls for
each connection at runtime when choosing the identity. My tests showed in the
past that making those calls for every connection was extremely expensive,
slowing down the server by more than 50%. This was determined to be due to some
key operations. This is why the model socket approach was chosen to implement
hardware virtual servers in the current web server. I think this is equally
suited for the DNSName case.

The other change that will be required is that the handshake should be able to
begin without the SSL socket being fully configured - ie, no cert, key, or
cipher suite has been selected yet.

So the web server would do something like the following :
fd = PR_Accept( nsprlistensocket ); // same as today

PR_GetSockname(fd, &localip); // same as today, get local connection IP to do
hardware virtual servers

fd = SSL_ImportFD(NULL, fd); // make the socket SSL, but without an identity yet

SSL_OptionSet(fd, SSL_ENABLE_DNSNAME, PR_TRUE); // tells SSL layer it's OK to do
handshake with clienthelo even though no identity has been selected yet

SSL_SetDNSNameCallback(fd, identityfunction, localip); // required if
SSL_ENABLE_DNSNAME is set. So we may actually want to combine the last two lines
of code into one single API

SSL_ForceHandshake(fd); // same as today

PR_Recv(fd, ... ); // begin HTTP processing, which will cause the handshake to occur

The callback function would be something like this :

PRFileDesc* chooseidentity(char* DNSName, localip)
{
   // choose an identity based on local IP and DNSName
   // I'm not going to detail here how web server actually does it
   // because it's rather complex
   return selectedidentity;
}

Comment 3

[Julien Pierre]

I forgot to add in the proposal that the SSL layer could pass a NULL DNSName to
the callback if the client doesn't send DNSName (as is the case for all current
clients). In that case the server must choose a default SSL policy based on the
TCP endpoint as it already does today - except currently it is done by an
"connectionsocket = SSL_ImportFD(modelsocket, connectionsocket)" right after the
PR_GetSockName .

Comment 4

[Julien Pierre]

I also want to add a clarification on the "identity" model socket that the
callback is required to return. This model socket would have all the needed
certs configured for different keytypes that the server wants to support for the
given DNSName identity (or default identity if DNSName is null).

Comment 5

[Wan-Teh Chang]

Changed the QA contact to Bishakha.

Comment 6

[Julien Pierre]

FYI, the extension has been renamed to "Server name indicatoin". It is defined
in section 3.1 of draft-ietf-tls-extensions-05 , which is available at
http://www.ietf.org/internet-drafts/draft-ietf-tls-extensions-05.txt .

Comment 7

[Julien Pierre]

Regarding Perry Loirier's the patch at
https://bugzilla.mozilla.org/attachment.cgi?id=182158

I
You are unconditionally adding the SNI extension to the v3 CLIENT HELO .
We may yet discover through testing that some servers will choke on this
extension. We need to have a bit to turn this on or off, even for v3. This would
be set through SSL_OptionSet . The option name could be SSL_CLIENTHELO_SNI .
I think re-using the URL from SSL_SetURL to determine the value of the SNI
within the CLIENT HELO is fine .

II
You only implemented the client side support - you added the SNI to the CLIENT
HELO . NSS also supports server-side. For server applications, NSS needs an
interface to parse the SNI out of an incoming client hello. We should not
implement the client side of this extension in NSS without also implementing the
server side.

For the server side, we need a new callback . There has been no decision yet on
what that callback interface should be like . See comment #1 and comment #2 for
the start of the discussion .

III

Since there are several new extensions being added to standard in the CLIENT
HELO, the server might choose its identity based on the value of any CLIENT HELO
extensions, not just the SNI. There are a few ways to solve that :

a) have the callback receive all the extensions in an array, and allow the
server to return its new identity altogether in one callback :

SECStatus ExtensionsCallback(SSLExtensions** extensions, CERTCertificate**
newServerIdentity, void* arg);

libssl would provide some helper functions to let apps decode the extensions
within the callback .

b) the server could register a callback for each specific extension :
SECStatus SSL_RegisterExtensionCallback(int extensionID, ExtensionCallback*
function);

SECStatus ExtensionCallback(SSLExtensions* extension,void* arg);

But then a separate callback is needed for the server's identity decision, which
would be called after all the extensions have been parsed :

SECStatus SSL_NewServerIdentityCallback(CERTCertificate** newServerIdentity,
void* arg);

I don't like this, because the multiple callbacks mean the application is
required to use the "void* arg" for state.

I don't know enough about the format of SSL extensions to know how practical it
is for libssl to register callbacks for specific ones . Is there a "type" field
somewhere or is the format completely extension specific ?

Also, I don't know if the same extension is allowed to be passed more than once.
The second interface wouldn't work as well as the first one if this happens.

Note that this callback model may be needed on the client side as well to allow
the client to receive SERVER HELO extensions .

Comment 8

[Perry Lorier]

Sorry for not replying sooner to your comments, I've been busy with one thing or
another, but I thought I should quickly respond to these to give people an idea
of whats going on :)

I) Using SNI By default
Ok, I've been trying to do this so far without any "user visible" changes to the
API, but I can easily add something to disable this if it's unwanted in some
situations.  Although I don't understand the NSS API that well, I assumed if you
didn't want SNI, you wouldn't set the URL but a seperate option makes that much
nicer.

II: Client support only
Yes, the current patch only provides support for clients, and only if the client
isn't using v2CompatibleHello's.  The idea was to have small easy to understand
and verify patches.  I'm still working on a patch that can use client side SNI
even when v2CompatibleHello's are in use.  Also, as this is probably my first
patch to get committed, early submission of a preliminary patch means I get
useful feedback like this before I spend too much time building a giant castle
on the wrong hill.

III: SNI callbacks for server side

As for how SNI would work server side, I'd suggest asking Paul Querna who has
been working on this for apache using gnutls, his experience is likely to be
useful here.  Paul, what are your thoughts on a nice server side API for SNI?

RFC 3546, Section 2,3 says "There MUST NOT be more than one extension of the
same type."  Each type is identified by an unsigned 16 bit value.

As the extensions either change the certificate used or modify the security of
the connection (eg truncated HMACs), I'd suggest just having a second version of
the SSL_AuthCertificate callback that also gets passed what extensions are in
use.  IMHO.

Comment 9

[Nelson Bolyard (seldom reads bugmail)]

In reply to comment 8,
> I assumed if you didn't want SNI, you wouldn't set the URL 

SSL Clients must specify the DNSname in the misnamed "URL" because it is 
necessary for validating the DNSname in the server's cert to avoid MITM attacks.

Let me restate here my remark in bug 116169 comment 19 :

I oppose checking in code that needs to be replaced shortly therafter.  
It appears to me that the proposed patch to implement this single extension 
would need to be entirely replaced as soon as a second extension is added.  
I'd rather see an extensible multi-extension design from the beginning. 
I don't mean to discourage you, Perry, but I don't think we can take the 
patch you attached to bug 116169, for that reason.  It's not a problem,
BTW, to add new NSS API functions as needed.  But once added, they must
be supported effectively forever therafter, so they need to be well thought
out before being introduced.  

Comment 10

[Nicolás Lichtmaier]

This will be included in the upcoming IE 7: 
http://blogs.msdn.com/ie/archive/2005/10/22/483795.aspx

Comment 11

[Julien Pierre]

The target milestone for this bug was originally set to NSS 4.0 back in 2001. There is no longer a plan for such a release. Given the growing importance of the TLS SNI extension, I'm resetting the target tentatively to 3.12 .

Comment 12

[Jesse Ruderman]

Nominating to block Firefox 2.  It would be lame if Firefox was the only major browser not supporting TLS SNI and prevented sites that share IPs from using https. (On DreamHost at least, I think the cost of a non-shared IP addresses is the largest part of the cost of using https.)

Comment 13

[Nelson Bolyard (seldom reads bugmail)]

Attachment: capture exploratory work in progress

No reviews requested at this time.  Just capturing some hacks.

Comment 14

[Nelson Bolyard (seldom reads bugmail)]

Attachment: More exploratory work

Comment 15

[Nelson Bolyard (seldom reads bugmail)]

Comment on attachment 216703 [details] [diff] [review]
More exploratory work

superseded by patch attached to bug 226271

Comment 16

[Mike Shaver (:shaver emeritus)]

Blocker for 1.8.1.

Comment 17

[Nelson Bolyard (seldom reads bugmail)]

This work is now done, and checked in on the trunk and the NSS_3_11_BRANCH.
The work was mostly done as part of bug 226271.  
A new CVS tag needs to be made so that mozilla clients can pick it up. 
I expec that will happen this week.

Comment 18

[Mike Beltzner [:beltzner, not reading bugmail]]

Nelson, is this fixed on the 1.8.1 branch as well? If so, can you please add fixed1.8.1 to the keyword list?

Comment 19

[Nelson Bolyard (seldom reads bugmail)]

The necessary NSS code is also on the MOZILLA_1_8_BRANCH. 
To use it, the browser must disable SSL2-format client hellos.
Bug 116169 tracks that work.
I think it is done, but bug 116169 does not clearly so indicate.

Comment 20

[u238590]

Wondering if comment #3 is applicable for me? I do not want to call SSL_ConfigureServer for performance reasons as Julien stated.

Comment 21

[Julien Pierre]

Meena,

Only the client side of server name indication was added to NSS. At least that's all that was discussed in this bug. I don't believe the server side work was ever completed.

Comment 22

[u238590]

Thanx. I suspected that code for

"SSL_OptionSet(fd, SSL_ENABLE_DNSNAME, PR_TRUE); // tells SSL layer it's OK to do handshake with clienthello even though no identity has been selected yet"

is not written. Just confirmed. 

I am now planning to call SSL_ReconfigFD as shown in
https://bugzilla.mozilla.org/show_bug.cgi?id=631986 

Hope that SSL_ReconfigFD is not so bad for performance. Especially CERT_DestroyCertificate, CERT_DupCertificate, CERT_DestroyCertificateList,  CERT_DupCertList, etc

Comment 23

[Wan-Teh Chang]

Meena: the server side of SNI was implemented in bug 360421.  This
bug is about the client side.

It is best to ask questions about SSL_ReconfigFD in the Mozilla
crypto newsgroup, and cc Alexei Volkov (who wrote the server
side SNI code).

Comment 24

[u238590]

Yes. Alexei only told me yesterday to use SSL_ReconfigFD.