1280076 - mozilla::pkix supports certificate trust path lengths of 8 or less

Status: NEW

(NSS :: Libraries, enhancement, P5)

Description

[Alexander Truskovsky]

Attachment: screenshots

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.84 Safari/537.36

Steps to reproduce:

I was experimenting with replacing the existing public-key algorithms with quantum-resistant equivalents (authentication and key establishment).  In some cases, public key, signature and ciphertext size cause the handshake messages to grow beyond the 16K message fragment limit.  What I noticed is that handshake messages that span over two 16K message fragments are truncated.

To test the effects of the increased handshake message size without using any quantum-resistant algorithms, I created a RSA 4096 certificate chain comprising 26 certificates: an end entity certificate, followed by 24 intermediate certificates and a root certificate at the end.  I also populated the SubjectAltNames fields with multiple very long email addresses, for example, having 100 characters.  On the back end, I configured an Apache server using OpenSSL for TLS.  On the front end, I used Firefox using NSS for TLS.


Actual results:

1. With Firefox 47, I get the SEC_ERROR_UNKNOWN_ISSUER.  When looking at the certificate chain I see that it is missing top 6 certificates (5 intermediates and root).

2. With Firefox 42, I got SSL_ERROR_RX_RECORD_TOO_LONG.  After rebuilding with SSL debug logging on, I noticed this error was set in nss/lib/ssl/ssl3gthr.c : ssl3_GatherData()


Expected results:

RFC 5246 doesn’t put a limit on the maximum number of fragments the handshake messages can be broken into.  The quantum-resistant algorithms I was using required at most 4 – 5 fragments in the most extreme case.

Connecting with Chrome using BoringSSL for TLS worked fine, with all 26 certificates showing correctly.

Comment 1

[Eric Rescorla (:ekr)]

Are you sure that this is a defect in NSS and not a limitation of the PKI validation library?

If the messages were indeed truncated I would expect some sort of TLS error at the handshake stage (more like you see with Firefox 42).

Comment 2

[Alexander Truskovsky]

I did all of my original experiments with Firefox 42 and found the function in the ssl code that was setting the SSL_ERROR_RX_RECORD_TOO_LONG error.  Reducing the total size of ServerHello|ServerCertificate|ServerKeyExchange|ServerHelloDone to just under 32Kbytes worked fine.

With Firefox 47, I wanted to ensure the issue remained before logging a bug, but didn’t investigate further.  I just assumed the Firefox mapped the ssl error to a more generic sec error.

Comment 3

[Eric Rescorla (:ekr)]

I suspect that that's not true. Do you have a test server that we can look at?

Comment 4

[Alexander Truskovsky]

I am having trouble reproducing the SSL_ERROR_RX_RECORD_TOO_LONG, even with Firefox 42 now.  Perhaps this bug should be closed until I can reproduce it again.

I investigated the SEC_ERROR_UNKNOWN_ISSUER further and found, as you suspected, a limitation in the PKI library.  It only allows 6 intermediate certificates at the most (max 8 for the whole chain, including the root and the end entity).

Comment 5

[Eric Rescorla (:ekr)]

Alexander: sounds like this shouldn't block you, then

Forwarding to Keeler for processing as part of moz::pkix

Comment 6

[Alexander Truskovsky]

Thanks

Comment 7

[Dana Keeler (she/her) (use needinfo) [:keeler] (on leave)]

The path length limit is by design to avoid stack exhaustion. We could increase the limit, but I don't think there's a practical limit we could set where someone couldn't come along and say, "hey, it doesn't work for my setup of n certificates". Perhaps the best thing would be to return a more descriptive error code/message.

Comment 8

[Alexander Truskovsky]

Yeah, the current limit seems reasonable.  The reason I reached it, is because I was trying to artificially increase the length of the Certificate message to see if the handshake message fragmentation worked properly.