Open Bug 1280076 Opened 4 years ago Updated 4 years ago
mozilla::pkix supports certificate trust path lengths of 8 or less
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.84 Safari/537.36 Steps to reproduce: I was experimenting with replacing the existing public-key algorithms with quantum-resistant equivalents (authentication and key establishment). In some cases, public key, signature and ciphertext size cause the handshake messages to grow beyond the 16K message fragment limit. What I noticed is that handshake messages that span over two 16K message fragments are truncated. To test the effects of the increased handshake message size without using any quantum-resistant algorithms, I created a RSA 4096 certificate chain comprising 26 certificates: an end entity certificate, followed by 24 intermediate certificates and a root certificate at the end. I also populated the SubjectAltNames fields with multiple very long email addresses, for example, having 100 characters. On the back end, I configured an Apache server using OpenSSL for TLS. On the front end, I used Firefox using NSS for TLS. Actual results: 1. With Firefox 47, I get the SEC_ERROR_UNKNOWN_ISSUER. When looking at the certificate chain I see that it is missing top 6 certificates (5 intermediates and root). 2. With Firefox 42, I got SSL_ERROR_RX_RECORD_TOO_LONG. After rebuilding with SSL debug logging on, I noticed this error was set in nss/lib/ssl/ssl3gthr.c : ssl3_GatherData() Expected results: RFC 5246 doesn’t put a limit on the maximum number of fragments the handshake messages can be broken into. The quantum-resistant algorithms I was using required at most 4 – 5 fragments in the most extreme case. Connecting with Chrome using BoringSSL for TLS worked fine, with all 26 certificates showing correctly.
Are you sure that this is a defect in NSS and not a limitation of the PKI validation library? If the messages were indeed truncated I would expect some sort of TLS error at the handshake stage (more like you see with Firefox 42).
I did all of my original experiments with Firefox 42 and found the function in the ssl code that was setting the SSL_ERROR_RX_RECORD_TOO_LONG error. Reducing the total size of ServerHello|ServerCertificate|ServerKeyExchange|ServerHelloDone to just under 32Kbytes worked fine. With Firefox 47, I wanted to ensure the issue remained before logging a bug, but didn’t investigate further. I just assumed the Firefox mapped the ssl error to a more generic sec error.
I suspect that that's not true. Do you have a test server that we can look at?
I am having trouble reproducing the SSL_ERROR_RX_RECORD_TOO_LONG, even with Firefox 42 now. Perhaps this bug should be closed until I can reproduce it again. I investigated the SEC_ERROR_UNKNOWN_ISSUER further and found, as you suspected, a limitation in the PKI library. It only allows 6 intermediate certificates at the most (max 8 for the whole chain, including the root and the end entity).
Alexander: sounds like this shouldn't block you, then Forwarding to Keeler for processing as part of moz::pkix
The path length limit is by design to avoid stack exhaustion. We could increase the limit, but I don't think there's a practical limit we could set where someone couldn't come along and say, "hey, it doesn't work for my setup of n certificates". Perhaps the best thing would be to return a more descriptive error code/message.
Assignee: nobody → nobody
Status: UNCONFIRMED → NEW
Component: Libraries → Security: PSM
Ever confirmed: true
Product: NSS → Core
Summary: TLS handshake message spanning more than two 16K message fragments is not supported → mozilla::pkix supports certificate trust path lengths of 8 or less
Version: 3.19.3 → Trunk
Yeah, the current limit seems reasonable. The reason I reached it, is because I was trying to artificially increase the length of the Certificate message to see if the handshake message fragmentation worked properly.
Priority: -- → P5
You need to log in before you can comment on or make changes to this bug.