Open Bug 1280076 Opened 4 years ago Updated 4 years ago

mozilla::pkix supports certificate trust path lengths of 8 or less


(Core :: Security: PSM, defect, P5)





(Reporter: alexander.truskovsky, Unassigned)


(Whiteboard: [psm-backlog])


(1 file)

Attached file screenshots
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.84 Safari/537.36

Steps to reproduce:

I was experimenting with replacing the existing public-key algorithms with quantum-resistant equivalents (authentication and key establishment).  In some cases, public key, signature and ciphertext size cause the handshake messages to grow beyond the 16K message fragment limit.  What I noticed is that handshake messages that span over two 16K message fragments are truncated.

To test the effects of the increased handshake message size without using any quantum-resistant algorithms, I created a RSA 4096 certificate chain comprising 26 certificates: an end entity certificate, followed by 24 intermediate certificates and a root certificate at the end.  I also populated the SubjectAltNames fields with multiple very long email addresses, for example, having 100 characters.  On the back end, I configured an Apache server using OpenSSL for TLS.  On the front end, I used Firefox using NSS for TLS.

Actual results:

1. With Firefox 47, I get the SEC_ERROR_UNKNOWN_ISSUER.  When looking at the certificate chain I see that it is missing top 6 certificates (5 intermediates and root).

2. With Firefox 42, I got SSL_ERROR_RX_RECORD_TOO_LONG.  After rebuilding with SSL debug logging on, I noticed this error was set in nss/lib/ssl/ssl3gthr.c : ssl3_GatherData()

Expected results:

RFC 5246 doesn’t put a limit on the maximum number of fragments the handshake messages can be broken into.  The quantum-resistant algorithms I was using required at most 4 – 5 fragments in the most extreme case.

Connecting with Chrome using BoringSSL for TLS worked fine, with all 26 certificates showing correctly.
Are you sure that this is a defect in NSS and not a limitation of the PKI validation library?

If the messages were indeed truncated I would expect some sort of TLS error at the handshake stage (more like you see with Firefox 42).
I did all of my original experiments with Firefox 42 and found the function in the ssl code that was setting the SSL_ERROR_RX_RECORD_TOO_LONG error.  Reducing the total size of ServerHello|ServerCertificate|ServerKeyExchange|ServerHelloDone to just under 32Kbytes worked fine.

With Firefox 47, I wanted to ensure the issue remained before logging a bug, but didn’t investigate further.  I just assumed the Firefox mapped the ssl error to a more generic sec error.
I suspect that that's not true. Do you have a test server that we can look at?
I am having trouble reproducing the SSL_ERROR_RX_RECORD_TOO_LONG, even with Firefox 42 now.  Perhaps this bug should be closed until I can reproduce it again.

I investigated the SEC_ERROR_UNKNOWN_ISSUER further and found, as you suspected, a limitation in the PKI library.  It only allows 6 intermediate certificates at the most (max 8 for the whole chain, including the root and the end entity).
Alexander: sounds like this shouldn't block you, then

Forwarding to Keeler for processing as part of moz::pkix
Flags: needinfo?(dkeeler)
The path length limit is by design to avoid stack exhaustion. We could increase the limit, but I don't think there's a practical limit we could set where someone couldn't come along and say, "hey, it doesn't work for my setup of n certificates". Perhaps the best thing would be to return a more descriptive error code/message.
Assignee: nobody → nobody
Component: Libraries → Security: PSM
Ever confirmed: true
Flags: needinfo?(dkeeler)
Product: NSS → Core
Summary: TLS handshake message spanning more than two 16K message fragments is not supported → mozilla::pkix supports certificate trust path lengths of 8 or less
Whiteboard: [psm-backlog]
Version: 3.19.3 → Trunk
Yeah, the current limit seems reasonable.  The reason I reached it, is because I was trying to artificially increase the length of the Certificate message to see if the handshake message fragmentation worked properly.
You need to log in before you can comment on or make changes to this bug.