356756 - Subjective DOS vulnerability in NSS cert path validation

Status: NEW

(NSS :: Libraries, defect, P2)

Description

[Nelson Bolyard (seldom reads bugmail)]

Reports of a DOS attack/vulnerability that doesn't cause a crash but
merely causes a slow-down for the duration of the attack are pretty
subjective.  It's up to the owners of the code to decide how much of a 
slow down is really a DOS vulnerabilty, and whether the given amount
of slow down is within the bounds of acceptable performance, or whether
it constitutes a DOS vulnerability, and that's often a judgement call.  
This is one of those cases where a judgement call comes into play.

As part of the NISCC work done for NSS 3.9, we set limits on public
key sizes, both upper and lower bounds on sizes of key components.
For RSA public keys, the upper bound is now ~8kb for the modulus and
64b for the public exponent.  

According to some correspondence I've had with Dr Stephen Hensen, who
works on OpenSSL, as I understand it, OpenSSL considers code that 
validates a signature on a cert BEFORE validating that the issuer cert 
is trusted (or was issued by a trusted issuer) to be vulnerable to DOS.  
They consider the possibility of validating one or more signatures 
BEFORE determining that the chain does chain to a trust anchor to be a 
DOS vulnerability.  

I believe they are happy with code that either

a) works the chain from trust anchor/root to leaf, rather than in the
other direction, so that signatures are only checked after the issuer
cert is known to be trusted or known to be a descendent of a trusted 
issuer, or 

b) code that checks signatures in a second path, after the first path 
has verified that the chain leads up to a trust anchor.  

Consider the scenario of SSL client auth, where a malicious client sends
in a fabricated cert chain for the client cert, where each CA cert in the
chain has a maximum size RSA public key, and the number of certs in 
the chain is the maximum allowed (?) by NSS, and the root cert in that 
chain has the name of one of the server's trusted issuers, but is not 
actually one of the trusted issuer's certs (is not one of the servers
client auth trust anchors).  The NSS-based server will validate the
signatures in the chain, from leaf to root, consuming lots of CPU time
before realizing that the chain is all baloney, not chaining to a trust
anchor.  

Clearly, it takes fewer such attacker clients to bog down a server than 
if the server checks first that the chain goes to a trust anchor before
validating the signatures.

We have numerous options there, including (not limited to):

a) declaring that our existing key size limits and path length limits 
put NSS's performance within acceptable limits in this scenario, making
it NOT a DOS vulnerability, or 

b) tightening our limits (e.g. shorter path length limits), or 

c) changing our path validation algorithm to not check signatures until
after the chain is shown to lead to a trust anchor.

This is obviously more of an issuer for servers than for clients.

Comment 1

[Nelson Bolyard (seldom reads bugmail)]

I wrote:
> b) code that checks signatures in a second path, after the first path 
I meant to write "pass" not "path", in both places.

Comment 2

[Nelson Bolyard (seldom reads bugmail)]

Alexei, how big a job is it to change libPKIX to behave as described in
comment 0?

Comment 3

[Josh Aas]

Any update here?

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

The bug assignee is inactive on Bugzilla, and this bug has priority 'P2'.
:beurdouche, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 5

[Marco Castelluccio [:marco]]

We have modified the bot to only consider P1 as high priority, so I'm cancelling the needinfo here.