1130580 - Avoid accepting ECDSA signatures when the digest needs to be truncated because the digest is larger than the curve's order

Status: RESOLVED INVALID

(Core :: Security: PSM, defect)

Description

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

One of the first steps in verifying an ECDSA signature is converting the digest into a bignum value that can be used in mathematical calculations in conjunction with the ECDSA public key. Those calculations need to be done within the field of the curve of the public key. This means in particular that digests that are larger than the order of the curve (e.g. a P-256 signature with a SHA-384 or SHA-512 digest, or a P-384 signature with a SHA-512 digest) need to be truncated in some way.

The NSA recommendations and the SECG specifications disagree about how to do the truncation. It seems everybody implements the SECG rules. Although I don't think we should blindly trust the NSA, I also think it might be better to avoid accepting any truncated digests in ECDSA signatures (i.e. accept only signatures that conform to both the NSA and SECG rules) rather than go against the NSA.

My suggestion is to remove support for SHA-512-based signatures completely, and then to remove support for SHA-384 signatures with P-256 keys. Also, we should document in the code clearly how and why we decided upon these restrictions, so that when we add support for new curves in the future, these considerations are addressed.

Background on implementations getting this wrong and fixing the issue rather recently:

gcrypt/gnutls/gnupg (fixed January 2014):
http://lists.gnupg.org/pipermail/gcrypt-devel/2013-December/002776.html
https://bugs.gnupg.org/gnupg/issue1583

Golang (fixed March 2011):
https://groups.google.com/forum/#!topic/golang-dev/CSL5EdWuyN8

OpenSSL (fixed December 2009):
http://marc.info/?l=openssl-cvs&m=125968875801044

NSS (fixed 2006): bug 323817.

Comment 1

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

Note that we probably can't avoid doing *some* truncation of the top bits of a SHA-384 digest in a P-384 signature or a SHA-256 digest with a P-256 signature, because the truncation isn't just to the size of the curve, but to a value less than the *order* of the curve. The curve order for P-384 is less than 2^384-1 and the curve order for P-256 is less than 2^256-1.

The point here, though, is to avoid the case of extreme, somewhat nonsensical truncations of 128+ or or 256+ bits.

Comment 2

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

The stuff I mentioned above doesn't really matter.