325498 - ECDSA verify does not conform to PKCS#11 v2.20

Status: VERIFIED FIXED

(NSS :: Libraries, defect)

Description

[Andreas Sterbenz]

I am using NSS 3.11 plus ECC patch from
https://bugzilla.mozilla.org/show_bug.cgi?id=236245 .

PKCS#11 v2.20 defines an ECDSA signature as "an octet string of even length which is *at most* two times nLen octets, where nLen is the length in octets of the base point order n." (my emphasis) See section 12.3.1.

However, if a signature less than 2*nLen long is passed to C_VerifyFinal() for CKM_ECDSA_SHA1, NSS crashes (pstack of core below).

Note that the relevant specification text is new in v2.20. v2.11 required the signature to be exactly 2*nLen bytes long. The problem is that the size of the underlying curve may not be readily available to the application. Even apart from that, a crash is obviously not acceptable, so this should be fixed.

 --- called from signal handler with signal 11 (SIGSEGV) ---
 fee645f4 memset   (5f147d98, cc839c7f) + 54
 d6772328 mp_clear (fe5cb960) + 60
 d6763420 ECDSA_VerifyDigest (86aeaa0, fe5cb9d4, fe5cb9c8) + 758
 d68e9184 ECDSA_VerifyDigest (86aeaa0, fe5cb9d4, fe5cb9c8) + 74
 d68cf369 nsc_ECDSAVerifyStub (86aea98, 838ff68, 30, fe5cba1c, 14) + 59
 d68d14e0 NSC_VerifyFinal (2, 838ff68, 30) + b0
 d6b18aba Java_sun_security_pkcs11_wrapper_PKCS11_C_1VerifyFinal (80a84e4, fe5cbb0c, 2, 0, fe5cbb00, 0) + 62

Comment 1

[Wan-Teh Chang]

Andreas,

Thanks for the bug report.

The crash of ECDSA_VerifyDigest inside mp_clear has been
fixed in bug 320583.

The length of the signature that ECDSA_VerifyDigest accepts
has been relaxed in bug 320589 (see bug 320589 comment 3,
item 4).  Now ECDSA_VerifyDigest only requires that the
signature's length be even; the length can be longer than
2*nLen octets as long as each signature component (r' and s')
is still less than the base point order (n), that is, the
portions of r' and s' in excess of nLen octets are leading
zeros.

To verify this bug, please use NSS_3_11_BRANCH.  If you
want to verify the crash and verification of a short
signature separately, you also need to
cvs update mozilla/security/nss/lib/freebl/ec.c with the
following two revisions:

-r 1.9.2.2  ECDSA_VerifyDigest won't verify a short
            signature, but it should not crash.
-r 1.9.2.4  ECDSA_VerifyDigest should verify a short signature.