I am using NSS 3.11 plus ECC patch from https://bugzilla.mozilla.org/show_bug.cgi?id=236245 . PKCS#11 v2.20 defines an ECDSA signature as "an octet string of even length which is *at most* two times nLen octets, where nLen is the length in octets of the base point order n." (my emphasis) See section 12.3.1. However, if a signature less than 2*nLen long is passed to C_VerifyFinal() for CKM_ECDSA_SHA1, NSS crashes (pstack of core below). Note that the relevant specification text is new in v2.20. v2.11 required the signature to be exactly 2*nLen bytes long. The problem is that the size of the underlying curve may not be readily available to the application. Even apart from that, a crash is obviously not acceptable, so this should be fixed. --- called from signal handler with signal 11 (SIGSEGV) --- fee645f4 memset (5f147d98, cc839c7f) + 54 d6772328 mp_clear (fe5cb960) + 60 d6763420 ECDSA_VerifyDigest (86aeaa0, fe5cb9d4, fe5cb9c8) + 758 d68e9184 ECDSA_VerifyDigest (86aeaa0, fe5cb9d4, fe5cb9c8) + 74 d68cf369 nsc_ECDSAVerifyStub (86aea98, 838ff68, 30, fe5cba1c, 14) + 59 d68d14e0 NSC_VerifyFinal (2, 838ff68, 30) + b0 d6b18aba Java_sun_security_pkcs11_wrapper_PKCS11_C_1VerifyFinal (80a84e4, fe5cbb0c, 2, 0, fe5cbb00, 0) + 62
Andreas, Thanks for the bug report. The crash of ECDSA_VerifyDigest inside mp_clear has been fixed in bug 320583. The length of the signature that ECDSA_VerifyDigest accepts has been relaxed in bug 320589 (see bug 320589 comment 3, item 4). Now ECDSA_VerifyDigest only requires that the signature's length be even; the length can be longer than 2*nLen octets as long as each signature component (r' and s') is still less than the base point order (n), that is, the portions of r' and s' in excess of nLen octets are leading zeros. To verify this bug, please use NSS_3_11_BRANCH. If you want to verify the crash and verification of a short signature separately, you also need to cvs update mozilla/security/nss/lib/freebl/ec.c with the following two revisions: -r 184.108.40.206 ECDSA_VerifyDigest won't verify a short signature, but it should not crash. -r 220.127.116.11 ECDSA_VerifyDigest should verify a short signature.