2075850 - C_Sign with CKM_EDDSA rejects a buffer of the size its own length query returns (CKR_ARGUMENTS_BAD unless the buffer is exactly 64 bytes)

Status: UNCONFIRMED

(NSS :: Libraries, defect, P3)

Description

[Eric Knauel]

Attachment: eddsa_sign_buffer.c

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/152.0.0.0 Safari/537.36

Steps to reproduce

build and run the attached eddsa_sign_buffer.c against softoken, with a fresh database created by certutil -N:

cc -I <dir containing the OASIS pkcs11.h> -o eddsa_sign_buffer eddsa_sign_buffer.c -ldl
NSS_LIB_PARAMS=configDir=<db dir> ./eddsa_sign_buffer <path to libsoftokn3> <user PIN>

The program:

1. generates an Ed25519 key pair (CKM_EC_EDWARDS_KEY_PAIR_GEN, curve given by OID 1.3.101.112);
2. asks C_Sign(CKM_EDDSA) for the signature length with a NULL buffer;
3. signs into a buffer of exactly that length;
4. repeats the signature with buffers of 64, 65 and 1024 bytes, and verifies each signature that succeeds.

Actual results

size query (NULL buffer) announces 144 bytes

signing into a buffer of the announced size:
  buffer  144  ->  rv 0x7 CKR_ARGUMENTS_BAD, length 144

other buffer sizes:
  buffer   64  ->  rv 0x0, length 64
                   verify -> rv 0x0
  buffer   65  ->  rv 0x7 CKR_ARGUMENTS_BAD, length 65
  buffer 1024  ->  rv 0x7 CKR_ARGUMENTS_BAD, length 1024

Only a buffer of exactly 64 bytes works. A caller that follows the standard two-call convention (query the length, then sign) therefore never gets an Ed25519 signature. The attached pkcs11-spy log (NSS 3.120) shows the same calls.

Expected results

PKCS#11 3.1, section 5.2:

If pBuf is NULL_PTR, then all that the function does is return (in *pulBufLen) a number of bytes which would suffice to hold the cryptographic output produced from the input to the function. [...]
If pBuf is not NULL_PTR, then *pulBufLen MUST contain the size in bytes of the buffer pointed to by pBuf. If that buffer is large enough to hold the cryptographic output [...], then that cryptographic output is placed there, and CKR_OK is returned by the function and *pulBufLen is set to the exact number of bytes returned.

So C_Sign into the 144-byte buffer (and into 65 or 1024 bytes) should return CKR_OK with *pulSignatureLen = 64.

For comparison, CKM_ECDSA on the same token behaves correctly. It also announces 144 bytes, accepts that buffer, and returns the shorter signature (132 bytes for P-521).

Comment 1

[Eric Knauel]

Attachment: eddsa_sign_buffer-pkcs11-spy.log