97381 - bogus decryption can cause incorrect behavior of PK11_DigestFinal

Status: RESOLVED FIXED

(NSS :: Libraries, defect, P1)

Description

[Jamie Nicolson]

This code in NSC_DecryptFinal will cause problems if the decrypted block is not
well-formed (for example, the key is incorrect):

rv = (*context->update)(context->cipherInfo, pLastPart, &outlen,
                 maxout, context->padBuf, context->blockSize);
if (rv == SECSuccess) {
   unsigned int padSize =
          (unsigned int) pLastPart[context->blockSize-1];
   *pulLastPartLen = outlen - padSize;
}

If the decrypted block is not well-formed, padSize will be set to something
arbitrary, and pulLastPartLen will be passed out of the function with a bogus
value, usually a very large unsigned int. Later code might try to allocate this
space, which will generate an out of memory condition.

I propose that if padSize > blockSize+1, the function should fail.

Comment 1

[Wan-Teh Chang]

Assigned the bug to Bob.

Comment 2

[Robert Relyea]

The code now makes the following check:

if ((padSize > context->blockSize) || (padSize == 0)) {
   fail;
}

padSize to be valid must be between 1 and countext->blockSize