1806496 - In FIPS mode, nss should reject RSASSA-PSS salt lengths larger than the output size of the hash function used, or provide an indicator

Status: RESOLVED FIXED

(NSS :: Libraries, enhancement)

Description

[Robert Relyea]

In FIPS 140-3, non-fips actions should be rejected or marked as non-fips. In NSS we do this with indicators controlled in the table in nss/softoken/fips-algorithms.h . Most of these indications are processed automatically, but sometimes there are algorithmic specific checks which are necessary.

PSS is one such case. In FIPS mode, the salt length should not exceed the output length of the underlying hash function. We need a specific check for this condition so that fips-algoithms.h can make PSS indicators dependent on this behavior.

Comment 1

[Robert Relyea]

Attachment: WIP: Bug 1806496 In FIPS mode, nss should reject RSASSA-PSS salt lengths larger than the output size of the hash function used, or provide an indicator

This patch adds a new mechanism specific check for PSS in fip_algorithms.h.
The new check uses the hash mechanism provided in the pss mechanism list to look up the hash length. A static utility function in pkcs11c.c is moved to pkcs11u.c and made global so it can be reused in this code.

We know that mechanism supplied in the parameters matches the hash because that check is enforces in pkcs11c.c for the combined hash and signed functions.

Comment 2

[Robert Relyea]

Checked in: https://hg.mozilla.org/projects/nss/rev/d2c2ef572b23ebd765dd000cb784dc9831809d30