1930800 - shlibsign is broken in System FIPS mode.

Status: RESOLVED FIXED

(NSS :: Libraries, defect, P3)

Description

[Robert Relyea]

Due to the update to the PKCS #11 module to select FIPS or non-FIPS based on the the state of the system, and the fact that shlibsign no longer works in raw FIPS mode when we went to hmac rather than DSA signing, if you run shlibsign now on a system in FIPS mode it will fail. Shlibsign currently has an option to force running in FIPS mode.

We need to:

1. Fix shlibsign so it works in FIPS mode (either by generating our HMAC key externally and importing it, or by using unwrap/decrypt tricks to export the key when in FIPS mode).
2. Update shlibsign so you can force non-FIPS mode.
3. We can continue to default to the system settings of FIPS mode if neither of the explicit options are selected.

(NOTE: downstream has a simple patch which restores the previous behavior where the tool runs in non-FIPS mode unless FIPS mode is explicitly selected but calling NSC_GetFunctionList rather C_GetFunctionList

Comment 1

[Robert Relyea]

Attachment: Bug 1930800 shlibsign is broken in System FIPS mode.

1. add the -C function to force non-FIPS mode in shlibsign.
2. update shlibsign to fetch the following pkcs #11 interfaces:
   a. NSC_ functions when -C is supplied (always use non-FIPS mode).
   b. C_ functions when neither -C nor -F is supplied (fetch non-FIPS if the system is non-FIPS otherwise fetch FIPS).
   .c FC_ functions when -F is supplied (always use FIPS mode).
3. update the hmac portion to generate the key externally and import it in FIPS mode so we don't have to extract it.
   a. record the key in the case it was provided to us so we don't have to extract it later.
   b. refactor how this happens into smaller functions so we can reuse them in FIPS mode.
4. fix error message issue by initializing the NSS error strings.
5. remove OS/2 specific .cmd file.
6. add -C to the build. In the build case we are using the newly built libraries to sign themselves. They can't function in FIPS mode until they are signed, so always sign them in non-FIPS mode as part of the build. (releases that want to sign them in FIPS mode can resign them after the initial signature).
7. add test to make sure FIPS mode of shlib sign continues to work.