If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Implement the continuous random number generator test on rand()



11 years ago
11 years ago


(Reporter: Wan-Teh Chang, Assigned: Wan-Teh Chang)


Firefox Tracking Flags

(Not tracked)



(3 attachments, 1 obsolete attachment)



11 years ago
The mpi library uses the Standard C Library function rand()
in the randomized algorithm Miller-Rabin primality test
(mpp_pprime).  rand() is not a FIPS-Approved random number
generator.  It is fine to use rand() for this purpose, but
we still need to perform the continuous random number test
on rand().

We need to fix this bug under the following constraints.
- On the NSS_3_11_BRANCH, we should not modify the code in
  lib/freebl/*.c that was exercised by the FIPS algorithm
  tests.  This is to avoid re-doing any FIPS algorithm test.
  This constraint doesn't apply to the NSS trunk.
- If the conditional RNG test on rand() fails, it must
  cause its caller to fail with the error code
  SEC_ERROR_LIBRARY_FAILURE so that the softoken can
  enter the Error state and return CKR_DEVICE_ERROR
  (FIPS 140-2 requirement AS09.04).  Here is the list of the
  freebl functions that call mpp_pprime, the only direct caller
  of rand():
  * lib/freebl/pqg.c: PQG_ParamGenSeedLen and PQG_VerifyParams.
    The call stack is
    PQG_ParamGenSeedLen/PQG_VerifyParams -> mpp_pprime.  They
    are used in the FIPS DSA algorithm tests.
  * lib/freebl/rsa.c: RSA_NewKey. The call stack is:
    RSA_NewKey -> generate_prime -> mpp_make_prime -> mpp_pprime
    This function is used in the FIPS RSA algorithm tests.
  * lib/freebl/dh.c: DH_GenParam. The call stack is:
    DH_GenParam -> mpp_make_prime -> mpp_pprime.  This function
    is not used by the softoken or any FIPS algorithm test.
- After a continuous RNG test failure, the user must be
  able to clear the error flag on rand() and resume normal
  operation by calling FC_Finalize and FC_Initialize (FIPS
  140-2 requirement AS04.03).


11 years ago
Priority: -- → P1

Comment 1

11 years ago
Created attachment 233161 [details] [diff] [review]
Proposed patch

I defined a wrapper for rand() in freebl called freebl_rand().
mpi continues to use rand() by default, but if mpi is built
as part of freebl, it uses freebl_rand() instead.  I added
a new macro MP_IN_FREEBL, which is defined when we compile
mpi from the freebl directory.

freebl_rand() uses some global variables.  These global variables
must be initialized with a rand_Init() call and freed with a
rand_Shutdown() call.  The call-once function rng_init() calls
rand_Init(), and RNG_RNGShutdown calls rand_Shutdown().

The continuous random number test is specified in FIPS 140-2
Section 4.9.2.  It has two cases, depending on how many bits
each call to the RNG produces.  Each call to rand() produces
an 'int' between 0 and RAND_MAX.  On some platforms such as
Windows, RAND_MAX is 32767, which is 15 bits, so unfortunately
we need to implement both cases of the continuous random number

Now mpp_random needs to handle the possible -1 return from
RANDOM().  In that case, mpp_random() returns MP_UNDEF, which
is the only MPI error code mapped to SEC_ERROR_LIBRARY_FAILURE
by the MP_TO_SEC_ERROR macro used by most freebl functions.
Unfortunately PQG_ParamGenSeedLen and PQG_VerifyParams don't map
MPI error code returned by mpp_pprime to NSS error code, so I'll
need to create more patches to address that.

Then I made sure we check the return values of all the functions
in the call stacks I listed.
Attachment #233161 - Flags: review?(nelson)

Comment 2

11 years ago
It turns out that we don't need to perform the continuous
random number generator test on rand(), so I marked this
bug invalid.  The investigation done in this bug may be
useful if we replace rand() with a thread-safe pseudorandom
number generator function in the future.
Last Resolved: 11 years ago
Resolution: --- → INVALID


11 years ago
Attachment #233161 - Flags: review?(nelson)

Comment 3

11 years ago
Created attachment 233175 [details] [diff] [review]
Patch to handle continuous RNG test failure in PQG_VerifyParams

Looking at PQG_ParamGenSeedLen more closely, I found that it
does map the MPI error code returned by mpp_pprime to NSS error
code.  So only PQG_VerifyParams needs work.

If we use the FIPS Approved RNG for Miller-Rabin primality test
in the future, the continuous RNG test may fail in PQG_VerifyParams.
This patch will handle that.  I attached this patch here for future

Comment 4

11 years ago
Created attachment 233320 [details] [diff] [review]
Alternative patch: use our global RNG in mpi

It just occurred to me that my previous patches could
be easily adapted to implement freebl_rand() using our
global RNG.  So I created this patch.  Not sure if this
is a good idea though.

Comment 5

11 years ago
Do you think it is a good idea to replace the use of
rand() in MPI (for the Miller-Rabin probabilistic
primality test) by our FIPS RNG in freebl?

Comment 6

11 years ago
Created attachment 233322 [details] [diff] [review]
Alternative patch: use our global RNG in mpi (corrected)

This is the correct patch (without the fix for another bug).
Attachment #233320 - Attachment is obsolete: true

Comment 7

11 years ago
In general I think it would be a good idea, though at this point we know it's not necessary for FIPS, so I wouldn't do it in 3.11.

You need to log in before you can comment on or make changes to this bug.