When given any number that is equal to 3 mod 4, mpp_pprime says that number is prime. The bug is simply that "res" is not set to MP_NO prior to the for (jx = 1; jx < b; jx++) loop, and if that loop executes zero times (because b is 1), the result is MP_OK, not MP_NO. Also, HoAC suggests using sequential values of a, rather than random values of a.
mpp_pprime is the Miller Rabin pseudo-primality test. I will check in a fix shortly. I'm also going to do some other cleanup to the code.
I believe this bug is fixed in revision 1.13 of mpprime.c. It passes the new test case I added to mpi-test
Changed target milestone to show this bug was fixed in NSS 3.1.1