306347 - JSS Key Pair generation fails with token Runtime Exception intermittently

Status: RESOLVED WONTFIX

(JSS Graveyard :: Library, defect)

Description

[Sandeep Konchady]

JSS Key Pair generation fails with token Runtime Exception intermittently

Error occured again 20050318 on mandela for 32-bit DBG for security310.
Generated 512-bit RSA KeyPair!
Modulus:
11225073746871895561973485952648240407085382041439799992187758171785047622195799120786867300839468320784116193530874523917269533928087826490155444977580609
Exponent: 65537
Generated 1024-bit RSA KeyPair!
Modulus:
161573903168255928249438464559208835537737544697161656354715098961686558480923350317097683069449455540112672649122606977120270849603771649364920835103473530919012250632708710356859771110375585411081310572342936219648888431158160337943452243388526483996810894233006963301069046635804400560267598205802460053231
Exponent: 65537
org.mozilla.jss.crypto.TokenRuntimeException: Keypair Generation failed on token:
        at
org.mozilla.jss.provider.java.security.JSSKeyPairGeneratorSpi.generateKeyPair(JSSKeyPairGeneratorSpi.java:85)
        at
java.security.KeyPairGenerator$Delegate.generateKeyPair(KeyPairGenerator.java:475)
        at java.security.KeyPairGenerator.genKeyPair(KeyPairGenerator.java:400)
        at org.mozilla.jss.tests.TestKeyGen.main(TestKeyGen.java:127)
TestKeyGen returned 1 at all.pl line 182.

Error occured on chavez for 32-bit DBG for securitytip.
Generated 512-bit RSA KeyPair!
Modulus:
9649553136748241194601118816343217167086028610540175554804269083200863547638953021725628380057810801381887553838829125427974741225628829606123563392524069
Exponent: 65537
Generated 1024-bit RSA KeyPair!
Modulus:
151629452268403648140148176076893656981169627624735448771530768349482333140819047928171071945964716492197647556241497189745650015534641345572147900856042148200061054292415691041535798568409692068855690693341008045814115370442959055046764778996091997834076329539867915741977592122472329167584029642975385594313
Exponent: 65537
org.mozilla.jss.crypto.TokenRuntimeException: Keypair Generation failed on token:
        at
org.mozilla.jss.provider.java.security.JSSKeyPairGeneratorSpi.generateKeyPair(JSSKeyPairGeneratorSpi.java:85)
        at
java.security.KeyPairGenerator$Delegate.generateKeyPair(KeyPairGenerator.java:475)
        at java.security.KeyPairGenerator.genKeyPair(KeyPairGenerator.java:400)
        at org.mozilla.jss.tests.TestKeyGen.main(TestKeyGen.java:127)
TestKeyGen returned 1 at all.pl line 182.
Test OPT version

Note from Nelson:

org.mozilla.jss.crypto.TokenRuntimeException: Keypair Generation
> failed on token:

KeyPair generation can fail, but if/when it does so, it should be
retried.  KeyPair Generation has two find two large primes.  The
technique for finding one is to start with a large random number
and then search the space of the next 128k numbers (starting with
that random number) looking for a prime.  If no prime is found in
that space (which can happen) keygen fails.  Generally, the solution
is to try again with another random number.  You don't want to try
an infinite number of times, but you need to try more than once.

This process can take a LONG time. If the operation is being killed
because it ran too long, that's bad.  It needs to be allowed to run
as long as it wants.

We need to try the Keypair Generation Several times before failing.

Comment 1

[Sandeep Konchady]

Attachment: Modified the code to try generating the keys for 5 times before continuing on to test next key

As per Nelson's comments I have modified the code to loop for 5 times and try
generating the keys.  If we are not successful in generating the keys within 5
attempts, we move on to test the next key.  At the end of 5th attempt, an
exception stack trace will be thrown to let the test result analyzer know that
there was a problem.

Comment 2

[Sandeep Konchady]

Hello Wan-Teh,
  This fix is at the application layer.  Glen suggested that I check with you
what you think about a couple of alternates.

[1] Fix org/mozilla/jss/crypto/KeyPairGenerator.java at genKeyPair() to do the
same looping as I did in the test.  This will fix similar issues when other
programs call genKeyPair() for all algorithms.

[2] Keep the test code as is, and file a bug against NSS to fix it at the root.
 This will make the test code more robust but will need a fix at the root.

Sandeep

Comment 3

[Sandeep Konchady]

Attachment: Application layer change with 5 iterations per key

Application layer change with 5 iterations per key

Comment 4

[Wan-Teh Chang]

*** Bug 313974 has been marked as a duplicate of this bug. ***

Comment 5

[Nelson Bolyard (seldom reads bugmail)]

I have a few suggestions for improving the ability to diagnose problems 
with this code.

#1) In function main at http://lxr.mozilla.org/mozilla/source/security/jss/org/mozilla/jss/tests/TestKeyGen.java#67
print out a message announcing what type of key pair you're about to 
generate BEFORE trying to generate it, so that if the keygen throws an
exception, you'll know right away which one had the problem.  E.g.
"About to generate 512-bit RSA KeyPair with public exponent=3"

#2) In function Java_org_mozilla_jss_pkcs11_PK11KeyPairGenerator_generateRSAKeyPair at
http://lxr.mozilla.org/mozilla/source/security/jss/org/mozilla/jss/pkcs11/PK11KeyPairGenerator.c#168
in the error code path, if PR_GetErrorTextLength() returns zero, (which it
will almost always), then get the error code number and print that, or 
lookup the standard NSS/NSPR error string for that error code number and
print that, or both.

#3, In debug builds, in function generate_prime, at http://lxr.mozilla.org/mozilla/source/security/nss/lib/freebl/rsa.c#185
at the comment "/* keep going while err == MP_NO */", if err is MP_NO, 
perhaps we should dump out the value of the variable "prime" at that point.

#4) Perhaps the loop in function generate_prime should search the space
lineraly rather than randomly.  That is, call RNG_GenerateGlobalRandomBytes
once before the loop, not inside the loop.

#5) Perhaps we need to increase the value of MAX_PRIME_GEN_ATTEMPTS.

Comment 6

[Sandeep Konchady]

I have added the test TestKeyGen.java to bug 310491 along with other test changes and fix #1 from Nelson's recomendation below.  I will leave this bug open for now to fix the other solutions recomended by Nelson.

Comment 7

[glen beasley]

Comment on attachment 200703 [details] [diff] [review]
Application layer change with 5 iterations per key

please, look into finding the root cause.

Comment 8

[BMO Automation]

JSS development has moved from the Mozilla community to the Dogtag PKI community. Please re-file this bug at https://github.com/dogtagpki/jss if it is still relevant. Thank you!