487630 - RSA premaster secret error.

Status: RESOLVED DUPLICATE of bug 492888

(JSS Graveyard :: Library, defect)

Description

[Slavomir Katuscak]

Build:
pool SunOS/sparc 64bit OPT on 2009/04/09 03:52:40 (Tinderbox)

It's SunFire T2000 with Niagara CPU

============= SSL Ciphersuite JSS Server with Bypass Off and JSSE client 
 ./startJssSelfServ.sh /export/tinderlight/data/pool_64_OPT/mozilla/dist/SunOS5.10_64_OPT.OBJ/../xpclass.jar /export/tinderlight/data/pool_64_OPT/mozilla/tests_results/jss/pool.1 localhost 8476 bypassOff /opt/jdk/1.6.0_01/SunOS64/jre/bin/java -d64 
/opt/jdk/1.6.0_01/SunOS64/jre/bin/java -d64 -classpath /export/tinderlight/data/pool_64_OPT/mozilla/dist/SunOS5.10_64_OPT.OBJ/../xpclass.jar org.mozilla.jss.tests.JSS_SelfServServer /export/tinderlight/data/pool_64_OPT/mozilla/tests_results/jss/pool.1 passwords localhost false 8476 bypassOff verboseoff &
***FilePasswordCallback returns m1oZilla
JSS_SelfServServ localhost ready to accept connections on 8476

SSL Server is envoked using port 8476 
/opt/jdk/1.6.0_01/SunOS64/jre/bin/java -d64 -cp /export/tinderlight/data/pool_64_OPT/mozilla/dist/SunOS5.10_64_OPT.OBJ/../xpclass.jar org.mozilla.jss.tests.JSSE_SSLClient /export/tinderlight/data/pool_64_OPT/mozilla/tests_results/jss/pool.1 8476 localhost JSS 
using port: 8476
Testing Connection:localhost:8476
connect isBound
javax.net.ssl.SSLKeyException: RSA premaster secret error
	at com.sun.net.ssl.internal.ssl.RSAClientKeyExchange.<init>(RSAClientKeyExchange.java:97)
	at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverHelloDone(ClientHandshaker.java:574)
	at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:197)
	at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:511)
	at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:449)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:817)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1029)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1056)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1040)
	at org.mozilla.jss.tests.JSSE_SSLClient.testSSLSocket(JSSE_SSLClient.java:432)
	at org.mozilla.jss.tests.JSSE_SSLClient.testCiphersuites(JSSE_SSLClient.java:195)
	at org.mozilla.jss.tests.JSSE_SSLClient.main(JSSE_SSLClient.java:579)
Caused by: java.security.InvalidKeyException: wrap() failed
	at sun.security.pkcs11.P11RSACipher.engineWrap(P11RSACipher.java:395)
	at javax.crypto.Cipher.wrap(DashoA13*..)
	at com.sun.net.ssl.internal.ssl.RSAClientKeyExchange.<init>(RSAClientKeyExchange.java:95)
	... 11 more
Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_MECHANISM_INVALID
	at sun.security.pkcs11.wrapper.PKCS11.C_WrapKey(Native Method)
	at sun.security.pkcs11.P11RSACipher.engineWrap(P11RSACipher.java:391)
	... 13 more
JSSTEST_CASE 24 (SSL Ciphersuite JSS Server with Bypass Off and JSSE client): FAILED return value 1

============= SSL Ciphersuite JSS Server with Bypass On and JSSE client 
 ./startJssSelfServ.sh /export/tinderlight/data/pool_64_OPT/mozilla/dist/SunOS5.10_64_OPT.OBJ/../xpclass.jar /export/tinderlight/data/pool_64_OPT/mozilla/tests_results/jss/pool.1 localhost 8477 bypass /opt/jdk/1.6.0_01/SunOS64/jre/bin/java -d64 
/opt/jdk/1.6.0_01/SunOS64/jre/bin/java -d64 -classpath /export/tinderlight/data/pool_64_OPT/mozilla/dist/SunOS5.10_64_OPT.OBJ/../xpclass.jar org.mozilla.jss.tests.JSS_SelfServServer /export/tinderlight/data/pool_64_OPT/mozilla/tests_results/jss/pool.1 passwords localhost false 8477 bypass verboseoff &
***FilePasswordCallback returns m1oZilla
JSS_SelfServServ localhost ready to accept connections on 8477

SSL Server is envoked using port 8477 
/opt/jdk/1.6.0_01/SunOS64/jre/bin/java -d64 -cp /export/tinderlight/data/pool_64_OPT/mozilla/dist/SunOS5.10_64_OPT.OBJ/../xpclass.jar org.mozilla.jss.tests.JSSE_SSLClient /export/tinderlight/data/pool_64_OPT/mozilla/tests_results/jss/pool.1 8477 localhost JSS 
using port: 8477
Testing Connection:localhost:8477
connect isBound
javax.net.ssl.SSLKeyException: RSA premaster secret error
	at com.sun.net.ssl.internal.ssl.RSAClientKeyExchange.<init>(RSAClientKeyExchange.java:97)
	at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverHelloDone(ClientHandshaker.java:574)
	at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:197)
	at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:511)
	at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:449)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:817)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1029)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1056)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1040)
	at org.mozilla.jss.tests.JSSE_SSLClient.testSSLSocket(JSSE_SSLClient.java:432)
	at org.mozilla.jss.tests.JSSE_SSLClient.testCiphersuites(JSSE_SSLClient.java:195)
	at org.mozilla.jss.tests.JSSE_SSLClient.main(JSSE_SSLClient.java:579)
Caused by: java.security.InvalidKeyException: wrap() failed
	at sun.security.pkcs11.P11RSACipher.engineWrap(P11RSACipher.java:395)
	at javax.crypto.Cipher.wrap(DashoA13*..)
	at com.sun.net.ssl.internal.ssl.RSAClientKeyExchange.<init>(RSAClientKeyExchange.java:95)
	... 11 more
Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_MECHANISM_INVALID
	at sun.security.pkcs11.wrapper.PKCS11.C_WrapKey(Native Method)
	at sun.security.pkcs11.P11RSACipher.engineWrap(P11RSACipher.java:391)
	... 13 more
JSSTEST_CASE 25 (SSL Ciphersuite JSS Server with Bypass On and JSSE client): FAILED return value 1