372628 - certificates larger than 8kbit (16384) not handled correctly

Status: RESOLVED WONTFIX

(NSS :: Libraries, defect)

Description

[Szikra Istvan]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.0.10) Gecko/20070216 Firefox/1.5.0.10
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.0.10) Gecko/20070216 Firefox/1.5.0.10

I have an Apache 2.0.59 server with mod_ssl OpenSSL 0.9.8e.
I created a self signed Root CA with 16kbit long key, a CA with 8kbit long key signed with Root CA, a web server certificate 2kbit long signed with 2nd CA.
configured ssl.conf SSLCertificateFile, SSLCertificateKeyFile, SSLCertificateChainFile / SSLCACertificateFile.
IE 7 and Opera 8 and 9 works fine, handles the certificate chain...
Firefox 2.0.0.2, SeaMonkey 1.1 gives an error message:
"Could not establish an encrypted connection because certificate presented by * has an invalid signature."
Firefox 1.5.0.10 gives a similar error with Error Code: -8182
the error message (and other stuff) can be found here:
http://stevenspark.no-ip.info/tutorials/https/FF1/
the server and the certificate cahin can be tested here:
https://foton.no-ip.com/
(it's on an ADSL line so don't except much)
Browsers tested on Win2k3 (SP1) and XP (SP2)

I also tried some other certificate chain configs:
"16k+4k+1k-FAIL"
"16k+8k+2k-FAIL"
"4k+2k+1k-OK"
"8k+4k+2k-OK"
"8k+8k+2k-OK"
"8k+8k+8k-OK"
the first is the bit lenght of the Root CA the last is the web server's...
now I'm trying to test with 32kbit key, but it's taking a while generating it :)

If I install manually the Root and the 2nd CA into firefox, then it works, it's all explained at the link above.
There are other interresting stuff is the certificate chain contains only the 2nd CA but not the Root:
 if Mid CA installed web trusted -> OK
 if root is installed (any way), (and mid is not installed, or not web trusted) -> ERROR
 if root is not installed, (Mid CA is not installed, or not web trusted) -> warning



Reproducible: Always

Steps to Reproduce:
Setup:
1-Create Self Signed Root CA with CA.pl ("CA-Root.pl -newca") (with 16kbit key)
2-run the script again to create a second Mid CA (self signed) ("CA.pl -newca")
  sign the Mid CA with the Root CA ("CA-Root.pl -signCA")
  (relpace the self signed cert with the Root CA signed)
3-create a CSR for web server ("CA.pl -newreq")
4-sign the CSR with Mid CA ("CA.pl -sign")
5-remove password from web server private key ("openssl rsa -in key -out key.pem")
6-copy cert.pem and key.pem to apache conf dir
  concate root and mid CA crtificate in one file (ca.crt) and copy to apache conf dir

7-setup ssl.conf:
SSLCertificateFile conf/ssl.crt/cert.pem
SSLCertificateKeyFile conf/ssl.key/key.pem
SSLCertificateChainFile conf/ssl.crt/ca.crt

8-restart apache


Actual Results:  
9-test with browsers

-IE7 -  warning, cert chain ok, install root cert, restart IE7, OK!
-Opera9 - warning, all certs in chain ok, install Root cert, OK!
-Firefox - ERROR -8182!
-SeaMonkey - ERROR!


Expected Results:  
OK, a 16kbit key might seem an overkill, but still it should work...

10-WorkAround:
-Manual install (Root and) Mid CA certificate (from disk, http, ftp...) under Firefox, 
 restart firefox, connects fine, certificate ok, cert chain ok

Here is a test chain generator package:
http://stevenspark.no-ip.info/tutorials/https/CA/OpenSSLCAtest.zip
it contains:
CreateCA.bat
CreateRootCA.bat
job.bat
job_low.bat
newcert.bat
openssl.cnf
opensslRoot.cnf
opensslRootCreate.cnf
CA.pl
CA-Root.pl
CA-perl.txt
CARoot-perl.txt
config.txt
info.txt
newcert.txt

The http://stevenspark.no-ip.info/tutorials/https/ca/info.txt contains some additional info regarding the bug.

Comment 1

[Szikra Istvan]

Attachment: Auto generate CAs, and certificates for test

the same pack (and other info) can be found at http://stevenspark.no-ip.info/tutorials/https/

Comment 2

[Nelson Bolyard (seldom reads bugmail)]

We have intentionally ikmposed an upper bound on the allowable sizes of keys.
So, this behavior is as intended, not a bug.

/data/lxr-data/security/mozilla/security/nss/lib/freebl/blapit.h,
    * line 123 -- #define RSA_MAX_MODULUS_BITS 8192
    * line 124 -- #define RSA_MAX_EXPONENT_BITS 64