1174384 - mozpkix throws SEC_ERROR_BAD_DER on certificates signed by internal CA

Status: RESOLVED DUPLICATE of bug 1166216

(Core :: Security: PSM, defect)

Description

[Jonathan Vaughn]

Attachment: example.com_test_certificates.zip

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0
Build ID: 20150513174244

Steps to reproduce:

Problem has occurred since at least FF37. Last known working was FF34 - versions in between were not tested. Still occurs as of FF39b.

We have an internal CA that signs certificates for our websites / devices / etc, and they worked fine with the root CA added to the trust store on FF34 and older at least. Chrome and IE work too, as well as Safari (across all their various platforms). Only FF37+ has stopped working. All SSL testing sites / tools I've tried indicate the certificates are fine.

To eliminate things such as many names in SAN and IP addresses and other such items, I created a new root CA for testing purposes, and then used it to sign a certificate for example.com (also, so that I can attach these without leaking internal website information). Using hosts to override DNS and with a machine set up to respond to example.com, I get the same error as do all our other sites.

See attached certificates in zip example.com_test_certificates.zip for Root CA cert and Server cert for example.com, which show this error (trust the Root CA in your FF37+ and then connect to a server using the server cert with name example.com).

Since all other browsers work, I consider this to be some kind of regression / bug on Firefox's part, but if it's going to take weeks/months for a fix to trickle down into release, and you can tell me what specifically FF chokes on, I can try to change how the certificates are generated to make FF happy in the mean time.


Actual results:

An error occurred during a connection to example.com:443. security library: improperly formatted DER-encoded message. (Error code: sec_error_bad_der) 


Expected results:

example.com should load

Comment 1

[Dana Keeler (she/her) [:keeler]]

Hi Jonathan, thanks for filing this. Looks like those certificates have the same issue outlined in bug 1166216 comment 7 (namely, the subject public key information in those certificates are not properly encoded according to the relevant specifications).

Comment 2

[Jonathan Vaughn]

Thank you. It looks like we're missing the NULL after rsaEncryption, though at least our INTEGERs are encoded correctly it appears.

I should be able to fix our certificate generation with this information.