I created a self-signed server certificate and added it to my NSS certificate database with the "P,," trust flags. This allows CERT_VerifyCertNow to verify the certificate successfully. However, CERT_PKIXVerifyCert still fails with the SEC_ERROR_UNKNOWN_ISSUER error (-8179). I have to set the "C,," trust flags to make CERT_PKIXVerifyCert succeed, but trusting a self-signed server certificate as a CA ('C' means "trusted CA to issue server certificates") gives more trust to the certificate than necessary.
I'm using this for a specific domain's self signed certificate. Does anyone know if the "C" flag allows the domain to sign arbitrary other domains, like gmail.com? Thanks, Nicholas
Nicholas, questions like yours should be asked in mozilla.dev.tech.crypto newsgroup or dev-tech-crypto mailing list.
Marked this bug as a duplicate even though this bug was filed first. The other bug has more info.