Closed Bug 1398269 Opened 6 years ago Closed 5 years ago

DigiCert: Non-BR-Compliant OCSP Responders

Categories

(CA Program :: CA Certificate Compliance, task)

Product:
CA Program
Component:
CA Certificate Compliance
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: kwilson, Assigned: jeremy.rowley)

References

Details

(Whiteboard: [ca-compliance] [ocsp-failure]) 

Problems have been found with OCSP responders for this CA, and reported in the mozilla.dev.security.policy forum here:

https://groups.google.com/d/msg/mozilla.dev.security.policy/o1MX07iWDco/RuM1NK_0AQAJ

As per section 4.9.10 of the BRs, OCSP responders MUST NOT respond with a “good” status for unissued certificates. The effective date for this requirement was 2013-08-01.

Please provide an incident report in this bug, as described here:
https://wiki.mozilla.org/CA/Responding_To_A_Misissuance#Incident_Report
I will file separate bugs for the subCAs, if requested by DigiCert.
I don't think separate bugs are necessary.  There are essentially only four entities involved with the OCSP responder bug.  They are ABB, Intesa Sanpaolo, Multicert/SCEE and Wells Fargo.  We are asking each of them to provide an incident report.  Meanwhile, for what it's worth, as an update for ABB, we are in the process of planning a certificate ceremony to add IPv6 constraints to the ABB Intermediate CA 3 (to replace serial number 0727cd79, which is the parent of the ABB Issuing CA 6) so that it is properly technically constrained in line with section 7.1.5 of the Baseline Requirements.  The aforementioned CA certificate is only missing the IPv6 constraints and is otherwise technically constrained.
I included Wells Fargo on their other bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1397963. Do you want me to replicate here?
(In reply to Jeremy from comment #3)
> I included Wells Fargo on their other bug:
> https://bugzilla.mozilla.org/show_bug.cgi?id=1397963. Do you want me to
> replicate here?

No need to replicate the data for Well Fargo subCA in this bug. It's fine to keep all of that subCA's response in Bug #1397963.
It's now been 13 days; have any of the SubCAs provided an incident report?

Gerv
Wells Fargo did, which I captured in the other bug. I'll provide more details to Ryan today. There aren't a lot, but I've pinged Justin at Wells Fargo to get additional information about why the misconfigured the servers. Sounds like they just screwed up the deployment since the OCSP issue is opposite of the RFC.  

ABB is supposed to be technically constrained against TLS, which means that the BRs aren't supposed to apply. As Ben noted, we improperly created the issuing CA, missing the IPv6 restriction. We've brought this up a couple of times in the Mozilla community, and I thought the takeaway was that it was a non-issue on the IPv6 error.  However, as this keeps arising, we are going to fix the technical constraint in the next key ceremony to take them out of scope of the BRs.

SCEE only uses the issuing CAs for The CAs “Cartão de Cidadão 001” and “Cartão de Cidadão 002” are used only to issue certificates for Citizen. SCEE didn't seem to understand that the BRs applied equally to these certificates. I'm not sure how they didn't understand that the BRs applied to these certificates as every communication to the SubCA emphasizes the need to comply with the BRs. Because they are only issuing client certificates, we are adding them to OneCRL today. 

I'm trying to find the info from Intesa
(In reply to Jeremy from comment #6)
> ABB is supposed to be technically constrained against TLS, which means that
> the BRs aren't supposed to apply. As Ben noted, we improperly created the
> issuing CA, missing the IPv6 restriction. We've brought this up a couple of
> times in the Mozilla community, and I thought the takeaway was that it was a
> non-issue on the IPv6 error.  However, as this keeps arising, we are going
> to fix the technical constraint in the next key ceremony to take them out of
> scope of the BRs.

Can you explain what you mean by "the BRs aren't supposed to apply"? While 4.9.10 does (unfortunately) exempt properly constrained CAs from this specific OCSP requirement, the rest of the BRs and the Mozilla Root Store Policy definitely apply with only a few exceptions around audits, disclosure, and CAA.
If the CA is technically constrained to only client certs (like ABB), then the BRs don't apply as neither the intermediate certificates nor the leaf certificates are intended for website security. This means the BR requirement does NOT require Sub CAs to respond something other than "good" for non-issued certificates.  In that case, imo, the CA SHOULD reply using the RFC recommendation ("good" for non-issued certificates).  That's exactly what happened with ABB. Believing they are technically constrained, ABB build their system in accordance with the RFC and is responding "good" for non-issued certs from this particular sub CA. They definitely are still required to meet the Mozilla policy of audits and disclosure.  CAA is not applicable as the CA is technically constrained to client certs.
(In reply to Jeremy from comment #8)
> If the CA is technically constrained to only client certs (like ABB), then
> the BRs don't apply as neither the intermediate certificates nor the leaf
> certificates are intended for website security.

My understanding is that the CA in question here is "ABB Issuing CA 6"[0] which appears to issue certificates with id-kp-serverAuth[1] that look like they may be used for TLS server authentication. The one "ABB Issuing CA 6" intermediate that is known to CT[2] has no EKU extension, which makes it valid for TLS server authentication. This intermediate is issued by "ABB Intermediate CA 3"[3] which has one unrevoked intermediate known to CT[4] that has id-kp-serverAuth in the EKU extension and has some name constraints (but no IPv6 constraints as noted).

So even ignoring the missing IPv6 constraint, this CA was never constrained to only client certificates and has issued certificates that are valid for TLS server authentication.

[0] https://crt.sh/?caid=1953
[1] https://crt.sh/?Identity=%25&iCAID=1953
[2] https://crt.sh/?id=6906660
[3] https://crt.sh/?caid=1952
[4] https://crt.sh/?id=7739892
Thanks Jonathan. Let me investigate and reply.
Flags: needinfo?(ben.wilson)
I think this discussion of other things that "should" apply to CAs, besides OCSP not responding with "good" for unissued certificates, has taken us off course here. We are working on the issues with these CAs.  Jeremy has sent them a reminder inquiry and we're waiting on their responses.  Meanwhile, I am going to add "Cartão de Cidadão 001" and "Cartão de Cidadão 002" to OneCRL.
Flags: needinfo?(ben.wilson)
Jeremy: Any update?

(In reply to Jeremy from comment #6)
> ABB is supposed to be technically constrained against TLS, which means that
> the BRs aren't supposed to apply. As Ben noted, we improperly created the
> issuing CA, missing the IPv6 restriction. We've brought this up a couple of
> times in the Mozilla community, and I thought the takeaway was that it was a
> non-issue on the IPv6 error.  However, as this keeps arising, we are going
> to fix the technical constraint in the next key ceremony to take them out of
> scope of the BRs.

Can you provide links to any of these discussions?

> SCEE only uses the issuing CAs for The CAs “Cartão de Cidadão 001” and
> “Cartão de Cidadão 002” are used only to issue certificates for Citizen.
> SCEE didn't seem to understand that the BRs applied equally to these
> certificates. I'm not sure how they didn't understand that the BRs applied
> to these certificates as every communication to the SubCA emphasizes the
> need to comply with the BRs. Because they are only issuing client
> certificates, we are adding them to OneCRL today. 

Can you provide links to the OneCRL request?

> I'm trying to find the info from Intesa

Do you have an update?
Flags: needinfo?(jeremy.rowley)
Ben will provide the Intesa update. Links to the discussion that referenced ABB:

https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/o1MX07iWDco

https://groups.google.com/forum/#!searchin/mozilla.dev.security.policy/ABB%7Csort:relevance/mozilla.dev.security.policy/tHUcqnWPt3o/U2U__7-UBQAJ

I thought it was discussed here, but we never posted it publicly:
https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/f5-URPoNarI


Looks like Ben hasn't added SCEE to OneCRL yet. Ben?
Flags: needinfo?(jeremy.rowley) → needinfo?(ben.wilson)
On 6-October-2017, Intesa Sanpaolo responded that they had opened a case with Microsoft on their OCSP responder issue. I'll ask them for an update on this issue.
Flags: needinfo?(ben.wilson)
We received the following report concerning ABB's OCSP responder issue from  Stanislaw Kaminski, ABB's PKI SME.  

On September 25th ABB was informed by Digicert that an issue with ABB's Issuing CA6 OCSP responder had been identified.

ABB implements a three-tier PKI infrastructure, with Intermediate CAs certificates implementing policy restrictions:
Root CA: Baltimore CyberTrust Root
Intermediate CA: ABB Intermediate CA3
Issuing CA: ABB Issuing CA6

As Jeremy has stated in this bug discussion, ABB believed that constraints on certificates provided by Digicert's predecessor were complete (as per BR 7.1.5) and that because of that, ABB's OCSP responder does not implement BR 4.9.10 response.

To remediate the IPv6 defect in name constraints, ABB is waiting on Digicert to issue a properly constrained Intermediate CA3 certificate to take advantage of the exception found in the Baseline Requirements, and the Issuing CA6 OCSP responder operating model will not be changed.
SCEE and Multicert have indicated that their systems are now patched.  

Intesa Sanpaolo has stated that Microsoft was unable to help them solve the problem with their Microsoft Windows Server 2012, but that Microsoft gave them a fix to solve the known problem for the Windows Server R2. Last report was that Intesa Sanpaolo was proceeding with the move to Windows Server R2.  I will check with them and report back on the status of this effort.
The new crt.sh report [1] identified two additional OCSP issues:
1. 'OCSP response contains bad number of responses' for the Baltimore CyberTrust root
2. A revoked certificate signing the TI Trust Technologies GLobal CA responses

Please report on these issues as well.

[1] https://crt.sh/ocsp-responders?trustedExclude=constrained%2Cexpired%2Conecrl&trustedBy=Mozilla&trustedFor=Server+Authentication&randomserial=Good
> 1. 'OCSP response contains bad number of responses' for the Baltimore
> CyberTrust root

I did some digging and have convinced myself that this one is a false positive. The error message appears to be coming from a Go library that doesn't like the fact that there are about responses for about 20 certificates in the signed response object. RFC 6960 4.2.2.3 indicates that this is permitted.
The new crt.sh report is listing a non-compliant Justica intermediate (https://crt.sh/?caID=606) that is run by SCEE and owned by DigiCert. Please investigate.
(In reply to Wayne Thayer [:wayne] from comment #20)
> > 1. 'OCSP response contains bad number of responses' for the Baltimore
> > CyberTrust root
> 
> I did some digging and have convinced myself that this one is a false
> positive. The error message appears to be coming from a Go library that
> doesn't like the fact that there are about responses for about 20
> certificates in the signed response object. RFC 6960 4.2.2.3 indicates that
> this is permitted.

Hi Wayne.  Yes, that error was a false positive.  I fixed [1] the relevant crt.sh code earlier today.  However, it turns out that that error was masking the fact that this particular responder [2] currently returns a "Good" response for a non-issued serial number [3].


[1] https://github.com/crtsh/ocsp_monitor/commit/4aba2032ae6dc1e5173a5db61fe93b2718b08c6f

[2] https://crt.sh/ocsp-responders?url=http%3A%2F%2Focsp.omniroot.com%2Fbaltimoreroot

[3] https://crt.sh/ocsp-response?caID=8&url=http%3A%2F%2Focsp.omniroot.com%2Fbaltimoreroot&request=randomserial
The CA Certificate referenced at the URL under [3] in Comment 22 doesn't have http://ocsp.omniroot.com/baltimoreroot as an AIA.  So where is crt.sh getting that URL?
(In reply to Ben Wilson from comment #23)
> The CA Certificate referenced at the URL under [3] in Comment 22 doesn't
> have http://ocsp.omniroot.com/baltimoreroot as an AIA.  So where is crt.sh
> getting that URL?

https://crt.sh/?id=12721653 is an example cert that refers to that OCSP URL.

It extracts the OCSP responder from the certificates issued by that CA.
Jeremy: it's unclear from the thread above if the ABB Issuing CA is going to be properly constrained, or if the intent is to fix this problem? In either case, what is the timeline for doing so?

Ben: the Balltimore Cybertrust Root is still violating the BRs by returning good responses for unknown certificates. When will this be fixed?
Flags: needinfo?(jeremy.rowley)
Flags: needinfo?(ben.wilson)
We continue working on both of these issues. They involve a transition from remote systems in Europe that need to be replaced. If you need a timeframe for when these will be fixed, I would say April 30th.
Flags: needinfo?(ben.wilson)
Whiteboard: [ca-compliance] → [ca-compliance] - Next Update - 30-April 2018
The intent is to constrain all sub CAs to they cannot issue TLS certs. Ben's been working with ABB on the issue and has clearer insights on timeframes. Ben - did you mean April 30th for both issues?
Flags: needinfo?(jeremy.rowley) → needinfo?(ben.wilson)
Changing QA contact per https://bugzilla.mozilla.org/show_bug.cgi?id=1438254
QA Contact: gerv → wthayer
ABB indicates that they have patched their OCSP responder.  

We are still in the process of moving the domain (omniroot.com / http://ocsp.omniroot.com/baltimoreroot) to DigiCert's infrastructure in order to respond to queries (for CA certificates issued by the Baltimore Cybertrust Root).  We should have an update by the end of the week.
Flags: needinfo?(ben.wilson)
We are still working on moving the OCSP responders over to DigiCert.  We're hopeful that we'll have tested our side of the OCSP responders by COB Wednesday 5-May-2018 and be able to move the domains to DigiCert  early next week.
Ben: please provide an update. This does not appear to yet be fixed.
Flags: needinfo?(ben.wilson)
We are still working on DNS issues. We are hoping to have this resolved by Thursday, 14-June-2018.
Flags: needinfo?(ben.wilson)
Migration of http://ocsp.omniroot.com/baltimoreroot to a new server and DNS were completed this past week.  The server now responds with an "unauthorized" error for unissued certificates.
Confirmed that this issue has been fixed.
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Product: NSS → CA Program
Whiteboard: [ca-compliance] - Next Update - 30-April 2018 → [ca-compliance] [ocsp-failure]
You need to log in before you can comment on or make changes to this bug.