Closed Bug 1925106 Opened 1 year ago Closed 10 months ago

DigiCert: Incorrect CP listed in CCADB

Categories

(CA Program :: CA Certificate Compliance, task)

Product:
CA Program
Component:
CA Certificate Compliance
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: tim.hollebeek, Assigned: dcbugzillaresponse)

Details

(Whiteboard: [ca-compliance] [disclosure-failure])

Attachments

(1 file)

1925106_Apple_SubCAs_CCADB.csv
10.97 KB, text/csv
Details

Steps to reproduce:

Incident Report

Summary

This is a preliminary report and DigiCert will provide updates. We will provide a full report by October 30th.

On October 15th, a Sectigo employee pointed out that some of DigiCert’s CCADB entries for Apple have a value in the CP field that points to Apple’s private CP, which appears to be incorrect. We are working with Apple and the root programs to find the correct resolution to the problem.

Assignee: nobody → tim.hollebeek
Status: UNCONFIRMED → ASSIGNED
Type: defect → task
Ever confirmed: true
Whiteboard: [ca-compliance] [disclosure-failure]

Can someone help me understand the issue with the applicable CP for the Apple issuing CAs? For example, let's take a look at the Apple Public Server ECC CA 11 - G1 issued by the COMODO ECC Certification Authority and the Apple Public EV Server ECC CA 1 - G1 issued by the DigiCert Global Root G3. My preliminary belief is that the issue raised in this bug is: which are the appropriate CP and CPS to reference in the CCADB for these kinds of CAs?

Typically, a CP is "a named set of rules that indicates the applicability of a certificate to a particular community and/or class of application with common security requirements". A CP says what needs to be done. Whereas a CPS is "[a] more detailed description of the practices followed by a CA in issuing and otherwise managing certificates." [RFC 3647] Usually, a CPS describes how a CA complies with the CP.

Here are some initial questions that I have (for anyone to answer):

  • Does Apple have sole physical control of the private key associated with each of these CAs?
  • Are there other scenarios not covered by the two example CAs mentioned above?
  • What do the related audit reports say about the CP or CPS under which these two example CAs are operated?
  • Which provisions of RFC 3647 (or in other authoritative resources) are the most helpful in understanding the appropriate CP or CPS to reference in the CCADB?

Hello Ben,

Apple does have sole physical control of the private key associated with each of these CAs (Question 1).

And thank you for taking the time to explicitly describe the situation. There are no other scenarios (Question 2).

The audit reports state “Apple’s Certification Practice Statement is consistent with the relevant governing Certificate Policies” (Question 3).

The audit report does not specifically identify the relevant governing Certificate Policies, but they would include the Baseline Requirements, EV Guidelines, S/MIME Baseline Requirements, and NCSSRs.

As far as guidance on the appropriate certificate policy, the most appropriate answer is probably actually the Baseline Requirements themselves, which clearly identify themselves as a certificate policy (Question 4).

For example, the TLS BRs state in section 1.2 that they are a Certificate Policy, and this is consistent with ballot 146 which converted the BRs into RFC 3647 format, work which was done by a body called the “CP Review Working Group”. CAs may layer additional policy requirements on top of the Baseline Requirements, but at a minimum, the Baseline Requirements are the relevant governing Certificate Policy, and perhaps referencing them directly is the best path forward here.

Flags: needinfo?(bwilson)

Ben, Tim,

Since the SubCA Certificates issued by Sectigo were also mentioned in comment 1 we wanted to chime in.

Does Apple have sole physical control of the private key associated with each of these CAs?

We can confirm that for the SubCA Certificates issued by Sectigo this is also correct.

What do the related audit reports say about the CP or CPS under which these two example CAs are operated?

While a CA is audited against the requirements and its CP/CPS, I don’t believe the actual CP/CPS is listed within the audit report in general.

Which provisions of RFC 3647 (or in other authoritative resources) are the most helpful in understanding the appropriate CP or CPS to reference in the CCADB?

None I would expect, since RFC 3647 has no knowledge about CCADB. However, the CCADB policy itself refers to the CP, CPS, or combined CP/CPS that the “certificate is issued and managed under”. It’s up to the CA to correctly determine which document that is.
In the specific context of this bug, note that in the applicable and disclosed CPS Apple explicitly states in Appendix A that DigiCert’s CP and Sectigo’s CP are the applicable Certificate Policies that that CPS is paired with. To disclose any other CP to CCADB is, at the very least, inconsistent with that statement.

As far as guidance on the appropriate certificate policy, the most appropriate answer is probably actually the Baseline Requirements themselves, which clearly identify themselves as a certificate policy (Question 4).

Tim, while we’d currently be hesitant on stating the BRs can be the CP for a CA (especially when taking into account the EVGs, which would then potentially be a second CP?), especially without further research, we are intrigued by the thought of this.

Though maybe this should be a discussion more suited for the CCADB or MDSP public lists, we’d be interested in seeing what other community members think of this concept. For such a discussion, I’d also pose the follow-up question (or thought experiment): If indeed specifying the BRs as a CA’s CP, that supposedly would in time be something many, if not all CAs, might follow. At such a point, does it become optional, or even a deprecated practice, for a CA to have its own CP?

Martijn,

We think that is a perfectly reasonable question. I remember being surprised at the Shanghai Face to Face, shortly after Google Trust Services and Amazon wrote CPSs that basically just restated the BRs, the Chrome Root Program claimed that that was not only acceptable, but more desirable. Since then, I think, the pendulum has swung back towards more reasonable views, but it is still worth thinking about and discussing from time to time. What should be in each of these documents, and why?

Now, of course, CPs are different from CPSs. The traditional model where this came from is that the Trust Consumer writes their policies in the form of a CP, and the Trust Provider writes their compliant practices into a CPS. I've always thought that CAs writing their own CP is a bit odd, though it can be useful for adding additional policies on top of existing policies. It would actually make sense if most CAs could move away from maintaining a custom CP, and just declare compliance with the standard WebPKI policies, and then also publish a CPS that explains how.

And I agree about this being better handled as a discussion elsewhere. I think it's best if incidents remain focused on the specific occurrence/situation, and general discussions about how to make the requirements better are handled in more appropriate places.

-Tim

Dear Martijn and Tim,

This is an interesting question of first impression for the CCADB. As Tim mentioned, a CP usually states the higher-level policy standards, and it is often set by the community-of-interest, while a CPS provides the CA’s implementation details that show conformity with the CP. This distinction raises an interesting question of whether a CA-specific CP offers meaningful value over existing CABF and Root Program requirements.

In the short term, it seems practical to preserve the status quo of allowing CAs discretion in populating the CCADB’s CP field with the document that best presents the high-level policy document that best represents the policy to which the CA is adhering. Note that the CCADB also currently allows the posting of multiple policy (“non-audit”) documents of similar types (CP, CPS, etc.), so a CA might populate the CCADB with multiple CPs, but I'm not saying that is a requirement.

Currently, even though the BRs are considered CPs in their own right, I am not sure that they are a perfect fit, and a CA-specific CP will likely assist the CA in gathering all applicable requirements into one document—this helps ensure that it will remain compliant, and that is why CAs should make sure to conduct a CP-CPS comparison whenever an edit occurs with either document (and that they perform self-assessment exercises at least annually to align their practices with all applicable requirements).

To move beyond the status quo and ensure that any CA’s approach has broader consensus, these matters should be discussed further on mdsp and among members of the CABF and the CCADB Steering Committee. I look forward to input from the CABF, and I will share any updates from the CCADB SC, and hopefully some consensus will emerge to guide CAs toward the appropriate practice.

Thanks,
Ben

Flags: needinfo?(bwilson)

(In reply to Martijn Katerbarg from comment #3)

Tim, while we’d currently be hesitant on stating the BRs can be the CP for a CA (especially when taking into account the EVGs, which would then potentially be a second CP?), especially without further research, we are intrigued by the thought of this.

Though maybe this should be a discussion more suited for the CCADB or MDSP public lists, we’d be interested in seeing what other community members think of this concept. For such a discussion, I’d also pose the follow-up question (or thought experiment): If indeed specifying the BRs as a CA’s CP, that supposedly would in time be something many, if not all CAs, might follow. At such a point, does it become optional, or even a deprecated practice, for a CA to have its own CP?

(In reply to Tim Hollebeek from comment #4)

Now, of course, CPs are different from CPSs. The traditional model where this came from is that the Trust Consumer writes their policies in the form of a CP, and the Trust Provider writes their compliant practices into a CPS. I've always thought that CAs writing their own CP is a bit odd, though it can be useful for adding additional policies on top of existing policies. It would actually make sense if most CAs could move away from maintaining a custom CP, and just declare compliance with the standard WebPKI policies, and then also publish a CPS that explains how.

It should be pointed out that a "Combined CP/CPS" is basically exactly what is described here. While the combined document itself technically fulfills both CP and CPS roles, it generally does so via a neat trick. As suggested by the BRs Section 2.2, the combined document incorporates the BRs by reference by stating

[Name of CA] conforms to the current version of the Baseline Requirements for the Issuance and Management of Publicly-Trusted TLS Server Certificates published at http://www.cabforum.org. In the event of any inconsistency between this document and those Requirements, those Requirements take precedence over this document.

and then the rest of the document takes the form of a CPS, stating how the CA complies with those requirements. I think the introduction of the combined CP/CPS as a concept was a step in the right direction, and moving towards a world where CAs can simply say "the BRs are our CP" or "the BRs and the EVGs are our CP" is another step in that same direction.

Thanks Aaron. I agree with you that moving to a combined CP/CPS is the correct approach given the fact the CA/Browser Forum and root stores are the entities creating the policies. This is why we moved to a combined CP/CPS. Having CAs simply copy the policies set by third parties and pretending that they are their own policy requirements seems like a waste of effort. Looking at the Sectigo CPS compared to their CP, for example, all of the information of substance is in the CPS and the CP appears to primarily be a version of the CPS with information removed. From reviewing the language, the main differences are items like this: “Specific Certificate usage will be defined in the CPS“ which add little value to anyone reading the document. One of the more interesting differences is the different locations where notices need to be sent under the CP vs the CPS. In the CP they are sent to Salford while the CPS notices are sent to Bradford. How would an ordinary user, one not well-versed in PKI policies, know which of these documents should apply and where to send notice? Another funny issue is where CAs define SHOULD, MUST, and other terms in their CPS but not the CP but then use those terms in the CP. I’d assume RFC2119 applies to both. Unfortunately, splitting the document has unintentionally obscured this fact. Because these kinds of issues abound when companies have a separate CP vs. CPS, the most user-friendly approach (as user-friendly as a CPS can be) is to have a practice statement from the CAs that dictates how the CA meets the policies set by the browsers.

This is a bit of a tangent for Bugzilla but given the discussion on CP/CPS I figure we should have a bit of a historical overview as it's something that seems to be fundamentally misunderstood.

RFC 3647 is where the terms come from, specifically Section 3.5

The CP and CPS address the same set of topics that are of interest to the relying party in terms of the degree to and purpose for which a public key certificate should be trusted. Their primary difference is in the focus of their provisions. A CP sets forth the requirements and standards imposed by the PKI with respect to the various topics. In other words, the purpose of the CP is to establish what participants must do. A CPS, by contrast, states how a CA and other participants in a given domain implement procedures and controls to meet the requirements stated in the CP. In other words, the purpose of the CPS is to disclose how the participants perform their functions and implement controls.

...

The main differences between CPs and CPSs can therefore be summarized as follows:
(a) A PKI uses a CP to establish requirements that state what participants within it must do. A single CA or organization can use a CPS to disclose how it meets the requirements of a CP or how it implements its practices and controls.

(b) A CP facilitates interoperation through cross-certification, unilateral certification, or other means. Therefore, it is intended to cover multiple CAs. By contrast, a CPS is a statement of a single CA or organization. Its purpose is not to facilitate interoperation (since doing so is the function of a CP).

(c) A CPS is generally more detailed than a CP and specifies how the CA meets the requirements specified in the one or more CPs under which it issues certificates.

I looked into this before as the Policy -> Statement parenting is what regulatory documents in other fields follow. The complete reversal in the world of CAs was rather bizarre to me. In any case at some point a CA flipped the terms, others followed, and the RFC was forgotten.

I suppose we could start joking about incidents for non-compliance with the RFC and the explicit fields on CCADB? The use of the terminology is a bit of a mess if I were to be honest though. The rare instance of a CA following the RFC order is noteworthy, but mainly people have combined them as that makes the most sense.

Incident Report

Summary

On October 15th, a Sectigo employee emailed DigiCert with a question on DigiCert’s CCADB entries for Apple controlled issuing CAs. These CCADB entries include a CP field that points to Apple’s private CP. This entry is incorrect as the Apple CP is for private certificates and not publicly trusted issuance. The CPS is correctly listed in CCADB and is the document audited under Webtrust, not the CP, which is why the error escaped notice.

Impact

The incorrect CP was first uploaded in 2018 when the Appe ICAs were created. The incorrect CP information was updated each year when updating the CPS information. The potential impact is that someone looking at CCADB would see the incorrect CP. There isn’t an impact on certificates as the CP is not audited annually by Webtrust nor is this information included in certificates.

Timeline

All times are UTC.

CCADB lacks historical information so some of these dates are our best guess based on key ceremony dates and operations information.

2018-12-12: DigiCert creates issuing CAs for use by Apple. DigiCert uploads these ICAs to CCADB and sets the CPS field to the Apple CPS. DigiCert uploads the Apple private CP as the CP.

2024-06-13: DigiCert updates the CPS with latest version. When updating the CPS, DigiCert uploads the same incorrect CP as was uploaded each year for the past 6 years.

2024-09-25: DigiCert consolidates and publishes its WebPKI CP and CPS as a single document after deciding that the true CP in the industry are the root store policies and the CA/Browser Forum Baseline Requirements.

2024-10-15 21:24: Sectigo notified DigiCert that the CP information in CCADB included the private Apple CP instead of a CP relevant to public trust. DigiCert acknowledges the incident report and begins investigating following its standard incident response policy.

2024-10-16 12:38: DigiCert is unsure what to list in the CP section of CCADB as we decided the relevant CP is the BRs. DigiCert reaches out to Mozilla for guidance on the correct value for this field. We believe listing the Baseline Requirements is the correct answer as 7.1.6.1 calls out the specific CA/Browser certificate policies CAs are required to use. Mozilla asked us to file a bug and will bring the question to the CCADB committee. We also confirm with our auditors that the CPS is the relevant document for WebTrust audit, not the CP.

We still haven’t updated the CCADB listing as the discussion has not reached a conclusion. We believe this should be the BRs or Mozilla CA policy.

Root Cause Analysis

The root cause analysis is that the CP information in CCADB is never used and, thus, was never inspected or reviewed on what the correct policy should be. Once DigiCert combined the policy and practice document, this field became obsolete. In 2018, CCADB was a newer program and lacked some of the clarifying guidance that has been added over the years.

Lessons Learned

The discussion on this bug is very good and has not yet settled on a definitive answer for what should be included in this CCADB field. We still believe the correct answer should be the Baseline Requirements as CABForum policy identifiers are required in the certificates instead of the CA’s CP OIDs. Section 2 only requires a CP or a CPS:

“The CA SHALL develop, implement, enforce, and annually update a Certificate Policy and/or Certification Practice Statement that describes in detail how the CA implements the latest version of these Requirements.”

What went well

This field is not used in practice, and the CP information was not included in the certificates. The CP listed in CCADB was not considered part of the Webtrust audit.

The CPS is correct.

What didn't go well

This information was uploaded to CCADB six years ago. No one reviewed the information since then to see if the CP provided was appropriate despite the many changes made to CCADB over the last 6 years.

Where we got lucky

This seems to be an industry-wide issue as no CAs are listing the BRs as their applicable CP in CCADB. We think, based on the language in the BRs, that the BR requirements should be the applicable CP.

Action Items

| Action Item | Kind | Due Date |

|Update the CCADB listing| Remediation | TBD. We will upload the information as soon as there is agreement on the correct value. |

Discussions continue.

Thank you for the report in Comment 9. A few questions come to mind.

In the Timeline Section:

CCADB lacks historical information so some of these dates are our best guess based on key ceremony dates and operations information.

  1. Can you elaborate on what specific historical information is lacking in the CCADB, which if available would have informed this Incident Report Timeline? It’s possible this could result in a CCADB Enhancement Request.

  2. Can you help us understand if and why the CCADB should be considered the canonical source of truth of this historical information when compared to DigiCert’s own records?

2024-09-25: DigiCert consolidates and publishes its WebPKI CP and CPS as a single document after deciding that the true CP in the industry are the root store policies and the CA/Browser Forum Baseline Requirements.

  1. At the time of this activity, was there any consideration for encouraging DigiCert CCADB entries for other controlled issuing CAs (e.g., Apple) to adopt a single CP/CPS? If so, what conclusion did you reach? If not, is there any specific reason why?

2024-10-16 12:38: DigiCert is unsure what to list in the CP section of CCADB as we decided the relevant CP is the BRs. DigiCert reaches out to Mozilla for guidance on the correct value for this field. We believe listing the Baseline Requirements is the correct answer as 7.1.6.1 calls out the specific CA/Browser certificate policies CAs are required to use. Mozilla asked us to file a bug and will bring the question to the CCADB committee. We also confirm with our auditors that the CPS is the relevant document for WebTrust audit, not the CP.

  1. As described, an indication that the CP is not a relevant component of the audit appears to conflict with the Illustrative Controls in Section 2.3 of the WebTrust Principles and Criteria for Certification Authorities Version 2.2.2. Was this accidentally misstated, and if not, can you help us better understand how this opinion aligns with those expectations described in the Principles and Criteria?

In the Root Cause Analysis Section:

The root cause analysis is that the CP information in CCADB is never used and, thus, was never inspected or reviewed on what the correct policy should be. Once DigiCert combined the policy and practice document, this field became obsolete.

  1. Can you elaborate on how you came to the conclusion that “the CP information in CCADB is never used”? Maybe this reference intended to only implicate the CP information for this specific ICA, but that’s not entirely clear and as stated in this report, DigiCert was updating this value annually for the past 6 years.

  2. Related to question #3 and specific to Apple at this point in time, have you considered suggesting they adopt a combined CP/CPS?

  1. Can you elaborate on what specific historical information is lacking in the CCADB, which if available would have informed this Incident Report Timeline? It’s possible this could result in a CCADB Enhancement Request.

DC: We can only see the latest upload and not historical upload information. A log file would be a great improvement. We can’t tell exactly when items were uploaded, updated, or changed.

  1. Can you help us understand if and why the CCADB should be considered the canonical source of truth of this historical information when compared to DigiCert’s own records?

DC: CCADB should be the canonical record for what is uploaded to CCADB and what has changed in CCADB reporting. We have our own records on changes to the CPS and CP. However, we are not sure exactly what was uploaded to CCADB and when because that’s not something we would ever track. We also do not track at which expectations changed with respect to which fields at the time.

  1. At the time of this activity, was there any consideration for encouraging DigiCert CCADB entries for other controlled issuing CAs (e.g., Apple) to adopt a single CP/CPS? If so, what conclusion did you reach? If not, is there any specific reason why?

DC: No, because these entities should not actually have a CP for public trust. They should only have a practice statement because the policies are the CAB Forum policies or, at least, the DigiCert certificate policy. The policies for external parties are all set by DigiCert. I would not have thought that any entity operating just an intermediate would have a certificate policy outside of the DigiCert certificate policy.

  1. As described, an indication that the CP is not a relevant component of the audit appears to conflict with the Illustrative Controls in Section 2.3 of the WebTrust Principles and Criteria for Certification Authorities Version 2.2.2. Was this accidentally misstated, and if not, can you help us better understand how this opinion aligns with those expectations described in the Principles and Criteria?

DC: This was not misstated. We have never been audited on the contents of our certificate policy, just our certificate practice statement. There is a check that we have a certificate policy, but the verification is only on practices.

  1. Can you elaborate on how you came to the conclusion that “the CP information in CCADB is never used”? Maybe this reference intended to only implicate the CP information for this specific ICA, but that’s not entirely clear and as stated in this report, DigiCert was updating this value annually for the past 6 years.

DC: The field was updated with the wrong value each year and no one noticed. I assume this means no one is using the field. Plus, we asked Ben what this field is used for.

  1. Related to question #3 and specific to Apple at this point in time, have you considered suggesting they adopt a combined CP/CPS?

DC: No, because they should not have a CP related to public trust, just a CPS. We consider the BRs to be the CP.

Any further questions?

(In reply to Martijn Katerbarg from comment #3)

In the specific context of this bug, note that in the applicable and disclosed CPS Apple explicitly states in Appendix A that DigiCert’s CP and Sectigo’s CP are the applicable Certificate Policies that that CPS is paired with. To disclose any other CP to CCADB is, at the very least, inconsistent with that statement.

We'd just like to note that Sectigo is aware of the Chrome Root Program's preference for Combined CP/CPS documents, and we're onboard with this as the future direction to head in.

(In reply to Tim Hollebeek from comment #0)

We will provide a full report by October 30th.

October 30th was the latest permitted date for providing the full report, not only because DigiCert promised to meet this deadline but also because “The full incident report must be posted within two weeks of the incident”. However, the full incident report wasn't posted until November 8th, which was 9 days after that deadline.

I'll withhold judgment on how to handle this particular infringement, but I would like to ask a couple of questions:

(In reply to Tim Hollebeek from comment #13)

Any further questions?

Actually, I also have a question for Chris Clements relating to this incident:

Flags: needinfo?(tim.hollebeek)
Flags: needinfo?(cclements)

After reviewing the timeline for this incident more closely, I believe that I may have been the person at DigiCert who populated the CCADB with the URL for the Apple CP for the Apple CAs back in December 2018. This creates a potential conflict of interest, so I'm recusing myself from further involvement other than to provide any additional factual information that might be requested. We'll work on identifying someone to take overall responsibility for this issue, and I’ll be happy to assist with that transition. Thanks.

In response to Comment 14:

Actually, I also have a question for Chris Clements relating to this incident:

  • Noting that “The Chrome Root Program considers CA policy documentation in the CCADB to be authoritative”, does BR 4.9.1.2(5) come into play here? Or is “the applicable Certificate Policy” in the CA's eyes not necessarily that which is indicated “in the CCADB to be authoritative”?

We don’t think BR 4.9.1.2(5) comes into play here.

We understand that more than just Chrome relies on CA policy documentation, but for those of us supporting the Chrome Root Program who regularly have to review these documents, having a single authoritative source to reference is a goal. This is especially true in time-sensitive situations like incident response, where we need to understand the CA Owner’s practices quickly. Instead of searching through various CA Repositories, which may not be in our native language, we have found that using the clearly-labeled field within the CCADB leads to better results.

Flags: needinfo?(cclements)

In response to Comment 12:

  1. Can you elaborate on what specific historical information is lacking in the CCADB, which if available would have informed this Incident Report Timeline? It’s possible this could result in a CCADB Enhancement Request.

DC: We can only see the latest upload and not historical upload information. A log file would be a great improvement. We can’t tell exactly when items were uploaded, updated, or changed.

Thank you for that suggestion. With the October 28 CCADB Non-Audit Document Enhancement CA Owners can view Non-Audit Document history to include the ID, notes, last modified date, last modified by, document details, and all of the certificates associated with the document for records that are synced from a CCADB Case. The CCADB Steering Committee can look for ways to improve this further.

  1. As described, an indication that the CP is not a relevant component of the audit appears to conflict with the Illustrative Controls in Section 2.3 of the WebTrust Principles and Criteria for Certification Authorities Version 2.2.2. Was this accidentally misstated, and if not, can you help us better understand how this opinion aligns with those expectations described in the Principles and Criteria?

DC: This was not misstated. We have never been audited on the contents of our certificate policy, just our certificate practice statement. There is a check that we have a certificate policy, but the verification is only on practices.

We’re still trying to understand how the CP was not a relevant component of previous audits. For instance, can you help us understand how the Illustrative Controls in Sections 2.2 or 2.3 of the WebTrust Principles and Criteria for Certification Authorities were previously evaluated?

The Explanatory Guidance in Section 2.2 states: “A CA may either have separate CP and CPS documents, or a combined CP/CPS. If the CA has a combined CP/CPS, or it implements the CP defined by another CA, then Criterion 2.2 is not applicable.”

In the case of this specific incident, Section 2.3 might be most applicable to other controlled issuing CAs, which states: “A CA may either have separate CP and CPS documents, or a combined CP/CPS. If the CA has a combined CP/CPS, then Criterion 2.3 is not applicable. However, if the CA implements the CP defined by another CA, then this criterion is relevant for ensuring the CA’s developed CPS is consistent with the provided CP.”

Again, we’re trying to best understand the applicability of the CP, as it relates to the audit.

October 30th was the latest permitted date for providing the full report, not only because DigiCert promised to meet this deadline but also because “The full incident report must be posted within two weeks of the incident”. However, the full incident report wasn't posted until November 8th, which was 9 days after that deadline.

Ben – what would you like us to do about this? We erroneously counted the two week date as starting on when we received a reply from CCADB on our questions about what should be in this field, not when Rob submitted the bug. We acknowledge the date should have been two weeks from when Rob filed the bug.

What procedures does DigiCert already have in place to monitor and ensure compliance with incident response deadlines, including deadlines for posting preliminary incident reports (“should be filed within 72 hours”), for posting full incident reports (“must be posted within two weeks of the incident”), and for answering questions (“promptly...and in no circumstances...more than one week”)?

DigiCert has a bug tracking process and a Bugzilla alerting system that tells us when we need to file the next update. The system can sometimes get confused as “updates” are made whenever someone adds themselves to a bug or a new response is filed. We also have a program manager that tracks the relevant dates and holds weekly meetings to discuss what needs to be filed and when.

Are there any potential improvements that could be made to these procedures that should be Action Items for this bug?

None that come to mind. The biggest change is that Jeremy used to track this stuff. His leaving lost some of the process knowledge that DigiCert formerly followed. Better succession planning is probably the key to avoiding these types of mistakes.

Ben – what would you like us to do about this? We erroneously counted the two week date as starting on when we received a reply from CCADB on our questions about what should be in this field, not when Rob submitted the bug. We acknowledge the date should have been two weeks from when Rob filed the bug.

Ben recused himself in Comment 15. Sharing the Chrome Root Program’s opinion, below.

The CCADB.org Incident Reporting Guidelines (IRGs) state:

There should be a single incident report for each distinct matter, and CA Owners should submit an additional, separate incident report when:
-Policy requires the revocation of one or more certificates by a certain deadline, such as those in BR section 4.9, but that deadline will not be or has not been met by the CA Owner.
-In the process of researching one incident, another incident with a distinct root cause and/or remediation is discovered.
-After an incident is marked resolved in Bugzilla, the incident reoccurs.

In our opinion, a failure to provide a full incident report by the designated time is a separate incident, which should be covered by a different incident report. While we appreciate the context offered in Comment 18, it appears there’s an opportunity for a deeper root cause analysis to explore more reliable methods of satisfying the expectations defined in the IRGs, given the existing controls did not produce the intended outcome.

For examples of secondary incident reports filed due to a failure of adhering to the IRGs, see Bug 1890901, Bug 1893546, and Bug 1904402.

If a separate report is not filed, we would hope for a deeper evaluation of root causes and more substantive commitment to action than provided in Comment 18 which is summarized as:

  • Sometimes, there are challenges with DigiCert’s Bugzilla alerting system.
  • Human-based controls to ensure timely responses exist, but did not work as expected.
  • Key staff departure resulted in a loss of some process knowledge.
  • There are opportunities for improvement, but no specificity is offered.

Filing the secondary report is Chrome’s preferred outcome.

Concerning resolution of this bug and the action items in Comment 9, it seems the correct CP value for the Apple ICA’s chaining to DigiCert Roots at this time would be the corresponding DigiCert CP (now a combined CP/CPS). This conclusion is interpreted from “Appendix A: Apple Subordinate CAs Hierarchy” of Apple’s in-force CPS, which includes “The Root CA Provider CP lists the name of the Root CA provider, which is correlated with the following Certificate Policies:”.

Somewhat unrelated, the report in Comment 9 would have been more helpful if it enumerated the specific set of CAs whose corresponding CCADB records where considered incorrect.

Feedback from the community is always welcome.

Report posted.

https://bugzilla.mozilla.org/show_bug.cgi?id=1937210

Flags: needinfo?(tim.hollebeek)

We will be updating bugs on Mondays over the holidays.

Happy holidays!

Onward to 2025!

Nothing new here.

Going forward, we will be posting updates as DigiCert, as suggested by a proposed update to CCADB policy. This will make it clear these are official DigiCert responses.

Assignee: tim.hollebeek → dcbugzillaresponse

As suggested in Comment 17 the CCADB SC has created 1942309 to track the deployment of a new CCADB feature that will preserve Non-Audit Document history associated with ICAs.

Also in Comment 17 we asked if DigiCert could help us understand how the CP was not a relevant component of previous audits. We do not believe this has been answered.

In Comment 19 we proposed an interpretation that the correct CP value for Apple ICA’s chaining to DigiCert Roots at this time would be the corresponding DigiCert CP. We’re still open to feedback on this interpretation, but do not yet see any CA Owner activity in the CCADB related to this incident.

The statement made in Comment 25 references a proposed update to the CCADB policy. For clarity, the proposed update is related to the CCADB incident reporting guidelines (“IRGs”), which was/is being discussed in this thread and is still subject to change pending the incorporation of final language. The CCADB policy and the CCADB IRGs are considered related, but separate.

Hi Chris - thank you for the questions!

Also in Comment 17 we asked if DigiCert could help us understand how the CP was not a relevant component of previous audits. We do not believe this has been answered.

Sorry - we thought this was answered already above. For clarity, the CP is not checked by auditors during the audit process. Instead, the auditors look at the CPS and the relevant industry CP - ie the Baseline Requirements for TLS. The auditors check based on the audit criteria whether we have a CP or not, do not evaluate the CP itself.

In Comment 19 we proposed an interpretation that the correct CP value for Apple ICA’s chaining to DigiCert Roots at this time would be the corresponding DigiCert CP. We’re still open to feedback on this interpretation, but do not yet see any CA Owner activity in the CCADB related to this incident.

We were waiting to see what the official ruling from CCABD was to avoid multiple updates. Sounds like listing DigiCert's CP instead of the BRs is the correct approach. We have updated the CCADB entries.

The statement made in Comment 25 references a proposed update to the CCADB policy. For clarity, the proposed update is related to the CCADB incident reporting guidelines (“IRGs”), which was/is being discussed in this thread and is still subject to change pending the incorporation of final language. The CCADB policy and the CCADB IRGs are considered related, but separate.

Thank you for the clarification.

Digicert completed all action items listed for this bug.

Action Items

Action Item Kind Due Date
Update the CP listing in CCADB with the correct information Remediation Completed

Incident Report Closure Summary

Incident Description:

The CP associated with Apple’s ICAs in CCADB was the Apple Private PKI CP, which was clearly wrong. The CP entry was originally made over 6 years ago.

Incident Root Cause(s):

The root cause is unknown as CCADB history is being implemented here: https://bugzilla.mozilla.org/show_bug.cgi?id=1942309. The root cause for why this was missed during the audits is that the audits do not cover the CP information. Instead, audits for TLS certificates focus on the information in the CPS and BRs and whether the CA complies with those requirements. There is a control objective listed for the CA having a CP, but we’ve never seen an audit that looks at the CP in addition to the CPS. Because this information was never used, the CP was never updated with the DigiCert CP instead of the Apple CP.

Remediation Description:

We updated the CP information in CCADB to list the DigiCert CP instead of the Apple CP.

Commitment Summary:

Our commitment here is to include the correct information going forward. We would love to automate this upload process if CCADB APIs are available in the future.

@Ben Wilson – As all action items are completed, will you please close this bug?

From Comment #28:

Our commitment here is to include the correct information going forward. We would love to automate this upload process if CCADB APIs are available in the future.

To clarify, APIs are currently available for the CCADB (documented here), which allow for the update of Audit and Non-Audit Documents on certificate records. We are under the impression that DigiCert is already using the API (or minimally, already has access to the API).

Can you expand on this commitment and your desire to automate the upload process?

Further, we think a more helpful action item or commitment from DigiCert would be in the spirit of driving the CA/B Forum towards a shared understanding of the combined CP/CPS, or that the TLS BRs could act as the CP, as stated in Comment #7, and originally referenced as “an industry-wide issue” in Comment #9.

Are there other action items or commitments that DigiCert would consider to improve upon this issue?

Chris, we can check with the appropriate folks what additional CCADB API features they feel would be helpful. As you note, DigiCert has a long history of working with CCADB on various API and automation improvements.

We think people are well aware we are always interested in improving the relevant requirements and their understanding, which is why we generally don't include that as an action item or commitment in these reports. We would be more than happy to work with others on clarifying that the relevant part when evaluating CA audits is their practices as stated in their CPS, which is the part they are audited against. We'll noodle a bit on what the concrete best path forward is here.

We'd actually like to thank the participants in this thread for having an intelligent and reasonable discussion on this issue, which raised a lot of interesting points, and shined some useful light on aspects of CA audits that are not as well understood in the industry as they need to be.

Chris, according to our CCADB automation folks, one of the CCADB improvements that would be useful is the ability to create Cases via the API. The lack of such functionality prevents us from fully automating document upload.

Note that Cases are not required for subCAs, but are required for roots, which is the reason for the confusion. We have automated upload for subCAs, but roots are still tricky.

(In reply to DigiCert from comment #28)

Commitment Summary:

Our commitment here is to include the correct information going forward.

I'd like to point out that to "include the correct information" initially is not the same thing as regularly checking that the already-included information is still correct.

Q1: Can DigiCert please clarify whether or not it is committing to regularly reviewing the accuracy of the already-included information in its CCADB records?

The Chrome Root Program Policy (CRPP) v1.1, which came into force on 2022-06-01, introduced a requirement that:

"Minimally, CA operators must ensure information stored in the CCADB is reviewed monthly and updated as needed." (emphasis mine)

This requirement remained in the CRPP, albeit with some tweaks to the language, until v1.4. CRPP v1.5 removed the explicit "must ensure information stored in the CCADB is reviewed" requirement, relying instead on equivalent language in the CCADB Policy, which says that:

"CA Owners have an overarching responsibility to keep the information in the CCADB about themselves, their operations and their certificates accurate, and to make updates in a timely fashion". (emphasis mine)

Furthermore, CRPP v1.5 reduced the upper bound on "timely fashion" from monthly to fortnightly:

"When a timeline is not defined for a requirement specified within the CCADB Policy, updates MUST be submitted to the CCADB within 14 calendar days of being completed". (emphasis mine)

(In reply to Tim Hollebeek from comment #9)

What didn't go well

This information was uploaded to CCADB six years ago. No one reviewed the information since then to see if the CP provided was appropriate despite the many changes made to CCADB over the last 6 years.

Q2: Can DigiCert provide details of its procedures, from when CRPP v1.1 came into force until now, that were designed to ensure compliance with the CRPP requirements I've quoted above, which have been in force for a significant portion of "the last 6 years"?

I'd like to establish whether or not "No one reviewed the information since then" was an isolated mishap.

Flags: needinfo?(dcbugzillaresponse)

(In reply to DigiCert from comment #28)

Remediation Description:

We updated the CP information in CCADB to list the DigiCert CP instead of the Apple CP.

https://crt.sh/mozilla-disclosures#disclosureincomplete is currently flagging CP disclosure problems for the following 6 Intermediate Certificate records issued by DigiCert and operated externally by Apple:

https://ccadb.my.site.com/s/account/0011J00001KtSgpQAF/apple-ist-ca-2-g1
https://ccadb.my.site.com/s/account/0011J00001KtSfIQAV/apple-public-server-ecc-ca-2-g1
https://ccadb.my.site.com/s/account/0011J00001KtSfmQAF/apple-public-server-rsa-ca-2-g1
https://ccadb.my.site.com/s/account/0011J00001KtSeKQAV/apple-public-server-rsa-ca-1-g1
https://ccadb.my.site.com/s/account/0011J00001KtSgzQAF/apple-ist-ca-8-g1
https://ccadb.my.site.com/s/account/0011J00001KtSdqQAF/apple-public-server-ecc-ca-1-g1

All 6 have the "CP Same as Parent" box ticked, but the parent Root Certificate records (listed below) no longer have a Standalone CP document listed in the Non-Audit Documents tab; instead, the parent Root Certificate records now only have a Combined CP/CPS, because all versions of DigiCert's Standalone CP have been marked as Superseded:

https://ccadb.my.site.com/s/account/001o000000HshE3AAJ/baltimore-cybertrust-root?tabset-3f9f8=74624
https://ccadb.my.site.com/s/account/001o000000HshEXAAZ/digicert-global-root-g2?tabset-3f9f8=74624
https://ccadb.my.site.com/s/account/001o000000HshEYAAZ/digicert-global-root-g3?tabset-3f9f8=74624

In comment 3, Martijn pointed out that the Apple CPS "states in Appendix A that DigiCert's CP and Sectigo's CP are the applicable Certificate Policies that that CPS is paired with".

Sectigo's view is that a Combined CP/CPS is unsuitable for use as a Standalone CP, because it is unreasonable to expect relying parties to be able to isolate the CP component of a Combined CP/CPS. Attempting to use a Combined CP/CPS as a Standalone CP in conjunction with an externally-operated Standalone CPS is particularly problematic. So for as long as Apple operates its own Standalone CPS that states that Sectigo's Standalone CP is applicable, Sectigo will continue to maintain its Standalone CP.

Q3: Does DigiCert agree that there is a CP disclosure problem with the CCADB records listed above that needs to be rectified; and if so, how does DigiCert intend to rectify it?

Responding to Rob Stradling in comment 33 and comment 34

Q1: Can DigiCert please clarify whether or not it is committing to regularly reviewing the accuracy of the already-included information in its CCADB records?

Yes. We are committed to regular reviews of accuracy.

Q2: Can DigiCert provide details of its procedures, from when CRPP v1.1 came into force until now, that were designed to ensure compliance with the CRPP requirements I've quoted above, which have been in force for a significant portion of "the last 6 years"?

We have relied on our automation to ensure all information is uploaded to CCADB within the specified timelines for several years and the scope of the automation has expanded to cover additional tasks. During the past year we added automation to check;

  • Validity of our Docs.
  • Check for CRLs
  • Check for revocation status
  • Upload CAs to CCADB to confirm accuracy

Since these CP/CPS issues have surfaced we have added checking for CP/CPS accuracy. We commit to correcting all errors or omissions within 14 days of detection and are continuing to deploy automated checks against CCADB.

We’ll also continue working with the CCADB community to identify problems with the system and interface and suggest solutions to make it easier for all CAs to review and correct the data, and to submit additional enhancement requests to help improve the process and notification and make the system more automatable.

Q3: Does DigiCert agree that there is a CP disclosure problem with the CCADB records listed above that needs to be rectified; and if so, how does DigiCert intend to rectify it?

DigiCert's CP/CPS is the CP and it is the controlling document for any certificate issued under the hierarchy of any of our root certificates, so no, we do not agree that there is a problem.

Flags: needinfo?(dcbugzillaresponse)

As there have been no additional comments on this bug, we will draft and post a closing summary.

Please could we keep this bug open for now. Yesterday I asked CCADB Support to take a look at the topics I mentioned and the opinions I expressed in comment 34, along with DigiCert's different opinion in comment 35. Depending on the outcome of the CCADB Support team's review, I expect there will either be more for DigiCert to do on this bug or it will be appropriate for me to change some of the logic behind crt.sh's CCADB disclosure pages.

(In reply to DigiCert from comment #35)

Responding to Rob Stradling in comment 33 and comment 34

Q1: Can DigiCert please clarify whether or not it is committing to regularly reviewing the accuracy of the already-included information in its CCADB records?

Yes. We are committed to regular reviews of accuracy.

Q2: Can DigiCert provide details of its procedures, from when CRPP v1.1 came into force until now, that were designed to ensure compliance with the CRPP requirements I've quoted above, which have been in force for a significant portion of "the last 6 years"?

We have relied on our automation to ensure all information is uploaded to CCADB within the specified timelines for several years

These two statements appear to conflict. The second statement seems to suggest that you are relying solely on your automation, and not actually verifying that the data you think you're uploading to CCADB is actually reflected in the CCADB records. Whereas the first statement seems to suggest that you are checking what's actually reflected in the CCADB records.

Or do you mean that your commitment to "regular reviews of accuracy" only extends to checking that you're feeding the correct information to your automation (and that you don't typically verify that the correct information is actually reflected in the CCADB records)?

Flags: needinfo?(dcbugzillaresponse)

In Comment 34, Sectigo offered a perspective, that in this instance of an intermediate certificate record referencing a CP/CPS as a standalone CP, it creates an unreasonable expectation for relying parties to be able to isolate the CP component of that combined document with the separately referenced CPS for that same certificate.

In Comment 35, DigiCert stated: “DigiCert's CP/CPS is the CP and it is the controlling document for any certificate issued under the hierarchy of any of our root certificates, so no, we do not agree that there is a problem.”

Question 1: How would DigiCert expect a relying party to know that DigiCert’s CP/CPS is the controlling document for any certificate issued under that hierarchy, while also accounting for the separate CPS of the issuing CA?
Question 2: If the CP/CPS is the controlling document, is there value in the separate CPS, or does that just create an opportunity for confusion?

For example, if we were to study the topic of CAA:

  • The DigiCert CP/CPS discloses, in Section 4.2.1.1 (“CAA Checking”), a set of domains interpreted as permission to issue (including www.digicert.com, digicert.com, digicert.ne.jp, etc.)
  • The Apple CPS discloses, in Section 3.2.2.8 (“CAA Records”, redirected from 4.2.1 (“Performing Identification and Authentication Functions”)), only pki.apple.com is interpreted as permission to issue.
  • The Microsoft CPS discloses, in Section 4.2.4 (“Verification of CAA Records”), only microsoft.com is interpreted as permission to issue.

Question 3: If the set of CAA domains disclosed in the DigiCert CP/CPS are considered as a maximum set of possible options (i.e., requirements, as one would expect of a CP), how would pki.apple.com or microsoft.com (when viewed as the in-use practice) be compatible?

Specific to the CCADB, we have proposed technically enforcing the following logic for evaluating policy disclosures in the system to the other members of the Steering Committee, and they are currently under review. If there is consensus, we will move forward with a CCADB Enhancement Request to add these technical controls to ensure alignment across CCADB disclosures.

If the parent certificate record discloses a CP:

A) It MUST disclose a CPS

B) It MUST NOT disclose a CP/CPS

C) All child certificate records MUST either:

  • (1) disclose a combined CP/CPS; OR
  • (2) disclose (a) a CP (either select "same as parent" OR a different CP) AND (b) a CPS (either select "same as parent" OR a different CPS).

If the parent certificate record discloses a CPS:

A) It MUST disclose a CP

B) It MUST NOT disclose a CP/CPS

C) All child certificate records MUST either:

  • (1) disclose a combined CP/CPS; OR
  • (2) disclose (a) a CPS (either select "same as parent" OR a different CPS) AND (b) a CP (either select "same as parent" OR a different CP).

If the parent certificate record discloses a combined CP/CPS:

A) It MUST NOT disclose a CP

B) It MUST NOT disclose a CPS

C) All child certificate records MUST either:

  • (1) disclose the same combined CP/CPS ("same as parent");
  • (2) disclose a different CP/CPS; OR
  • (3) disclose a CP and a CPS.

We may also add a technical enforcement to prevent a policy document URL disclosed as a specific type for one certificate record (e.g., https://my-ca.com/cp_1_4.pdf, type = “CP”) from being disclosed on a separate record using a different policy type (i.e., “CPS” or “CP/CPS”).

Feedback on the above processing logic is welcome from anyone.

(In reply to chrome-root-program from comment #38)

In Comment 34, Sectigo offered a perspective, that in this instance of an intermediate [...]

Feedback on the above processing logic is welcome from anyone.

I think this is a great topic that should probably get more visibility and discussion in the public@ccadb.org mailing list.

(In reply to chrome-root-program from comment #38)

Feedback on the above processing logic is welcome from anyone.

I strongly support this proposed CCADB enhancement. The logic looks correct to me.

If this is the direction the Chrome team wants to go, we're fine with that. We also agree with Dimitris that this would be a useful topic of conversation going forward, as there are probably many additional simplifications and improvements that can be made in this area, but we don't need to solve all the world's future CP/CPS problems here.

Since we’ve already added an item to our policy roadmap to separate out a TLS CP/CPS in response to Chrome’s recent policy update we will address these issues at the same time. We currently have the target date for completion March 31, but may need to adjust that due date to account for the additional work.

Flags: needinfo?(dcbugzillaresponse)

Rob,

We have been automating verification of CCADB information. You missed the rest of the post, which is not suprising - humans make mistakes and forget things. Here it is again:

“We have relied on our automation to ensure all information is uploaded to CCADB within the specified timelines for several years and the scope of the automation has expanded to cover additional tasks. During the past year we added automation to check;

Validity of our Docs.

Check for CRLs

Check for revocation status

Upload CAs to CCADB to confirm accuracy

Since these CP/CPS issues have surfaced we have added checking for CP/CPS accuracy. We commit to correcting all errors or omissions within 14 days of detection and are continuing to deploy automated checks against CCADB.”

If you’d like to suggest additional automated checks in place, please do so. We do not want to rely on human processes for anything related to compliance, especially when working with CCADB.

We are currently working on our stand-alone TLS CP for third party Sub CAs. This will be the controlling CP for external subordinate CAs once completed. We are targeting completion for March 31. We will continue to post updates advising the community of our progress.

Status update: Our policy team is in the process of mapping requirements from BR, EVG and the various trust store policies in preparation for the drafting of a new stand alone TLS CP.

I think splitting out the CP/CPS again is a bad idea, and I think the guidance from chrome root program is wrong and the BRs are the cert policy. I also think that splitting the CP/CPS is a bad idea as it potentially contradicts the chrome policy under 2.1 and 2.2:
2.1 - "Applicants MUST accurately describe the policies and practices of their CA(s) within a single CA policy document that is:
in the form of a combined CP/CPS."
2.2 - "accurately describe the policies and practices of their CA(s) within a Certificate Policy (CP) and corresponding Certification Practice Statement (CPS), or preferably, a single document combined as a CP/CPS."

I think the chrome post was confusing at it (unintentionally) encouraged splitting the CP/CPS again.

Here are my personal thoughts on this bug:

Question 1: How would DigiCert expect a relying party to know that DigiCert’s CP/CPS is the controlling document for any certificate issued under that hierarchy, while also accounting for the separate CPS of the issuing CA?

The CPS is the only relevant document for relying parties. If you review everyone's CP or CPS, you'll find the CPS contains both policy and practices. I think the answer DigiCert originally gave is correct - the BRs are the CP in all cases for TLS. Having a separate CP from the CPS just adds complexity without value.

Question 2: If the CP/CPS is the controlling document, is there value in the separate CPS, or does that just create an opportunity for confusion?

There should be a separate CPS for each external CA. For example, Apple should have a CPS because it controls its own issuance and may not use all the validation methods DigiCert adopted.

Question 3: If the set of CAA domains disclosed in the DigiCert CP/CPS are considered as a maximum set of possible options (i.e., requirements, as one would expect of a CP), how would pki.apple.com or microsoft.com (when viewed as the in-use practice) be compatible?

The apple CPS would control for the Apple CA and ICAs operated under the Apple root. CAA is worse if this isn't the case as DigiCert would need to include apple.com and microsoft.com in its own CPS which would widen the issuance of its own PKI. Bad idea IMO.

I think the intent of the chrome questions were to gather information, but they are leading to DigiCert splitting its CPS/CP which is going to be worse for relying parties and not align with chrome policy 1.6.

Flags: needinfo?(chrome-root-program)

We agree that it would be far better for the clarity of all parties in the WebPKI to consider the relevant CA/B work product(s), with the addition of relevant requirements from the various trust store operator policies, as the CP and for CAs to then maintain only a CPS stating their specific practices for adhering to the policy. At this time we continue to map requirements for a stand alone CP, but we would like to see Chrome reconsider this matter. Awaiting clarification from the Chrome team.

Thanks for the thoughts in Comment 45, Jeremy. Several of our comments in this incident report suggest that community feedback is welcome, yours included!

I think splitting out the CP/CPS again is a bad idea, and I think the guidance from chrome root program is wrong and the BRs are the cert policy.

Can you help us understand what specific guidance the Chrome Root Program offered, and why you consider it wrong?

Question 1: How would DigiCert expect a relying party to know that DigiCert’s CP/CPS is the controlling document for any certificate issued under that hierarchy, while also accounting for the separate CPS of the issuing CA?
The CPS is the only relevant document for relying parties. If you review everyone's CP or CPS, you'll find the CPS contains both policy and practices. I think the answer DigiCert originally gave is correct - the BRs are the CP in all cases for TLS. Having a separate CP from the CPS just adds complexity without value.

We try to avoid generalities and we don’t think the statement: “If you review everyone’s CP or CPS, you’ll find the CPS contains both policy and practices.” is absolutely correct. As we’ve seen instances where that is not true. For example, here we see a CP with practices that are absent from the CPS. Also, here we see practice detailed in the CP (e.g., Section 1.5.2) that is not included in the CPS. This is not exhaustive and only illustrative.

We agree that having separate policy documents adds complexity. This was the motivation for our recent policy update that requires Applicants to maintain a combined CP/CPS. However, the question we asked was how would DigiCert expect a relying party to know that “DigiCert’s CP/CPS is the controlling document for any certificate issued under that hierarchy” (as stated in Comment 35) when there is more than one CPS (or minimally a different CPS) applicable to the issuing CA? Your response to question 2 seems to be in agreement that the separate CPS of the external CA is most appropriate because the external CA controls its own certificate issuance and may not use certain elements of the DigiCert CPS. That said, it’s unclear what would have stopped DigiCert from describing those practices belonging to external CAs in its combined CP/CPS such that it could be relied upon by both DigiCert and those external organizations.

Combined with Comment 3 that highlights: “In the specific context of this bug, note that in the applicable and disclosed CPS Apple explicitly states in Appendix A that DigiCert’s CP and Sectigo’s CP are the applicable Certificate Policies that that CPS is paired with.” its not clear how all of the specific policy documents referenced in this report relate to each other, and further how a relying party is to distinguish what's applicable.

I think the intent of the chrome questions were to gather information, but they are leading to DigiCert splitting its CPS/CP which is going to be worse for relying parties and not align with chrome policy 1.6.

Indeed, our intent was to better understand this situation and the consideration for external CAs. Community members suggested the topic of technically enforcing policy document logic in the CCADB become a post in public@ccadb.org for further discussion. We posted, as suggested, and did not receive any feedback.

It might also be worthwhile to revisit the comments made in relation to audits. Comment 11 highlights an indication that the CP is not a relevant component of the audit (as presented by DigiCert in Comment 9) which appears to conflict with the Illustrative Controls in Section 2.3 of the WebTrust Principles and Criteria for Certification Authorities Version 2.2.2. Similarly, ETSI EN 319 411-1 - V1.4.1 includes several CP related requirements, which we would interpret to be a relevant component of audits.

Comment 30 later engages with this point and states: “We think people are well aware we are always interested in improving the relevant requirements and their understanding, which is why we generally don't include that as an action item or commitment in these reports. We would be more than happy to work with others on clarifying that the relevant part when evaluating CA audits is their practices as stated in their CPS, which is the part they are audited against. We'll noodle a bit on what the concrete best path forward is here.”

We’ve seen no “concrete best path forward” presented, but given the WebTrust criteria and ETSI standards, its still not clear how one would expect an auditor to evaluate controls such as the following if a CP was not a relevant component of an audit.

WebTrust Principles and Criteria for Certification Authorities Version 2.2.2:
- 2.3.1 The PA is responsible for ensuring that the CA’s control processes, as stated in a Certification Practice Statement (CPS) or equivalent, fully comply with the requirements of the CP.
- 2.3.2 The CA addresses the requirements of the CP when developing its CPS.
- 2.3.3 The CA assesses the impact of proposed CPS changes to ensure that they are consistent with the CP.
- 2.3.4 A defined review process exists to ensure that Certificate Policy(s) are supported by the CA’s CPS.

ETSI ETSI EN 319 411-1 - V1.4.1:
- OVR-5.1-03: The TSP's CP should specify the requirements for the use of certificate profiles.
- OVR-5.3-01: If any changes are made to a CP as described in clause 4.2.2 which affects the applicability then the policy identifier should be changed.
- OVR-6.3.5-01: The subscriber's obligations (see clause 6.3.4) shall include:
- [...]
- d) [CONDITIONAL] if the subscriber or subject generates the subject's keys:
- i) an obligation or recommendation to generate the subject keys using an algorithm as specified in ETSI TS 119 312 [i.10] for the uses of the certified key as identified in the CP; and
- ii) an obligation or recommendation to use key length and algorithm as specified in ETSI TS 119 312 [i.10] for the uses of the certified key as identified in the CP during the validity time of the certificate;

This is not an exhaustive list, but it highlights some of the potential challenges with the statement that a CP is not a relevant part of the audit.

Flags: needinfo?(chrome-root-program)

Thanks for the thoughts in Comment 45, Jeremy. Several of our comments in this incident report suggest that community feedback is welcome, yours included!

Thanks! For transparency, I rejoined DigiCert just barely in a new role. All posts on bugzilla are part of the community as my responsibilities do not include responding to incidents. Wanted to add that as it is a change in my position.

Can you help us understand what specific guidance the Chrome Root Program offered, and why you consider it wrong?

To be clear, I don't think there was any specific guidance provided by Chrome that said, "Don't combine your CP/CPS". Instead, I think the questions asked combined with Rob's comment and the technical processing proposal led Digicert to the incorrect answer on the next steps. Combining the CP/CPS was the correct move for DigiCert as it simplified compliance and having a separate CP didn't add value. Sure, there is a potential for customer confusion because anyone reading the CP/CPS might think it applies wholesale to the Apple-issued certificates. However, a separate CP vs. CPS doesn't reduce this risk as there's no way to tie the CP to the certificates themselves. Entities looking at the certificate will see the CPS listed. Entities looking in CCADB will see the CCADB listing. Therefore, I don't think there is any more or less confusion with a combined CP/CPS compared separate documents.

An example of a specific questions I think led DigiCert to the wrong answer are:

Question 1: How would DigiCert expect a relying party to know that DigiCert’s CP/CPS is the controlling document for any certificate issued under that hierarchy, while also accounting for the separate CPS of the issuing CA?

I think a better question is how a separate CP would make it clearer than a combined CP/CPS. I don't think it does.

We try to avoid generalities and we don’t think the statement: “If you review everyone’s CP or CPS, you’ll find the CPS contains both policy and practices.” is absolutely correct. As we’ve seen instances where that is not true. For example, here we see a CP with practices that are absent from the CPS. Also, here we see practice detailed in the CP (e.g., Section 1.5.2) that is not included in the CPS. This is not exhaustive and only illustrative.

I wasn't clear what I meant. The definition between a policy and a practice is unclear. For example, in the cited CP from Globalsign, Globalsign describes trusted roles and their responsibilities. Is that a policy? Or is that their practice? Similarly, the d-trust statement talks about how their policy identifiers are assigned in the CPS. Should that be a policy? I don't find the distinction that helpful as the CPS isn't really a SOP and the CP often has elements of practices included. I also think splitting the policy/practices out isn't particularly helpful. The d-trust example cited references their CP, which means you have to switch back and forth between documents continuously to see what is going on. Is that helpful?

Indeed, our intent was to better understand this situation and the consideration for external CAs. Community members suggested the topic of technically enforcing policy document logic in the CCADB become a post in public@ccadb.org for further discussion. We posted, as suggested, and did not receive any feedback.

I figured this was the case, and I wish you'd received feedback! I think the Chrome policy is the correct approach (of combining the CP/CPS).

We’ve seen no “concrete best path forward” presented, but given the WebTrust criteria and ETSI standards, its still not clear how one would expect an auditor to evaluate controls such as the following if a CP was not a relevant component of an audit.

I think this is a different question than whether a combined CP/CPS is a recommended path forward.

The reason I commented now is because I think it would be a mistake to separate out the CP/CPS documents. My recommendation is that DigiCert does not do that and reconsiders how it addresses the root cause (primarily because I do not think separating the CP/CPS fixes the root cause of the incident).

Finally, Sectigo recently moved to a combined CP/CPS: https://www.sectigo.com/uploads/files/Sectigo_TLS_CPS_v6_1_0.pdf. This is only relevant because Sectigo is on this bug and has read the comments. This means that Sectigo did not read Chrome's questions and come to the same answer as DigiCert.

In response to Comment 48, immediately above:

The reason I commented now is because I think it would be a mistake to separate out the CP/CPS documents. My recommendation is that DigiCert does not do that and reconsiders how it addresses the root cause (primarily because I do not think separating the CP/CPS fixes the root cause of the incident).

In Comment 38, we suggested the following technical controls be added to the CCADB to ensure alignment across CCADB disclosures:

If the parent certificate record discloses a combined CP/CPS:

A) It MUST NOT disclose a CP
B) It MUST NOT disclose a CPS
C) All child certificate records MUST either:
- (1) disclose the same combined CP/CPS ("same as parent");
- (2) disclose a different CP/CPS; OR
- (3) disclose a CP and a CPS.

We interpret DigiCert's Comment 41 to understand those options without objecting to them, albeit misappropriating the CCADB Steering Committee's conclusions for Chrome's own. It remains unclear why DigiCert opted to split its combined CP/CPS in the presence of those other options, but that’s DigiCert’s choice to make. Regardless, at no point did the Chrome Root Program suggest DigiCert take that specific course of action.

It seems that Apple, at one point (and possibly still today) maintained its own “public trust” CP (albeit, this version now appears a few days stale). Rather than DigiCert splitting its CP/CPS, DigiCert could have aligned with option C3 by disclosing Apple’s CP and CPS to the CCADB. Option C2 also could have been considered.

Can you help us understand:

  • (i) why were approaches C2 and C3 not considered viable options?
  • (ii) why was DigiCert splitting its combined CP/CPS considered preferable when compared to those other options?

Finally, Sectigo recently moved to a combined CP/CPS: https://www.sectigo.com/uploads/files/Sectigo_TLS_CPS_v6_1_0.pdf. This is only relevant because Sectigo is on this bug and has read the comments. This means that Sectigo did not read Chrome's questions and come to the same answer as DigiCert.

We can't speak on behalf of Sectigo. However, we understand Rob's comment in 40 to express agreement with the proposed CCADB technical controls introduced in Comment 38 to ensure alignment across CCADB disclosures. We also understand Comment 14 to express Sectigo's support for using a combined CP/CPS.

We interpret the quoted statement above to imply that Sectigo has violated the Chrome Root Program Policy concerning CCADB disclosures. While that’s certainly a possibility, it’s also possible that Sectigo, given their understanding of the set of available options presented in Comment 38, intends to implement C2 or C3. So long as changes are disclosed to the CCADB as described by the Chrome Root Program Policy - the disclosures are consistent with our expectations.

If you or anyone else feel that interpretation is incorrect, we’d appreciate understanding why.

Finally, in response to Comment 48 mentioning the questions the Chrome Root Program asked (among other things) may have led DigiCert to an incorrect answer on next steps. We feel it’s important to clarify that the questions we asked were directly related to statements DigiCert made, which seemed to contradict what we saw in the ecosystem - such as the DigiCert CP/CPS being the controlling document for any certificate issued under the hierarchy of any of DigiCert’s root certificates and the CP not being a relevant component of audits.

Flags: needinfo?(dcbugzillaresponse)

Thank you Chrome team for the response! I really appreciate how quickly you addressed this. And, for the record, I definitely agree that the Chrome team did not lead DIgiCert to an incorrect answer. I think its the stress of the bugs combined with mis-reading the response. I do not think Chrome said something incorrect. Rather, I think DigiCert arrived at the wrong conclusion based on Chrome's post.

In Comment 38, we suggested the following technical controls be added to the CCADB to ensure alignment across CCADB disclosures:
If the parent certificate record discloses a combined CP/CPS:
A) It MUST NOT disclose a CP
B) It MUST NOT disclose a CPS
C) All child certificate records MUST either...

We interpret DigiCert's Comment 41 to understand those options without objecting to them, albeit misappropriating the CCADB Steering Committee's conclusions for Chrome's own. It remains unclear why DigiCert opted to split its combined CP/CPS in the presence of those other options, but that’s DigiCert’s choice to make. Regardless, at no point did the Chrome Root Program suggest DigiCert take that specific course of action.

This is the correct approach. I do not know why DigiCert did not adopt it.

We can't speak on behalf of Sectigo. However, we understand Rob's comment in 40 to express agreement with the proposed CCADB technical controls introduced in Comment 38 to ensure alignment across CCADB disclosures. We also understand Comment 14 to express Sectigo's support for using a combined CP/CPS.

I read it as the opposite even while not at DigiCert - that Sectigo thought a combined CP/CPS was a bad idea considering that the Apple CPS would need to be governed by the DigiCert CP. I reread the thread when I saw that Sectigo combined their CP/CPS and realized there was an issue.

We interpret the quoted statement above to imply that Sectigo has violated the Chrome Root Program Policy concerning CCADB disclosures. While that’s certainly a possibility, it’s also possible that Sectigo, given their understanding of the set of available options presented in Comment 38, intends to implement C2 or C3. So long as changes are disclosed to the CCADB as described by the Chrome Root Program Policy - the disclosures are consistent with our expectations.

I do not think Sectigo violated the Chrome root program and did not mean to imply that if I did. I think they arrived at the right answer - that a combined CP/CPS with C2 or C3 is the correct approach.

If you or anyone else feel that interpretation is incorrect, we’d appreciate understanding why.

I think its correct.

Finally, in response to Comment 48 mentioning the questions the Chrome Root Program asked (among other things) may have led DigiCert to an incorrect answer on next steps. We feel it’s important to clarify that the questions we asked were directly related to statements DigiCert made, which seemed to contradict what we saw in the ecosystem - such as the DigiCert CP/CPS being the controlling document for any certificate issued under the hierarchy of any of DigiCert’s root certificates and the CP not being a relevant component of audits.

Makes sense. Thank you! I can't speak to the relevancy of CP on audits and my comment was only about DigiCert's decision to separate the CP/CPS into two separate documents. I think that's a mistake and not the intent of Chrome's response. I appreciate the response and think its very clear there is a path to use a combined CP/CPS with an external CA.

Hi Jeremy,

You are correct that we have (recently) published trust purpose specific Combined CP/CPSes. That is a path we started out on almost a year ago. When the Chrome Root Program Policy v1.6 draft was circulated, we put more priority into this effort, which has now led to the publication of these documents. Our path forward since the beginning of this effort has been to also maintain a standalone CP that can be and is being utilized by some externally operated CAs, at least for a transitional period until they too are able to migrate to a Combined CP/CPS.

This approach is also in line with the proposed logic updates to CCADB provided by the Chrome Root Program in comment 38, which I take from your comment 50 you also believe is the correct path.

You are correct that we have (recently) published trust purpose specific Combined CP/CPSes. That is a path we started out on almost a year ago. When the Chrome Root Program Policy v1.6 draft was circulated, we put more priority into this effort, which has now led to the publication of these documents. Our path forward since the beginning of this effort has been to also maintain a standalone CP that can be and is being utilized by some externally operated CAs, at least for a transitional period until they too are able to migrate to a Combined CP/CPS.

Awesome! Thank you Martijin! The transition period makes sense. I noticed 1.4.0 of the CP is still on your active legal repo. Just a suggestion - does it make sense to clarify that this applies only to external CAs and that your combined CP/CPS applies to all Sectigo certs? Otherwise it might be confusing about which is the actual CP (do they both apply in this case)?

This approach is also in line with the proposed logic updates to CCADB provided by the Chrome Root Program in comment 38, which I take from your comment 50 you also believe is the correct path.

100%.

Just a suggestion - does it make sense to clarify that this applies only to external CAs and that your combined CP/CPS applies to all Sectigo certs? Otherwise it might be confusing about which is the actual CP (do they both apply in this case)?

Thank you. We agree a clarification could be added on the landing page and/or the document itself that this affects only specific CAs.

We would like to thank the Chrome team for their clarifications. Based on the conversation above, we did mis-interpret Chrome's post as guidance rather than questions. We plan to keep our combined CP/CPS and change our CCADB records to reflect our CP/CPS for our roots and the subCAs we operate (option #3-C1 from comment 38), and specify the Apple CP/CPS for the subCAs that are signed by our roots, but operated by Apple (option #3-C2 from comment 38).

Regarding the Chrome team's statement in comment 47, "We’ve seen no “concrete best path forward” presented, but given the WebTrust criteria and ETSI standards, it’s still not clear how one would expect an auditor to evaluate controls such as the following if a CP was not a relevant component of an audit," based on our experience the auditors treat the Baseline Requirements as the CP and map the requirements in the BRs to the CPS, ignoring our CP other than to note that it exists.

Flags: needinfo?(dcbugzillaresponse)

We currently list the DigiCert CP/CPS in the CP field and the Apple CPS in the CPS field. This is the most accurate depiction of the relationship right now and believe it is correct. When our Sub CAs finish migrating to the combined CP/CPS, we will update the CCADB listing to reflect the combined document. Ben - is this satisfactory for now? If so, and because the CCADB listings are as accurate as possible, we will draft and post a closing summary next week. Thank you for the discussion and clarification on how CCADB works.

Flags: needinfo?(bwilson)

Thanks. Please file a Closing Summary.

Flags: needinfo?(bwilson)

It has been alleged in bug 1957499 that there is a question that remains unanswered in comment 49. Between our stated agreement with the general discussion which directly follows this comment, and specifically with the points made by Jeremy, along with our own posts in response, we are not aware of a question in this post that has not been answered but it’s certainly possible that we’ve missed something or perhaps not been as clear as may be necessary. Please let us know if this is the case.

(In response to comment 55 and comment 57)

We currently list the DigiCert CP/CPS in the CP field and the Apple CPS in the CPS field. This is the most accurate depiction of the relationship right now and believe it is correct.

In comment 38 Chrome asked three labeled questions, none of which DigiCert has answered after more than a month. In comment 45 Jeremy replied to them, but he has made it clear in that comment and elsewhere that he does not speak for DigiCert in his contributions on CA incident bugs. The subject matter of all three questions was “the DigiCert CP/CPS in the CP field,” so these questions remain relevant. Even if they had become irrelevant, answers within 7 days are still expected.

In comment 49 Chrome asked,

Can you help us understand:

  • (i) why were approaches C2 and C3 not considered viable options?
  • (ii) why was DigiCert splitting its combined CP/CPS considered preferable when compared to those other options?

We don’t believe DigiCert has answered these questions either.

Flags: needinfo?(dcbugzillaresponse)

In comment 38 Chrome asked three labeled questions, none of which DigiCert has answered after more than a month. In comment 45 Jeremy replied to them, but he has made it clear in that comment and elsewhere that he does not speak for DigiCert in his contributions on CA incident bugs.

This is correct. I do not speak for DigiCert at CABF nor on bugs. I think DigiCert did answer the questions (or believe they did) in comment 54:

We would like to thank the Chrome team for their clarifications. Based on the conversation above, we did mis-interpret Chrome's post as guidance rather than questions. We plan to keep our combined CP/CPS and change our CCADB records to reflect our CP/CPS for our roots and the subCAs we operate (option #3-C1 from comment 38), and specify the Apple CP/CPS for the subCAs that are signed by our roots, but operated by Apple (option #3-C2 from comment 38).

The answer seems pretty obvious that the answer was that DigiCert misread the question. I'll let DigiCert add more specific details, but this seems to be overly strict on what constitutes an answer to a question. Is there a basis in the policy on how CAs must answer questions - other than the 7 days?

(In reply to Jeremy from comment #59)

The answer seems pretty obvious that the answer was that DigiCert misread the question. I'll let DigiCert add more specific details, but this seems to be overly strict on what constitutes an answer to a question.

Yes, it’s obvious DigiCert stated it misinterpreted what Chrome wrote. It’s equally obvious DigiCert didn’t answer the questions. DigiCert stated in comment 54 that it had mis-interpreted the three questions in comment 38 as guidance rather than questions. That means that as of March 18, 2025 DigiCert knew that these were questions and that these questions already had gone unanswered for a month. Now another two weeks have passed, during which time DigiCert has stated that it was unaware of any open questions and asked to close this bug, while not having answered these or the other questions.

I don’t feel it’s overly strict to expect a supposed answer to a question actually to answer the question, rather than just explain why the question has gone unanswered. Thank you for calling my attention to the fact that I should have drilled down on this.

Question 1: How exactly is it that you were unaware of these unanswered questions? What is your method for tracking Bugzilla questions and comments that require responses? What is your method for bug review before asking to close an incident bug?

I ask these questions because the evidence indicates that DigiCert needs to look at and improve its process in this regard. And once again, can DigiCert please look at its own bugs for unanswered questions before asking to close them?

Is there a basis in the policy on how CAs must answer questions - other than the 7 days?

I believe the guidance is pretty general, and if CAs sincerely engage in the process, most of the time it’s fine. Unfortunately, though, we regularly see CA behavior on bugs that is counterproductive to the incident reporting process and its goal of identifying and implementing improvements to prevent repetition of errors – and to help all CAs up their game before similar problems occur for them.

While there may not be a hard-and-fast rule against this behavior, I dare say it is a bad look for a public CA. There are a lot of ways that poor use of Bugzilla – either intentionally or otherwise – can obscure and impede discussion of important matters. For any CA who wants to avoid this bad look and be truthfully seen to facilitate and welcome open discussion, I have some recommendations. These are all based on real examples, both good and bad, that I have observed on Bugzilla in the past year.

  • Signpost your comments. If you are responding to a specific question or comment from this or another thread, tell us which one. Bugzilla even has functionality for this if you use the “Respond” button at the top of any given post.
  • Use Markdown. Markdown is quite easy to use and can go a long way in making posts easier to understand. While most comments use Markdown correctly (or are short enough not to really need it), sometimes we see long posts with no Markdown (even when it’s prescribed, as in an Incident Report) or posts with Markdown that actually makes reading them more difficult. Bugzilla includes a Preview mode, which is incredibly helpful in making sure your Markdown is showing as you expect.
  • Answer questions and respond to comments directly, fully, and in good faith.
  • Don’t allow your negative emotions to take the driver’s seat. Avoid churlishness and name calling.
  • Be on time. Meet the requirements for responding to comments and questions, the weekly posting cadence, deadlines for initial and full Incident Reports, and Next Update dates.
  • Follow clear procedures where they exist for such activities as bug-closing and incident reporting. As a procedure becomes updated, follow the current version.
  • Follow more than just your own bugs. CAs should track all open bugs on Bugzilla, which includes reading all posts and examining errors and lessons for applicability to their own operations.
  • Admit your errors. We understand that mistakes occur, and a good faith error is rarely a serious problem for a CA. Where it becomes a problem is if you don’t accept and correct your errors or if you repeat them.
  • Say yes to continual improvement. Full compliance and use of best practices is a journey for every CA. Rather than resisting feedback or denying errors that factually occurred, a good CA focuses on learning from mistakes and improving.

Question 1: How exactly is it that you were unaware of these unanswered questions?

Answer 1: As noted by Jeremy in Comment #59, we considered the Chrome questions to have been guidance that informed and was incorporated into our answer in an earlier response at Comment #54. To avoid confusion in future, unless it interferes with readability, we will explicitly call out questions and their answers.

For the sake of such clarity, please see below for more detail of the Chrome questions detailed in Comment #38.

Chrome 1: How would DigiCert expect a relying party to know that DigiCert’s CP/CPS is the controlling document for any certificate issued under that hierarchy, while also accounting for the separate CPS of the issuing CA?

As noted in our earlier responses, CCADB will show our CP/CPS for our roots and the subCAs we operate (option 3-C1 from Comment #38), and show the Apple CP/CPS for the subCAs that are signed by our roots, but operated by Apple (option 3-C2 from Comment #38).

Chrome 2: If the CP/CPS is the controlling document, is there value in the separate CPS, or does that just create an opportunity for confusion?

Does not apply. We have a CP/CPS, and do not have a separate CPS. Apple and Microsoft are in the process of updating their respective CP/CPS documents.

Chrome 3: If the set of CAA domains disclosed in the DigiCert CP/CPS are considered as a maximum set of possible options (i.e., requirements, as one would expect of a CP), how would pki.apple.com or microsoft.com (when viewed as the in-use practice) be compatible?

Does not apply. The CAA domains of the applicable CP/CPS apply. In other words, the DigiCert, Apple, and Microsoft CP/CPS documents list their respective CAA domains.

Question 2: What is your method for tracking Bugzilla questions and comments that require responses?

Answer 2: DigiCert has a process for tracking Bugzilla questions and actions. To avoid duplication, please see our Incident Report, once posted on Bug 1957499 on this same Subject and if needed, ask clarifying questions on that thread.

Question 3: What is your method for bug review before asking to close an incident bug?

Answer 3: DigiCert reviews the content of a Bug before requesting closure. This process will be amended to emphasize explicit conclusion of questions and actions rather than implied conclusion when the dialogue of the Bug shifts. To avoid duplication, please see Bug 1957499 on this same Subject. As noted there, our team is conducting a sweep of our other Bugs to address loose ends.

Flags: needinfo?(dcbugzillaresponse)

Removed - I forgot I had recused myself from this matter. I'll post what I had elsewhere.

(In reply to Ben Wilson from comment #62)

Removed - I forgot I had recused myself from this matter. I'll post what I had elsewhere.

Ben, FWIW, I've posted https://bugzilla.mozilla.org/show_bug.cgi?id=1950144#c10 which seems to have support from multiple members of this community (check the reactions).

Still, I think you're correct to provide the posting guidance in a more visible part of the policies/procedures area and not in a specific bug that can be missed by community members.

(In reply to DigiCert from comment #42)

Rob,

We have been automating verification of CCADB information. You missed the rest of the post, which is not suprising [sic] - humans make mistakes and forget things. Here it is again:

I did in fact read "the rest of the post" (comment 35), as well as your previous posts on this bug, which have described your CCADB automation capabilities as follows (emphasis mine):

  • Comment 35: "We have relied on our automation to ensure all information is uploaded to CCADB within the specified timelines for several years".
  • Comment 32: "We have automated upload for subCAs, but roots are still tricky."
  • Comment 31: "The lack of such functionality prevents us from fully automating document upload."
  • Comment 28: "We would love to automate this upload process if CCADB APIs are available in the future."

This focus on uploading, combined with your statement in comment 9 that "No one reviewed the information since then" in the CCADB records themselves over a six year period, strongly suggested to me that your "verification of CCADB information" was only occurring prior to the automated upload of that verified information to CCADB.

Automation can sometimes go wrong, and so it's wise to at least occasionally verify that it has behaved as expected. This is why in comment 37 I was seeking to discover if you were "actually verifying that the data you think you're uploading to CCADB is actually reflected in the CCADB records".

Comment 42 did not reveal an answer. Let me try rephrasing my question:

(In reply to DigiCert from comment #35)

Responding to Rob Stradling in comment 33 and comment 34

Q1: Can DigiCert please clarify whether or not it is committing to regularly reviewing the accuracy of the already-included information in its CCADB records?

Yes. We are committed to regular reviews of accuracy.

Q4: Do DigiCert's regular reviews of accuracy look at the data that is actually present in CCADB? If so, how? e.g., Do you periodically view CCADB records in a web browser and manually check the details for accuracy? Do you download the CCADB AllCertificateRecordsCSVFormatv2 report and use a set of automated checks to compare the information in it against your own records? Do you periodically review issues highlighted by the crt.sh reports (e.g., https://crt.sh/mozilla-disclosures) that compare the information in AllCertificateRecordsCSVFormatv2 against root store metadata and policy?

Flags: needinfo?(dcbugzillaresponse)

Our review of this bug, pursuant to the action item from bug 1957499 has resulted in three questions which were either not answered or where the answer was not clearly associated with the question. In addition there were some statements we made regarding the audit of our CP which were incorrect and have been corrected below.

From comment 49:

  • (i) why were approaches C2 and C3 not considered viable options?
  • (ii) why was DigiCert splitting its combined CP/CPS considered preferable when compared to those other options?

We’ve previously provided answers, but we are restating these to make sure the answers are clear.

Answer (i) We read the original comment as guidance rather than questions and made a decision based on that reading.

Answer (ii) Our reading of the original post led to the wrong decision. We clarified our plan based on our updated reading of the guidance. After speaking with Microsoft about the options, Microsoft requested to go with option C3 (posting the Microsoft CP and a separate Microsoft CPS) rather than option C2. Apple is drafting a combined CP/CPS (option C2)The expected completion date of the Apple combined CP/CPS is the end of June.

In response to comment 17 from Chris Clements and comment 47 from the Chrome team regarding statements made about the auditing of our CP:

DigiCert has transitioned to a combined CP/CPS so some of the dialogue in this Bug relating to CP documentation is no longer relevant. However, we wish to clarify that the team answering this bug is not intimately involved in the annual WebTrust audits and, therefore, was not aware of any past questions related to the CP. CPS questions are fairly common and often involve clarifying how the CPS control meets a BR requirement. This limited knowledge of the audit process led to an incorrect statement that the auditors were auditing against the BRs instead of the CP. WebTrust requires external auditors to review both the CA’s CP and CPS (in the context of the CABF BRs and other applicable standards that may serve, in effect, as external CP).

In response to Ryan Dickson in comment 19:

Somewhat unrelated, the report in Comment 9 would have been more helpful if it enumerated the specific set of CAs whose corresponding CCADB records where considered incorrect.

See attached list.

Question 1: How exactly is it that you were unaware of these unanswered questions?
Answer 1: As noted by Jeremy in Comment #59, we considered the Chrome questions to have been guidance that informed and was incorporated into our answer in an earlier response at Comment #54.

I don’t understand this response. Your comment 54 states that you knew these were questions. In comment 57 you stated you were unaware of any additional questions. What happened between comment 54 and comment 57 that caused you to lose track of these unanswered questions.

Perhaps with that clarification you can provide a new response that answers question 1.

Relating to Rob Stradling’s Comment 64.

Q4: Do DigiCert's regular reviews of accuracy look at the data that is actually present in CCADB? If so, how? e.g., Do you periodically view CCADB records in a web browser and manually check the details for accuracy?

These questions are similar those you asked in Comment 33, which were answered in Comment 35 and reiterated in Comment 42. At the time of uploading cases, we manually review the CCADB web interface for accuracy of the actions implemented. See also our discussion on automated checks in Comment 35.

Do you download the CCADB AllCertificateRecordsCSVFormatv2 report and use a set of automated checks to compare the information in it against your own records?

Yes, as part of our audit programs, we regularly (i.e., several times a year) download CCADB reports (including the particular one you reference) to compare them against information in our own records. We also have an automated check process; see also our discussion on automated checks in Comment 35. We continue to evolve our automated checks, and, as noted, have added a check on CP/CPS accuracy.

Do you periodically review issues highlighted by the crt.sh reports (e.g., https://crt.sh/mozilla-disclosures) that compare the information in AllCertificateRecordsCSVFormatv2 against root store metadata and policy?

As part of our automation, we run our own reports using the same datasource as used by crt.sh, (i.e., https://crt.sh/mozilla-disclosures). In addition, as part of our audit programs, we also manually review the crt.sh reports on a quarterly basis.

Relating to Tim Callan’s Comment 66.

I don’t understand this response. Your comment 54 states that you knew these were questions. In comment 57 you stated you were unaware of any additional questions. What happened between comment 54 and comment 57 that caused you to lose track of these unanswered questions. Perhaps with that clarification you can provide a new response that answers question 1.

We did not lose track of Chrome’s questions. As we have explained, at that time we believed the Chrome questions in Comment 38 were guidance, which we took onboard in forming the approach forward, which we described in subsequent comments.

In our Comment 61 we acknowledged our understanding was incorrect, and we answered those questions specifically. This Bug includes more than 5 dozen comments, which have included useful insight from root stores and the community. As laid out in this extended dialogue, DigiCert has amended its approach to Bug handling, and our approach to the core issues in this Bug has been updated based on new information.

Flags: needinfo?(dcbugzillaresponse)

In response to comment 66

I don’t understand this response. Your comment 54 states that you knew these were questions. In comment 57 you stated you were unaware of any additional questions. What happened between comment 54 and comment 57 that caused you to lose track of these unanswered questions.

We believe all these questions were answered.

Comment 54 answered both questions.

From the first paragraph, the answer to (i) was:

We would like to thank the Chrome team for their clarifications. Based on the conversation above, we did mis-interpret Chrome's post as guidance rather than questions.

The answer to (ii) was:

We plan to keep our combined CP/CPS and change our CCADB records to reflect our CP/CPS for our roots and the subCAs we operate (option #3-C1 from comment 38), and specify the Apple CP/CPS for the subCAs that are signed by our roots, but operated by Apple (option #3-C2 from comment 38).

Therefore both questions were answered when we posted comment 57,

During our review, we decided that we should explicitly tie the two statements from comment 54 cited above directly back to the questions to avoid any confusion. Subsequent conversations with Microsoft required that we update our answer to question (ii).

Perhaps with that clarification you can provide a new response that answers question 1.

Question 1 was already answered and that answer remains the same.

We have updated the entries in CCADB for Microsoft’s externally-operated subCAs to point to the Microsoft CP and CPS. The only remaining remediation item is for Apple to draft and publish their combined CP/CPS, which they plan to complete by the end of June.

@Ben, please set the next update on this bug to July 1, 2025.

Flags: needinfo?(bwilson)
Flags: needinfo?(bwilson)
Whiteboard: [ca-compliance] [disclosure-failure] → [ca-compliance] [disclosure-failure] Next update 2025-07-01

(In reply to DigiCert from comment #67)

Relating to Rob Stradling’s Comment 64.

Q4: Do DigiCert's regular reviews of accuracy look at the data that is actually present in CCADB? If so, how? e.g., Do you periodically view CCADB records in a web browser and manually check the details for accuracy?

These questions are similar those you asked in Comment 33, which were answered in Comment 35 and reiterated in Comment 42.

And yet further clarity was needed. Hence Q4.

Yes, as part of our audit programs, we regularly (i.e., several times a year) download CCADB reports (including the particular one you reference) to compare them against information in our own records.
...
As part of our automation, we run our own reports using the same datasource as used by crt.sh, (i.e., https://crt.sh/mozilla-disclosures). In addition, as part of our audit programs, we also manually review the crt.sh reports on a quarterly basis.

Thank you for providing this information.

In comment 33 I was seeking to 'establish whether or not "No one reviewed the information since then" was an isolated mishap'. These measures you've described, which periodically and proactively verify the data that is actually present in CCADB, should go a long way to ensuring that it was, indeed, an isolated mishap.

(In reply to DigiCert from comment #61)

Question 1: How exactly is it that you were unaware of these unanswered questions?

Answer 1: As noted by Jeremy in Comment #59, we considered the Chrome questions to have been guidance that informed and was incorporated into our answer in an earlier response at Comment #54. To avoid confusion in future, unless it interferes with readability, we will explicitly call out questions and their answers.

Thank you DigiCert for recognising that the clearly labelled and important questions asked by the Chrome Root Program in comment 38 (prompted by Sectigo's perspective that I shared in comment 34) were indeed questions, and for attempting to answer them.

Chrome 1: How would DigiCert expect a relying party to know that DigiCert’s CP/CPS is the controlling document for any certificate issued under that hierarchy, while also accounting for the separate CPS of the issuing CA?

As noted in our earlier responses, CCADB will show our CP/CPS for our roots and the subCAs we operate (option 3-C1 from Comment #38), and show the Apple CP/CPS for the subCAs that are signed by our roots, but operated by Apple (option 3-C2 from Comment #38).

I fully support this planned future state, but this reply does not actually address the question (Chrome 1).

The question pertained to the current state, which for Apple is that the following Non-Audit Documents are disclosed to CCADB for the Sub-CAs under DigiCert roots:

  • Apple's latest Public CPS (which is not a Combined CP/CPS).
  • "CP Same As Parent" ticked.
  • No Standalone CP document inherited from any parent (or grandparent, etc) CCADB record.
  • DigiCert's Combined CP/CPS disclosed for the corresponding DigiCert Root Certificate records, which DigiCert has asserted in comment 35 "is the CP and it is the controlling document for any certificate issued under the hierarchy of any of our root certificates".

Chrome 2: If the CP/CPS is the controlling document, is there value in the separate CPS, or does that just create an opportunity for confusion?

Does not apply. We have a CP/CPS, and do not have a separate CPS. Apple and Microsoft are in the process of updating their respective CP/CPS documents.

Again, "in the process of" describes a planned future state, but the question pertained to the then current state.

The "separate CPS" referred to in this question is the CPS of the externally-operated Sub-CA (e.g., the Apple Public CPS). Since it seems that DigiCert has misunderstood this, I think we can probably all agree that having a separate CPS in addition to a CP/CPS that is the controlling document does indeed "create an opportunity for confusion"!

Since the Apple Public CPS is still in use today, "Does not apply" is an entirely inappropriate response to the question. DigiCert needs to take responsibility for the compliance of the current state of its Non-Audit Document disclosures, not just the compliance of the planned future state.

Chrome 3: If the set of CAA domains disclosed in the DigiCert CP/CPS are considered as a maximum set of possible options (i.e., requirements, as one would expect of a CP), how would pki.apple.com or microsoft.com (when viewed as the in-use practice) be compatible?

Does not apply. The CAA domains of the applicable CP/CPS apply. In other words, the DigiCert, Apple, and Microsoft CP/CPS documents list their respective CAA domains.

This reply, again, was a forward-looking statement from DigiCert that did not reflect the then current reality. And it still does not reflect the current reality in the case of Apple. Although an Apple Combined CP/CPS document is planned, it does not yet exist, as DigiCert admitted in its answer to the previous question (Chrome 2) and again in comment 69. DigiCert's reply to this question (Chrome 3) expects us to accept that non-compliance today is absolutely fine on the grounds that a compliant solution is expected to be delivered at some point in the future.

As mentioned in comment 69, a Microsoft CP and Microsoft CPS document were disclosed just under a week ago on the Intermediate Certificate records in CCADB for the Sub-CAs operated by Microsoft under DigiCert's roots. However, for the last several months, the CCADB disclosures for these Microsoft Sub-CAs have looked like this:

  • No CPS, and "CPS Same As Parent" unticked.
  • No CP/CPS, and "CP/CPS Same As Parent" unticked.
  • "CP Same As Parent" ticked.
  • No Standalone CP document inherited from any parent (or grandparent, etc) CCADB record.
  • DigiCert's Combined CP/CPS disclosed for the corresponding DigiCert Root Certificate records, which DigiCert asserted in comment 35 "is the CP and it is the controlling document for any certificate issued under the hierarchy of any of our root certificates".

(Until the updated disclosure mentioned by comment 69 occurred, https://crt.sh/mozilla-disclosures was flagging the Intermediate Certificate records for the Microsoft Sub-CAs because neither a CPS nor CP/CPS had been disclosed (either directly, or by ticking “CPS Same As Parent” or “CP/CPS Same As Parent”)).

Q5: Does DigiCert accept that this long period without any disclosed CPS (or CP/CPS) for the Microsoft Sub-CAs under DigiCert roots constitutes another, new compliance incident?

In comment 5, Ben wrote that "a CPS provides the CA's implementation details that show conformity with the CP" to express that he agreed with Tim H's similar description in comment 4. Since, as asserted by DigiCert in comment 35, the DigiCert CP/CPS "is the CP and it is the controlling document for any certificate issued under the hierarchy of any of our root certificates", this means that the CAA identifiers in the DigiCert CP/CPS form part of the policy that the effective CPS MUST conform to.

In the case of Apple, the effective CPS (the Apple Public CPS) permits only "pki.apple.com" in the "issue" and "issuewild" CAA properties to be treated as permission to issue; whereas the effective CP (the DigiCert CP/CPS, according to DigiCert) permits 19 identifiers in the "issue" and "issuewild" properties to be treated as permission to issue, none of which are "pki.apple.com". For as long as these documents remain the effective CP and CPS for the Apple Sub-CAs, it stands to reason that Apple MUST block issuance whenever the Relevant RRset from a CAA lookup contains any property tags that restrict issuance. In particular, if Apple issues a certificate after seeing a CAA "issue" property that asserts "pki.apple.com" (or issues a wildcard certificate after seeing a CAA "issuewild" property that asserts "pki.apple.com"), then that's misissuance, because "pki.apple.com" is not permitted by the effective CP (the DigiCert CP/CPS).

I am not trying to get Apple (or Microsoft) into trouble here. Far from it! I'm keen to ensure that both of these Sub-CA operators are able to, and are enabled to, operate all of their Sub-CAs in a compliant manner. The Apple CPS clearly states that the DigiCert CP (and not the newer DigiCert Combined CP/CPS) applies, and I believe that Apple has been endeavouring to operate its Sub-CAs under DigiCert roots on that basis for many years.

This bug was created because DigiCert disclosed the wrong CP (Apple's Private CP) for Apple's Sub-CAs, which created a misalignment between Apple's actual issuance practice and the effective policy documents. The interim solution to that problem has not improved the situation: DigiCert's current CCADB disclosures and assertion that the DigiCert CP/CPS "is the CP and it is the controlling document for any certificate issued under the hierarchy of any of our root certificates" have also created a misalignment between Apple's actual issuance practice and the effective policy documents.

Let's be absolutely clear: A CA's actual issuance practice needs to be aligned with the corresponding effective policy documents (whether that's a separate CP and CPS, or a Combined CP/CPS). Misalignment equates to misissuance. Promising a future update to the policy documents that will fix such misalignment does not retroactively reclassify misissued certificates as correctly issued and it does not excuse misissuance today. Deliberately overlooking such misissuance because the CA considers the misalignment to be minor, or "not a security issue", or as something that can be fixed in the future, etc, will not end well - this is precisely what set Entrust on a path to distrust (see bug 1883843 and bug 1890896)!

Q6: Does DigiCert accept that any list of permitted CAA identifiers asserted by an effective CP constrains the set of CAA identifiers that any corresponding effective CPS may permit?

Q7: Does DigiCert accept that the current effective “CP” for the Apple Sub-CAs that are externally-operated under DigiCert roots does not permit Apple to treat "pki.apple.com" as permission to issue?

Q8: If yes to Q7, what does DigiCert intend to do about this?

Chrome Root Program,

Do you concur with my critique of DigiCert's replies to your questions from comment 38?

Flags: needinfo?(dcbugzillaresponse)
Flags: needinfo?(chrome-root-program)

Responding to Comment 71:

Chrome Root Program,
Do you concur with my critique of DigiCert's replies to your questions from comment 38?

Thank you for the detailed analysis and for helping to clarify the underlying concerns with previous responses. The critique does appear fair and reasonable, in so much that the previous responses from DigiCert to our questions did not address the current state, especially considering the DigiCert statement in Comment 35: “DigiCert's CP/CPS is the CP and it is the controlling document for any certificate issued under the hierarchy of any of our root certificates, so no, we do not agree that there is a problem.”

Misalignment between disclosed policies and actual issuance practices, regardless of intent to resolve in the future, can constitute non-compliance and potentially misissuance. This incident began as what one might perceive to be a relatively simple CCADB disclosure error, but has since introduced confusion around compliance responsibility and effective governance of externally-operated Subordinate CAs. We’re interested in understanding DigiCert’s perspective of the current state and hope clarity can be provided.

Separately, to clarify the expectations of the CCADB Steering Committee and do our part in reducing the likelihood for future issues similar to those described here, we’ve proposed a draft update to the CCADB Policy.

Flags: needinfo?(chrome-root-program)

Thank you for the detailed questions.

Current State
The documentation in CCADB associated with Apple’s externally operated Sub-CAs under DigiCert roots includes:

  • Apple’s standalone Public CPS (not a combined CP/CPS),
  • “CP Same As Parent” ticked,
  • No standalone CP linked to Apple in CCADB,
  • DigiCert’s combined CP/CPS published for the corresponding root certificates.

DigiCert has stated in Comment 35 that its Combined CP/CPS functions as the controlling CP for all certificates issued under its roots. The documents and settings currently in CCADB accurately represent the policy landscape governing the Apple hierarchy:

  • Apple’s CPS governs how Apple operates its PKI.
  • DigiCert’s Combined CP/CPS applies to Sub-CAs issued under DigiCert’s roots, unless a Sub-CA provides its own CP or CPS, as Apple does.
  • Since Apple publishes and operates under its own CPS, that CPS governs Apple’s issuance practices, including CAA processing.

Per Section 2.2 of the Baseline Requirements (BRs):

“The CA SHALL publicly disclose its Certificate Policy and/or Certification Practice Statement…”

This requirement is fulfilled through Apple’s published CPS and DigiCert’s Combined CP/CPS. There is not a requirement for a separate CP and CPS.

Per CCADB Policy Section 5, CAs must publish authoritative CP and/or CPS documents. Apple’s CPS is public and available via CCADB.

Response to Specific Questions and Issues

Q5: Does DigiCert accept that this long period without any disclosed CPS (or CP/CPS) for the Microsoft Sub-CAs under DigiCert roots constitutes another, new compliance incident?

No. Microsoft’s CPS has been properly uploaded and disclosed in CCADB throughout. We confirmed this while reviewing Apple’s documentation.

Concern: Misalignment between Apple CPS and DigiCert CP/CPS regarding CAA domain authorization.

The Apple CPS defines Apple’s CAA processing behavior. Section 3.2.2.8 of the BRs allows a CA to define authorized CAA domains in either the CP or the CPS. Since Apple maintains its own CPS, Apple’s CPS governs their interpretation of CAA records.

Specifically, Apple's CPS restricts issuance to domains with issue or issuewild tags containing "pki.apple.com". That behavior is explicitly permitted by their own CPS and aligns with Section 3.2.2.8 of the BRs.

While DigiCert’s CP/CPS contains a broader set of CAA domains, it does not override Apple’s CPS in the context of Apple-operated Sub-CAs.

Q6: Does DigiCert accept that any list of permitted CAA identifiers asserted by an effective CP constrains the set of CAA identifiers that any corresponding effective CPS may permit?

No. Section 3.2.2.8 of the BRs allows CAA identifiers to be defined in either the CP or CPS. There is no hierarchical constraint between the two in this context. Because Apple provides its own CPS, it is permitted to define pki.apple.com as its authorized CAA domain.

Q7: Does DigiCert accept that the current effective “CP” for the Apple Sub-CAs under DigiCert roots does not permit Apple to treat "pki.apple.com" as permission to issue?

No. Apple’s CPS is the governing document for issuance decisions by the Apple Sub-CAs. That CPS explicitly authorizes issuance based on the pki.apple.com CAA tag, which aligns with the BRs. Again, “pki.apple.com” is listed in CCADB under “Recognized CAA Domains,” further supporting alignment between operational practice and disclosed documentation.

Q8: If yes to Q7, what does DigiCert intend to do about this?

Not applicable. Apple’s current CPS permits the behavior, and the governing documents are aligned with policy requirements.

Additional Clarifications for Chrome:

  1. Apple’s CPS governs how the Apple PKI operates and is the definitive document about operation of Apple Sub-CAs.
  2. DigiCert has migrated to a Combined CP/CPS. There is no separate CP controlling Apple’s issuance practices.
  3. The Baseline Requirements allow the use of either a CP, CPS, or Combined CP/CPS. Current publication practices are compliant.
  4. The draft CCADB policy does suggest a preference for a Combined CP/CPS, but it is not yet mandatory. Apple is actively working toward publishing a Combined CP/CPS to align with future expectations.
  5. Chrome’s Root Inclusion Policy (Section 2.1) does require a Combined CP/CPS for applicants. Although Apple is not applying for root inclusion; we anticipate this will be a requirement for root renewal by DigiCert.

Conclusion
We believe the current documentation structure in CCADB reflects:

  • Accurate representation of applicable policies and practices,
  • Full compliance with both the BRs and CCADB disclosure requirements,
  • No material misalignment between Apple’s CPS and actual issuance behavior.

Apple’s CPS governs their issuance practices, including how they process CAA records. DigiCert’s Combined CP/CPS is applicable at the root level but does not override the Apple CPS in the operation of Sub-CAs.

We appreciate the ongoing dialogue and remain committed to improving clarity, transparency, and alignment as expectations evolve.

Flags: needinfo?(dcbugzillaresponse)

Last week, we answered all open questions. The only open remediation item is Apple’s publication of their combined CP/CPS. This item is still on schedule to be completed by the end of June.

We request that a next update of 2025-07-01 be set on this bug.

(In reply to DigiCert from comment #73)

...
Per Section 2.2 of the Baseline Requirements (BRs):

“The CA SHALL publicly disclose its Certificate Policy and/or Certification Practice Statement…”

This requirement is fulfilled through Apple’s published CPS and DigiCert’s Combined CP/CPS. There is not a requirement for a separate CP and CPS.

Per CCADB Policy Section 5, CAs must publish authoritative CP and/or CPS documents. Apple’s CPS is public and available via CCADB.

You are correct that this is what the BRs and CCADB Policy v1.3.1 require. However, those requirements are in fact overridden by stricter policy requirements and disclosure statements:

Firstly, as Martijn pointed out all the way back in comment 3, "In the specific context of this bug, note that in the applicable and disclosed CPS Apple explicitly states in Appendix A that DigiCert's CP and Sectigo's CP are the applicable Certificate Policies that that CPS is paired with. To disclose any other CP to CCADB is, at the very least, inconsistent with that statement."

Secondly, Chrome Root Program Policy (CRPP) section 2.2 requires that CA Owners "MUST minimally...accurately describe the policies and practices of their CA(s) within a Certificate Policy (CP) and corresponding Certification Practice Statement (CPS), or preferably, a single document combined as a CP/CPS" (emphasis mine) for all "PKI Hierarchies included in the Chrome Root Store". That's an "and" (not an "and/or"), meaning that both a CP and CPS (or a combined CP/CPS) are required in every case, including the case of the externally-operated Apple Sub-CAs.

I'll also note in passing that the draft CCADB Policy v2.0 proposes to bring the CCADB Policy in line with this CRPP "and" requirement (instead of "and/or").

Thirdly, DigiCert has ticked "CP Same As Parent" in the Intermediate Certificate records in CCADB for the externally-operated Apple Sub-CAs, which publicly discloses that a CP does exist and is applicable.

  • Apple’s CPS governs how Apple operates its PKI.
  • DigiCert’s Combined CP/CPS applies to Sub-CAs issued under DigiCert’s roots, unless a Sub-CA provides its own CP or CPS, as Apple does.
  • Since Apple publishes and operates under its own CPS, that CPS governs Apple’s issuance practices, including CAA processing.

That "unless" is deeply problematic here, because it is:

  • inconsistent with the fact that DigiCert has ticked "CP Same As Parent" (in the Intermediate Certificate records in CCADB for the externally-operated Apple Sub-CAs).
  • incompatible with the CRPP "and" requirement.
  • irrelevant, because Appendix A of the Apple CPS declares that the DigiCert CP is applicable.

And since "the DigiCert CP" is applicable and is (in DigiCert's words from comment 35) "the controlling document for any certificate issued under the hierarchy of any of our root certificates", the CAA requirements defined in "the DigiCert CP" are applicable to the Apple Sub-CAs too.

Q9: Does DigiCert accept that the (stricter) "and" requirement in the CRPP overrides the (less strict) "and/or" requirement in the TLS BRs and CCADB Policy v1.3?

Q10: If so, have I now persuaded DigiCert to accept that "the DigiCert CP" is currently applicable to the Apple Sub-CAs that are externally-operated under DigiCert roots?

Q6: Does DigiCert accept that any list of permitted CAA identifiers asserted by an effective CP constrains the set of CAA identifiers that any corresponding effective CPS may permit?

No. Section 3.2.2.8 of the BRs allows CAA identifiers to be defined in either the CP or CPS. There is no hierarchical constraint between the two in this context. Because Apple provides its own CPS, it is permitted to define pki.apple.com as its authorized CAA domain.

Yes, either document can define authorized CAA domains. However, I don't agree that there's "no hierarchical constraint between the two in this context". In comment 5, Ben wrote "a CPS provides the CA's implementation details that show conformity with the CP". So if the effective CP requires a particular set of authorized CAA domains, then the effective CPS must "show conformity with the CP" in that matter.

Q11: Does DigiCert agree with this reasoning?

Q12: If so, have I now persuaded DigiCert to accept that the mismatch in authorized CAA domains between "the DigiCert CP" and the Apple CPS is problematic?

Additional Clarifications for Chrome:
...
2. DigiCert has migrated to a Combined CP/CPS. There is no separate CP controlling Apple’s issuance practices.

In its first attempt to remediate this bug, DigiCert "updated the CP information in CCADB to list the DigiCert CP instead of the Apple CP" (comment 28), because the DigiCert CP "is the controlling document for any certificate issued under the hierarchy of any of our root certificates" (comment 35).

Yet now, DigiCert would also have us believe the conflicting statement that there "is no separate CP controlling Apple's issuance practices", even though "CP Same As Parent" remains ticked in the relevant CCADB Intermediate Certificate records and even though the Apple CPS still declares that the DigiCert CP is applicable.

Chrome Root Program,

  • Do you agree with my assertion that the CRPP requires a CP (either a standalone document or a Combined CP/CPS) to exist and to be disclosed to CCADB for each of the serverAuth-capable Apple Sub-CAs that are externally-operated under DigiCert roots that are trusted by Chrome?

(In reply to chrome-root-program from comment #72)

Misalignment between disclosed policies and actual issuance practices, regardless of intent to resolve in the future, can constitute non-compliance and potentially misissuance.

  • Do you agree with me that the current misalignment between the disclosed and actual issuance practices for the Apple Sub-CAs externally-operated under DigiCert roots does "constitute non-compliance and potentially misissuance"?

(In reply to DigiCert from comment #73)

Q5: Does DigiCert accept that this long period without any disclosed CPS (or CP/CPS) for the Microsoft Sub-CAs under DigiCert roots constitutes another, new compliance incident?

No. Microsoft’s CPS has been properly uploaded and disclosed in CCADB throughout. We confirmed this while reviewing Apple’s documentation.

Sigh. I know what I saw. I was planning to report this as a current problem, but DigiCert happened to fix it before I was able to do that. (I began drafting comment 71 a while before DigiCert posted comment 69 - these long and detailed comments take a considerable amount of time to thoroughly investigate, draft, and peer review). But alas, I have no photographic evidence with which I can prove to DigiCert that it is mistaken.

Flags: needinfo?(dcbugzillaresponse)
Flags: needinfo?(chrome-root-program)

[In response to Comment 75]

Do you agree with my assertion that the CRPP requires a CP (either a standalone document or a Combined CP/CPS) to exist and to be disclosed to CCADB for each of the serverAuth-capable Apple Sub-CAs that are externally-operated under DigiCert roots that are trusted by Chrome?

Yes. We feel this expectation is very clear in Section 2.2 of our policy by the following phrase: “...accurately describe the policies and practices of their CA(s) within a Certificate Policy (CP) and corresponding Certification Practice Statement (CPS), or preferably, a single document combined as a CP/CPS.”

This expectation has existed since Version 1.1 of our policy, which landed prior to the functional launch of the Chrome Root Program and Chrome Root Store in September 2022. Many times since then, all of the CA Owners included in the Chrome Root Store have acknowledged their (1) understanding of, and (2) commitment to adhere to this policy.

This position was further reiterated earlier in this discussion, and again in the set of updates proposed for CCADB Policy Version 2.0.

Do you agree with me that the current misalignment between the disclosed and actual issuance practices for the Apple Sub-CAs externally-operated under DigiCert roots does "constitute non-compliance and potentially misissuance"?

Yes, we agree that the current misalignment constitutes non-compliance with our disclosure requirements (described above).

However, it is not clear to us that this disclosure non-compliance constitutes misissuance. Taken as a more extreme example, if a CA Owner accidentally disclosed the location of its Subscriber Agreement as its CP in the CCADB, we would not automatically consider all subsequent certificate issuance as misissuance. Further, we have no indication Apple has violated its CPS at any point in the relevant period beyond one instance already disclosed to this community.

We’d like to emphasize our general disappointment with the response to this incident. This report was opened 7 months ago, and has now surpassed 75 comments — all for what should have been a very simple and uncontroversial CCADB disclosure failure. We believe there have been many avenues that could have instead been pursued to close this issue much sooner than proposed (i.e., July 1, 2025). We encourage DigiCert to prioritize alignment with the proposed future state of the CCADB Policy - including the pursuit of solutions that can be implemented without a dependency on the work of its partners moving to a combined CP/CPS.

From our opinion, as it is trending, continued discussion in this bug appears to offer the community very little benefit re: best practices, lessons learned, or improving user security. Back in Comment 29, we offered DigiCert the opportunity to commit to an action of leading positive change for what was described as “an industry-wide issue.” However, as far as we can tell, the only remaining action item associated with this report is assigned to Apple.

We feel our expectations re: policy disclosures are clear, and are being further underlined via the planned CCADB Policy update. If anyone else sees this differently, we’d still welcome those opinions.

Flags: needinfo?(chrome-root-program)

We understand that there may be differences in interpretation between our documentation and what Chrome expected in their policy. We aligned with the Chrome plan in these discussions by moving to a combined CP/CPS for DigiCert and signed CAs. We also provided feedback to CCADB as part of this effort. We apologize if we misunderstood the ask on policy alignment. We believe that requiring all parties to use a combined CP/CPS is the best path in meeting Chrome’s future policy goals. We agree with the Chrome root program’s analysis that the certificates were issued in accordance with the Apple CPS, which means mis-issuance did not occur.

Flags: needinfo?(dcbugzillaresponse)

The only open remediation item is Apple’s publication of their combined CP/CPS. This item is still on schedule to be completed by the end of June.

We request that a next update of 2025-07-01 be set on this bug.

(In reply to Rob Stradling from comment #75)

...Q9...Q10...Q11...Q12

I asked four questions of DigiCert just over two weeks ago. The CCADB Policy requires that CAs "MUST respond within 7 days", but those four questions have not yet been answered.

This oversight is surprising, given DigiCert's recently executed action items in bug 1957499 that were designed to put a stop to its persistent failure to answer questions in a timely manner.

(In reply to chrome-root-program from comment #76)

We encourage DigiCert to prioritize alignment with the proposed future state of the CCADB Policy - including the pursuit of solutions that can be implemented without a dependency on the work of its partners moving to a combined CP/CPS.

Comment 78 shows that DigiCert has absolutely no intention of prioritizing or pursuing such a solution. Instead, DigiCert expects the community to continue to accept the non-compliance for yet another month.

Let's put this into context: over in bug 1962829, due to a "typographical error" in a non-policy document, the community is expecting another CA to promptly revoke tens of millions of certificates; whereas here in this bug there is a non-policy document that is either:

  1. entirely missing (if we accept the statement in comment 73 that "There is no separate CP controlling Apple's issuance practices" even though "CP Same As Parent" is ticked), or

  2. present (if we instead accept the statement in comment 35 that "DigiCert's CP/CPS is the CP and it is the controlling document for any certificate issued under the hierarchy of any of our root certificates") but causing 100% misissuance of leaf certificates by Apple whenever any CAA issuance restriction records are present.

Surely an entirely-missing non-audit document is at least as serious a compliance issue as a non-audit document that is present but has a "typographical error"?

Surely misissuance due to CAA restrictions is at least as serious a compliance issue as missuance due to a “typographical error”?

Chrome Root Program,
Do you have anything else to say on these matters?

Flags: needinfo?(chrome-root-program)

Rob, thank you for pointing out that some of your questions did not get answers. Looking into it this morning, we do have a response to your questions, which was drafted and approved at the time, but not posted. We wanted to get you the answers immediately so we’re posting the response that was originally scheduled to go out:

Response to Questions

Header of Response: Response addresses Comment 75 and questions #9, #10, #11 and #12 within comment 75.

We are reluctant to repeatedly re-litigate past events in the flow of the Bug when, during the Bug, clarifications were made to root program policies, and CAs adapted their policy documentation correspondingly. That is the goal of the Bug process.

DigiCert, and our external cross-signed CAs Microsoft and Apple, seek to be compliant with the standards and community expectations.

Q9 Does DigiCert accept that the (stricter) "and" requirement in the CRPP overrides the (less strict) "and/or" requirement in the TLS BRs and CCADB Policy v1.3?

DigiCert Response #1

Please refer to our previous response in Comment 73 including:

  • Apple’s CPS governs how the Apple PKI operates and is the definitive document about operation of Apple Sub-CAs.

  • DigiCert has migrated to a Combined CP/CPS. There is no separate CP controlling Apple’s issuance practices.

  • The TLS BR allow the use of either a CP, CPS, or Combined CP/CPS. Current publication practices are compliant.

  • The draft CCADB policy suggests a preference for a Combined CP/CPS, but it is not yet mandatory. Apple is actively working toward publishing a Combined CP/CPS to align with future expectations.

  • Chrome’s Root Inclusion Policy (Section 2.1) requires a Combined CP/CPS for PKI applicants. Although Apple is not applying for root inclusion; we anticipate this will be a requirement for root renewal by DigiCert.

Q10 If so, have I now persuaded DigiCert to accept that "the DigiCert CP" is currently applicable to the Apple Sub-CAs that are externally-operated under DigiCert roots?

DigiCert’s Response #2

This question is a restatement of previous questions that were answered in our previous responses.

Specific to the subject of this Bug, DigiCert has stated in Comment 35 and Comment 73 that:

  • DigiCert’s Combined CP/CPS functions as the controlling CP for all certificates issued under its roots. Apple’s CPS governs how Apple operates its PKI.

  • DigiCert’s Combined CP/CPS applies to Sub-CAs issued under DigiCert’s roots, unless a Sub-CA provides its own CP or CPS, as Apple does.

  • Since Apple publishes and operates under its own CPS, that CPS governs Apple’s issuance practices, including CAA processing.

Q11 Does DigiCert agree with this reasoning?

Digicert Response #3

DigiCert has stated in Comment 35 and Comment 73 that the Apple CPS governs Apple’s issuance practices, including CAA processing.

Q12 If so, have I now persuaded DigiCert to accept that the mismatch in authorized CAA domains between "the DigiCert CP" and the Apple CPS is problematic?

Digicert Response #4

This question restates previous questions that were answered in our response Comment 73.

Specific to the subject of this Bug, Apple’s CPS governs that CA’s issuance practices, including how they process CAA records.

Action Item Update:

Apple reports that they are making good progress on creating their CP/CPS and expects to complete it by the 2025-07-01 deadline.

[In response to the questions in Comment 79]

From our view, it seems discussion in this report ultimately unraveled due to incomplete planning (i.e., DigiCert moving forward with a combined CP/CPS without considering the downstream consequence to partners like Apple), needlessly rigid views that DigiCert is unwilling to reconsider as being flawed (“DigiCert's CP/CPS is the CP and it is the controlling document for any certificate issued under the hierarchy of any of our root certificates” - especially when considering externally-managed partners), and an unwillingness to take what we consider simple and sensible steps forward (e.g., taking the last version of DigiCert’s CP (last effective 7/28/2024) and modifying it such that it was fit for Apple’s use) that could have satisfactorily addressed the precipitating issue months ago.

We agree that DigiCert did not respond to questions presented in Comment 75 as required by the CCADB Incident Reporting Guidelines. This behavior runs counter to the commitments made in 1957499. DigiCert has since responded.

We reiterate:

  • we have no indication Apple has violated its CPS at any point in the relevant period beyond one instance already disclosed to this community.
  • we feel our expectations re: policy disclosures are clear, and are being further underlined via the planned CCADB Policy update.
  • we encourage DigiCert to prioritize alignment with the proposed future state of the CCADB Policy - including the pursuit of solutions that can be implemented without a dependency on the work of its partners moving to a combined CP/CPS.
  • we feel there is diminishing value with the continued discussion of this specific incident. While there could be value in continued discussions re: the future of Web PKI policies, they are better fit for a distribution like public@ccadb.org.

If anyone else sees this differently or has any other ideas, your participation is welcome.

Flags: needinfo?(chrome-root-program)

In response to Comment 82, we thank the Chrome team for the additional feedback.

DigiCert planned the changes to our CP/CPS, in conjunction with our cross-signed CAs and made changes we thought best aligned with CCADB policy and the root store program. We have found the additional clarification provided during the course of this Bug to be very helpful.

We have taken note of the Chrome guidance and are drafting a DigiCert CP covering external CAs, until such time as their CP/CPS are published. This CP requires approval by the DigiCert Policy Authority; we expect that CP to be published next week.

I believe the context and wording of my question Q9 in comment 75 were very clear...

Secondly, Chrome Root Program Policy (CRPP) section 2.2 requires that CA Owners "MUST minimally...accurately describe the policies and practices of their CA(s) within a Certificate Policy (CP) and corresponding Certification Practice Statement (CPS), or preferably, a single document combined as a CP/CPS" (emphasis mine) for all "PKI Hierarchies included in the Chrome Root Store". That's an "and" (not an "and/or"), meaning that both a CP and CPS (or a combined CP/CPS) are required in every case, including the case of the externally-operated Apple Sub-CAs.

...
Q9: Does DigiCert accept that the (stricter) "and" requirement in the CRPP overrides the (less strict) "and/or" requirement in the TLS BRs and CCADB Policy v1.3?

...and yet this was DigiCert's response to Q9 in comment 80...

DigiCert Response #1

Please refer to our previous response in Comment 73 including:

  • Apple’s CPS governs how the Apple PKI operates and is the definitive document about operation of Apple Sub-CAs.
  • DigiCert has migrated to a Combined CP/CPS. There is no separate CP controlling Apple’s issuance practices.
  • The TLS BR allow the use of either a CP, CPS, or Combined CP/CPS. Current publication practices are compliant.
  • The draft CCADB policy suggests a preference for a Combined CP/CPS, but it is not yet mandatory. Apple is actively working toward publishing a Combined CP/CPS to align with future expectations.
  • Chrome’s Root Inclusion Policy (Section 2.1) requires a Combined CP/CPS for PKI applicants. Although Apple is not applying for root inclusion; we anticipate this will be a requirement for root renewal by DigiCert.

I understand from bug 1957499 comment 16 that before belatedly posting comment 80 DigiCert's representatives performed an internal review "with other DigiCert teams for fact-checking to ensure it answered the questions". However, all I can see in this "DigiCert Response #1" portion of comment 80 is a set of statements repeated from comment 73 that completely fail to answer my question Q9. The response does not even consider the "and" requirement in CRPP 2.2, which was the whole point of Q9!

I can't tell if DigiCert was deliberately dodging Q9 or somehow failing to understand it. Either way, I find this failure to answer the actual question very concerning.

(In reply to chrome-root-program from comment #82)

we feel there is diminishing value with the continued discussion of this specific incident
... If anyone else sees this differently or has any other ideas, your participation is welcome.

In comment 83, after repeated efforts from both chrome-root-program and myself, DigiCert has finally understood that it must disclose a standalone CP alongside Apple's standalone CPS, and so a resolution to this incident bug is at last in sight.

I do have some further questions for chrome-root-program at this point though...

(In reply to chrome-root-program from comment #76)

Do you agree with me that the current misalignment between the disclosed and actual issuance practices for the Apple Sub-CAs externally-operated under DigiCert roots does "constitute non-compliance and potentially misissuance"?

Yes, we agree that the current misalignment constitutes non-compliance with our disclosure requirements (described above).

However, it is not clear to us that this disclosure non-compliance constitutes misissuance.

The goalposts have moved since that Q&A. ISTM that disclosing the wrong non-audit document to CCADB (which is how this incident bug began) is very different to the issue of a required non-audit document not even existing (which is what we're talking about now).

Since the intended scope of this bug is the former (wrong CP document disclosed), should there be a new incident bug for the latter (non-existent CP document)?

Does failing to even have a CP (either a standalone CP or a combined CP/CPS) constitute misissuance?

(Related thought experiment): Would failing to have a CPS (either a standalone CPS or a combined CP/CPS) constitute misissuance?

Flags: needinfo?(chrome-root-program)

In response to comment 84.

The goalposts have moved since that Q&A. ISTM that disclosing the wrong non-audit document to CCADB (which is how this incident bug began) is very different to the issue of a required non-audit document not even existing (which is what we're talking about now).

Since the intended scope of this bug is the former (wrong CP document disclosed), should there be a new incident bug for the latter (non-existent CP document)?

We agree that the discussion has evolved in the course of this Bug, but disagree with your conclusion. This Bug has always been a case of disclosing the wrong CP, not disclosing no CP. DigiCert has maintained from the beginning that our combined CP/CPS was the controlling CP for Apple’s stand-alone CPS. The logic requiring only a standalone CP governing a standalone CPS was not made clear, nor even proposed, prior to Chrome’s comment 38 to this Bug.

Does failing to even have a CP (either a standalone CP or a combined CP/CPS) constitute misissuance?

There was no such failure related to this Bug.

(Related thought experiment): Would failing to have a CPS (either a standalone CPS or a combined CP/CPS) constitute misissuance?

That hypothetical case has no bearing on the facts of this Bug, as the relevant policy documents existed and were in effect. In the course of this Bug however, requirements were clarified and DigiCert complied with the updated requirement.

Action Item Update
We have completed a standalone CP which has been approved and posted to our Repository. See “DigiCert Public Trust Certificate Policy for Third Party TLS-Issuing CAs” at https://www.digicert.com/legal-repository. The links in the relevant CCADB subCA entries have been updated.

[In response to Comment 85]

In comment 83, after repeated efforts from both chrome-root-program and myself, DigiCert has finally understood that it must disclose a standalone CP alongside Apple's standalone CPS, and so a resolution to this incident bug is at last in sight.

We find it concerning that questions from Chrome earlier in this report were misinterpreted as guidance and that this misinterpretation negatively impacted the handling of this issue. More recently, an exempli gratia by Chrome again appears mistaken as guidance. It’s unclear to us how clearly marked questions and examples are considered “guidance,” or what’s causing DigiCert to misinterpret some of our responses but not others (e.g., in Comment 11 we asked if DigiCert had considered a combined CP/CPS for Apple and the answer at the time was no, but which was then considered by DigiCert in Comment 53.) Combined with the fact that a standalone CP was suggested back in Comment 34, it’s disappointing that an example provided ~50 comments later (i.e., Comment 82) seems to have landed a resolution. However, yes, we’ll be happy to move on from this report.

Rob, regarding your other questions about the non-existence of a CP, we've reviewed this entire report and, in general, found it somewhat challenging to discern due to the changing positions and ambiguity over time.

However, if we consider the artifacts listed in the DigiCerts' own publicly accessible repository to be true:

… then practically speaking we seem to have a CP (as part of a combined CP/CPS) in existence since July 29, 2024. Despite a literal and standalone CP document not being disclosed to the CCADB, the requirements found therein would be expected to exist in the disclosed CP/CPS, and at this time we do not have a reason to believe those requirements did not exist.

If we also consider the following statements from DigiCert in this report to be true:

  • “2024-09-25: DigiCert consolidates and publishes its WebPKI CP and CPS as a single document after deciding that the true CP in the industry are the root store policies and the CA/Browser Forum Baseline Requirements.” Comment 9
  • “DigiCert's CP/CPS is the CP and it is the controlling document for any certificate issued under the hierarchy of any of our root certificates” Comment 35

… then we again seem to have a CP in existence since September 25, 2024. There indeed appears to be a discrepancy in dates between the DigiCert statements and the artifacts in its publicly accessible repository; however, practically speaking it seems that a CP exists.

With all of that said, it seems that the first two questions asked are not applicable. For the third question (the thought experiment), we are curious to explore that, but it’s probably best suited for a venue like the CA/Browser Forum or public@ccadb.org.

Flags: needinfo?(chrome-root-program)

In response to [Comment 87]:

We find it concerning that questions from Chrome earlier in this report were misinterpreted as guidance and that this misinterpretation negatively impacted the handling of this issue. More recently, an exempli gratia by Chrome again appears mistaken as guidance. It’s unclear to us how clearly marked questions and examples are considered “guidance,” or what’s causing DigiCert to misinterpret some of our responses but not others (e.g., in Comment 11 we asked if DigiCert had considered a combined CP/CPS for Apple and the answer at the time was no, but which was then considered by DigiCert in Comment 53.) Combined with the fact that a standalone CP was suggested back in Comment 34, it’s disappointing that an example provided ~50 comments later (i.e., Comment 82) seems to have landed a resolution. However, yes, we’ll be happy to move on from this report.

We understand your concern about the misinterpretation of questions/examples, and we appreciate your patience with the back-and-forth that was required to reach resolution. As you know we have worked on refining our incident handling procedures and we'll work to ensure clearer communication going forward to avoid similar confusion. We appreciate the clarifications that were made to CCADB policy as a result of this dialogue.

Thank you for your patience and diligence in working through this incident to resolution

Incident Closure Summary

  • Incident Description: 

DigiCert disclosed an incorrect Certificate Policy (CP) in the Common CA Database (CCADB) for Apple-operated subordinate CAs (Sub-CAs) issued under DigiCert roots. The CP field pointed to Apple’s private CP, which is not applicable to publicly-trusted certificates. This incorrect link persisted from 2018 until it was reported in October 2024.

  • Incident Root Cause(s):

 The root cause was a longstanding oversight stemming from the CP field in CCADB being historically underutilized and unvalidated. DigiCert continued to upload the same incorrect CP annually without review. Misinterpretation of policy expectations around CP/CPS relationships for externally operated Sub-CAs further complicated the issue. These policies were restated and clarified by CCADB in the course of this Bug.

  • Remediation Description:

 DigiCert updated the CP field in CCADB to reflect the correct policy documents. For Sub-CAs operated by DigiCert, the combined CP/CPS is now disclosed. For externally operated Sub-CAs (e.g., Apple and Microsoft), DigiCert coordinated with those entities to ensure appropriate CP and CPS disclosures. Microsoft’s Sub-CAs now reference Microsoft’s standalone CP and CPS. DigiCert also created and published a new standalone CP specifically for third-party TLS-issuing Sub-CAs, which is now disclosed for Apple’s Sub-CAs until Apple completes its transition to a combined CP/CPS.

  • Commitment Summary:

 DigiCert has committed to:

  1. Regularly reviewing and verifying the accuracy of CCADB disclosures, including through automation and manual audits.

  2. Collaborating with externally operated Sub-CAs to ensure timely and accurate policy documentation.

  3. Supporting the CCADB Steering Committee’s efforts to clarify and enforce disclosure expectations.

  4. Leading or contributing to community discussions on CP/CPS best practices and policy alignment.

  5. Ensuring future updates to CP/CPS disclosures are made within 14 days of detection or change.

All Action Items disclosed in this report have been completed as described, and we request its closure.

(In reply to chrome-root-program from comment #87)

Rob, regarding your other questions about the non-existence of a CP, we've reviewed this entire report and, in general, found it somewhat challenging to discern due to the changing positions and ambiguity over time.

Some of the changing positions were due to DigiCert gradually gaining a better understanding of the problem that it needed to solve. I have no problem with that direction of travel, although it is disappointing that it took DigiCert so long to arrive at the destination.

However, DigiCert has expressed other changing positions and ambiguity in this bug that I do have a problem with, because in my view this has had the effect of derailing the conversation and discrediting legitimate questions.

However, if we consider the artifacts listed in the DigiCerts' own publicly accessible repository to be true: ... … then practically speaking we seem to have a CP (as part of a combined CP/CPS) in existence since July 29, 2024. Despite a literal and standalone CP document not being disclosed to the CCADB, the requirements found therein would be expected to exist in the disclosed CP/CPS, and at this time we do not have a reason to believe those requirements did not exist.
If we also consider the following statements from DigiCert in this report to be true:

  • “2024-09-25: DigiCert consolidates and publishes its WebPKI CP and CPS as a single document after deciding that the true CP in the industry are the root store policies and the CA/Browser Forum Baseline Requirements.” Comment 9
  • “DigiCert's CP/CPS is the CP and it is the controlling document for any certificate issued under the hierarchy of any of our root certificates” Comment 35
    … then we again seem to have a CP in existence since September 25, 2024. There indeed appears to be a discrepancy in dates between the DigiCert statements and the artifacts in its publicly accessible repository; however, practically speaking it seems that a CP exists.

I accept that some of DigiCert's statements are true, and I don't have a problem with those statements.

It's the demonstrably false statements that I have a problem with, such as the claim in comment 85 that "DigiCert has maintained from the beginning that our combined CP/CPS was the controlling CP for Apple's stand-alone CPS". To demonstrate that that statement is false, you need only look at comment 73, in which DigiCert wrote "DigiCert’s Combined CP/CPS applies to Sub-CAs issued under DigiCert’s roots, unless a Sub-CA provides its own CP or CPS, as Apple does" and "There is no separate CP controlling Apple’s issuance practices". It was necessary for me to wrestle with these mutually exclusive claims in comment 79, so that the misdirection in comment 73 could be seen for what it was.

The misdirection in comment 73 confounded the questions relating to the non-intersection of permitted CAA domains listed in the "controlling CP" (the DigiCert CP/CPS) and the Apple CPS, a topic that was first raised by chrome-root-program in comment 38, brushed aside with a forward-looking statement by DigiCert in comment 61, then picked up again by me in comment 71. Now that DigiCert has reverted to its original claim (that the DigiCert CP/CPS was the "controlling CP" for Apple), I will copy’n’paste (with light modifications) from comment 71 in order to re-ask the relevant questions that, in my view, deserve better answers...

In comment 5, Ben wrote that "a CPS provides the CA's implementation details that show conformity with the CP" to express that he agreed with Tim H's similar description in comment 4. Since, as asserted by DigiCert in comment 35, the DigiCert CP/CPS "is the CP and it is the controlling document for any certificate issued under the hierarchy of any of our root certificates", this means that the CAA identifiers in the DigiCert CP/CPS form part of the policy that the effective CPS MUST conform to.

In the case of Apple, the effective CPS (the Apple Public CPS) permits only "pki.apple.com" in the "issue" and "issuewild" CAA properties to be treated as permission to issue; whereas the effective CP (the DigiCert CP/CPS, according to DigiCert) permits 19 identifiers in the "issue" and "issuewild" properties to be treated as permission to issue, none of which are "pki.apple.com".

Let's be absolutely clear: A CA's actual issuance practice needs to be aligned with the corresponding effective policy documents (whether that's a separate CP and CPS, or a Combined CP/CPS). Misalignment equates to misissuance. Promising a future update to the policy documents that will fix such misalignment does not retroactively reclassify misissued certificates as correctly issued and it does not excuse misissuance today. Deliberately overlooking such misissuance because the CA considers the misalignment to be minor, or "not a security issue", or as something that can be fixed in the future, etc, will not end well - this is precisely what set Entrust on a path to distrust (see bug 1883843 and bug 1890896)!

Q6b: Does DigiCert accept that any list of permitted CAA identifiers asserted by an effective CP constrains the set of CAA identifiers that any corresponding effective CPS may permit?

Q7b: Does DigiCert accept that when the DigiCert CP/CPS was the effective "CP" for the Apple Sub-CAs (that are externally-operated under DigiCert roots) it did not permit Apple to treat "pki.apple.com" as permission to issue?

Q8b: If yes to Q7b, what does DigiCert intend to do about this?

chrome-root-program,

In comment 38 you asked "Question 3" on this same topic, but you didn't state your own view at that time. Do you agree with me that DigiCert should answer "Yes" to my questions Q6b and Q7b?

Flags: needinfo?(dcbugzillaresponse)
Flags: needinfo?(chrome-root-program)

Responding to Comment 90, these questions were previously asked and answered in this Bug. Nothing has changed since the last time the questions were asked.

Q6b: Does DigiCert accept that any list of permitted CAA identifiers asserted by an effective CP constrains the set of CAA identifiers that any corresponding effective CPS may permit?

No. See Comment 73 and Comment 80.

Q7b: Does DigiCert accept that when the DigiCert CP/CPS was the effective "CP" for the Apple Sub-CAs (that are externally-operated under DigiCert roots) it did not permit Apple to treat "pki.apple.com" as permission to issue?

Not applicable. See Comment 73 and Comment 80.

Q8b: If yes to Q7b, what does DigiCert intend to do about this?

Not applicable. See Comment 73 and Comment 80.

Flags: needinfo?(dcbugzillaresponse)

The Closure Summary has been posted. We have no additional updates.

Flags: needinfo?(incident-reporting)

(In reply to chrome-root-program from comment #82)

  • we encourage DigiCert to prioritize alignment with the proposed future state of the CCADB Policy - including the pursuit of solutions that can be implemented without a dependency on the work of its partners moving to a combined CP/CPS.

(In reply to DigiCert from comment #83)

We have taken note of the Chrome guidance and are drafting a DigiCert CP covering external CAs, until such time as their CP/CPS are published.

(In reply to Rob Stradling from comment #84)

I can't tell if DigiCert was deliberately dodging Q9 or somehow failing to understand it. Either way, I find this failure to answer the actual question very concerning.

DigiCert has "taken note of the Chrome guidance" in comment 82 relating to the "future state of the CCADB Policy" requiring publication of a CP; however, DigiCert has still failed to acknowledge the fact that publishing a CP (either a standalone CP or a combined CP/CPS) was already a MUST requirement in the Chrome Root Program Policy.

Helping DigiCert to understand this fact was the whole point of my Q9, for which DigiCert still has not provided an actual answer.

Q13: Why has DigiCert still not provided an actual answer to Q9?

Q14: Why is DigiCert seeking to close this bug before providing an actual answer to Q9?

Q15: Why did DigiCert seek to close bug 1957499 before providing an actual answer to Q9?

(In reply to DigiCert from comment #91)

Responding to Comment 90, these questions were previously asked and answered in this Bug. Nothing has changed since the last time the questions were asked.

So DigiCert is standing by its claim that a CPS can override a CP. I fundamentally disagree with that position, but it's clear that DigiCert is not going to accept my viewpoint.

I am still keen to hear what chrome-root-program has to say on this matter.

(In reply to DigiCert from comment #89)

DigiCert also created and published a new standalone CP specifically for third-party TLS-issuing Sub-CAs, which is now disclosed for Apple’s Sub-CAs until Apple completes its transition to a combined CP/CPS.

Apple has now published its combined CP/CPS - v7.0, effective 2025-07-01.

DigiCert has disclosed this document to CCADB (several days before its effective date) as the (standalone) CPS for Apple's Sub-CAs, with the "DigiCert CP covering external CAs" still disclosed as the (standalone) CP.

Q16: Can DigiCert explain why it has disclosed the new Apple CP/CPS to CCADB as a CPS, rather than as a CP/CPS?

Q17: Can DigiCert confirm whether or not the "DigiCert CP covering external CAs" is intended to apply to Apple's Sub-CAs going forwards?

Flags: needinfo?(dcbugzillaresponse)

[In response to Comment 93.]

So DigiCert is standing by its claim that a CPS can override a CP. I fundamentally disagree with that position, but it's clear that DigiCert is not going to accept my viewpoint.
I am still keen to hear what chrome-root-program has to say on this matter.

We also disagree with the position that a CPS can override a CP. At the same time, we disagree that DigiCert’s combined CP/CPS was an appropriate choice for the corresponding Apple CAs’ policy disclosure in the CCADB.

Since this bug was opened, Chrome and other CCADB SC members have defined new CCADB logic, held a public discussion period, and landed an update to the CCADB Policy describing that logic to further clarify CCADB Root Store Operator expectations - and to prevent such disclosure violations from taking place in the future. To the best of our abilities, we have attempted to promote positive change that reasonably addresses aspects of the root cause of this incident that are within our control. As the subject of this report, we would have welcomed DigiCert doing the same, and would encourage this type of behavior by CA Owners in response to future incidents going forward - and would consider doing so best practice.

In:

  • Comment 73, DigiCert stated “Apple’s CPS governs their issuance practices, including how they process CAA records. DigiCert’s Combined CP/CPS is applicable at the root level but does not override the Apple CPS in the operation of Sub-CAs.”
  • Comment 76, we stated the circumstances detailed in this incident report describe a violation of Section 2.2 of our program policy concerning the disclosure of PKI policy documentation. We also described our disappointment for DigiCert’s handling of this report.
  • Comments 76 and 82, we expressed and reiterated that we have no indication Apple has violated its CPS at any point in the relevant period beyond one instance already disclosed to this community.
  • Comment 82, we stated that we felt DigiCert’s needlessly rigid view (i.e., “DigiCert's CP/CPS is the CP and it is the controlling document for any certificate issued under the hierarchy of any of our root certificates" has exacerbated this incident and needlessly prolonged the corresponding discussion and resolution. It’s unclear how a direct acknowledgement from DigiCert that indicates this initial view was flawed changes the trajectory of this report or the handling of this incident going forward.
  • Comment 82 also acknowledges DigiCert has not done a satisfactory job responding to clear and direct community questions, including our own - something that was also raised in Comment 60, is the focus of 1957499, and consistently by Rob throughout this report (as done in the preceding comment).

If there are dissenting opinions from other community members, including other root programs, we encourage and welcome their feedback. However, after 9 months of ongoing discussion, we’re planning for this to be our final engagement on this incident report and recommend its closure following a sufficient response from DigiCert to the questions posed in Comment 93.

Flags: needinfo?(chrome-root-program)

Given that this incident occurred during CCADB's Incident Reporting Guidelines 2.0 I'll use that as a reference:

Incidents can happen. Things do not always go as planned, and that can be okay. However, when incidents occur, the underlying issue (i.e., root cause) should be identified and remediated to discourage the incident from occurring again. Formally documenting the incident in a report encourages an understanding of all contributing root cause(s), and it presents the opportunity to clearly communicate a remediation plan to reduce the probability of its reoccurrence.

So we can agree that a CA is expected to make an incident report, and ultimately a plan to stop an issue reoccurring. The documenting of a root cause merely 'encourages an understanding', it does not require it.

The purpose of incident reporting is to help us work together to build a more secure web. Therefore, the incident report should share lessons learned that could be helpful to all CA Owners in building better systems. The incident report should explain how systems or processes failed, how the mis-issuance or incident was made possible, and why the problem was not detected earlier. In addition to the timeline of responding to and resolving the incident, the incident report should explain how the CA Owner's systems or processes will be made more robust, and how others may learn from the incident.

Likewise incidents are created so that other CAs can learn from shared experiences. Unfortunately there isn't really a requirement for the CA who created an incident to actually understand the root cause of their mistakes, merely making sure they're not repeated. Detailing probable root causes, and understanding them are slightly different aspects.

Given the above I'd have to agree that closure would be beneficial if only because the parties capable of improving from this issue have nothing more to learn. If Digicert wishes to ascribe to a policy interpretation that is inconsistent with anyone else's view, then it hardly matters as long as they have systems in place to stop that incorrect view from creating another incident.

While a policy change for incident reports to acknowledge an actual understanding of what caused an issue may help the ecosystem, it'd ultimately be wasting resources. CAs are required to follow and incorporate changes from other CA's incidents, but strictly speaking self-improvement isn't a requirement merely an expectation.

If this were to re-occur without any acknowledgement, then that would be an entirely different matter.

Flags: needinfo?(incident-reporting)
Whiteboard: [ca-compliance] [disclosure-failure] Next update 2025-07-01 → [ca-compliance] [disclosure-failure]

In response to Comment 93:

Q13: Why has DigiCert still not provided an actual answer to Q9?

DigiCert answered Q9 in comment 80. We appreciate the guidance that was provided by the chrome-root-program and CCADB in the course of this Bug, which has helped clarify the policy requirements for both us and the wider community. Clearly, Chrome can make additional requirements for their root store program that go above and beyond the BR, and DigiCert has aligned with the Chrome requirements.

We further acknowledge and appreciate the feedback from the chrome-root-program in Comment 94.

Q14: Why is DigiCert seeking to close this bug before providing an actual answer to Q9?

As covered in our previous responses, DigiCert answered Q9. In the course of this Bug, the CCADB guidance has been clarified and DigiCert aligned with that policy.

Q15: Why did DigiCert seek to close bug 1957499 before providing an actual answer to Q9?

DigiCert answered Q9.

Q16: Can DigiCert explain why it has disclosed the new Apple CP/CPS to CCADB as a CPS, rather than as a CP/CPS?

DigiCert uses the CCADB API to automate document disclosures. Due to duplicate CPS URLs appearing in the AllCertificateRecordsCSVFormatv2 file, we have paused this automation until the issue is resolved, which also took away our associated scanning/reporting process. We have historically and continue to work closely with the CCADB maintainers in improving automation support in this area.

When using the CCADB web interface, we initially selected the CPS field when uploading the Apple CP/CPS (v.7.0). We updated the entry to reflect CP/CPS status starting July 1. We believe this to be correct usage. As noted in your comment, the Apple CP/CPS (v7.0) did not go into effect until July 1, 2025. Until that date, version 6.3.3 of the Apple CPS and the DigiCert CP remained valid for all of DigiCert's Apple SubCAs. That version was still available in CCADB as Doc #3878 (https://ccadb.my.site.com/s/viewnonauditdocumentdetail?c__id=a0RTO0000043wQC2AY&c__rtname=intermediate) and Doc #3886 (https://ccadb.my.site.com/s/viewnonauditdocumentdetail?c__id=a0RTO0000046ZyU2AU&c__rtname=intermediate)

Q17: Can DigiCert confirm whether or not the "DigiCert CP covering external CAs" is intended to apply to Apple's Sub-CAs going forwards?

We can confirm that the Apple combined CP/CPS will cover the Apple Sub-CAs going forward. We will be removing the DigiCert CP covering external CAs as an active document once both Microsoft and Apple are using combined CP/CPS documents.

Flags: needinfo?(dcbugzillaresponse)

In response to Wayne in Comment 95:

Thank you for your feedback. This Bug led to a clarification (for us) of the CCADB policy relating to CP and/or CPS coverage of cross certifications, which we believe will be important in the coming years of faster root rotation.

We note your reference to our use of the API for automated submissions to CCADB, which has at times led to unexpected outcomes. We continue to work with, and are grateful for the assistance of, the CCADB team to improve that facility and our automation with it. While our goal is to eliminate unnecessary manual steps in compliance processes, we’ve temporarily disabled the automated CP and/or CPS submission process pending the resolution of Bug 1971589. We will re-enable once that issue is resolved.

Responding to Comment 94:

Comment 82 also acknowledges DigiCert has not done a satisfactory job responding to clear and direct community questions, including our own - something that was also raised in Comment 60, is the focus of 1957499, and consistently by Rob throughout this report (as done in the preceding comment).

As noted in our [Comment 83)(https://bugzilla.mozilla.org/show_bug.cgi?id=1925106#c83) we are grateful for the clarification and guidance provided by the Chrome and CCADB relating to policy coverage of cross-certifications. We initially believed the process for our cross-signed CAs would be able to update their policy documents more quickly than was in fact possible .. We were hesitant to rush either the creation of a new CP or an underlying CP/CPS at the risk of unintentionally adding noncompliance issues. Our goal has always been to at least meet (if not, surpass) root program and community expectations.

We have posted our closing summary. As we have no additional comments at this time and there have been no additional comments or questions from the community we request that this bug be closed.

This is a final call for comments or questions on this Incident Report.

Otherwise, it will be closed on approximately 2025-07-22.

Flags: needinfo?(incident-reporting)
Whiteboard: [ca-compliance] [disclosure-failure] → [close on 2025-07-22] [ca-compliance] [disclosure-failure]
Status: ASSIGNED → RESOLVED
Closed: 10 months ago
Flags: needinfo?(incident-reporting)
Resolution: --- → FIXED
Whiteboard: [close on 2025-07-22] [ca-compliance] [disclosure-failure] → [ca-compliance] [disclosure-failure]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: