1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.
Apple received a notification from our root vendor (DigiCert) of intermediate certificate records in the Common CA Database (CCADB) that have outdated audit statements on August 03, 2021 at 10:23 PT
The notification prompted a review of Apple’s intermediate CA records, which resulted in identifying 3 EV Sub-CAs (below) that were not listed on the recently issued “WebTrust for Certification Authorities – SSL Baseline with Network Security” (WTBR) audit statement, though they were listed on the “WebTrust Principles and Criteria for Certification Authorities – Extended Validation SSL” (WTEV) and the “WebTrust Principles and Criteria for Certification Authorities” (WTCA) audit statements:
- Certificate Name: Apple Public EV Server ECC CA 1 - G1
SHA-256 Fingerprint: 2585928D2C5BFD952E025BD12E27C6776224CF752EC362D3031CDD49351844D4
- Certificate Name: Apple Public EV Server RSA CA 1 - G1
SHA-256 Fingerprint: 340CA5BA402D140B65A2C976E7AE8128A1505C29D190E0E034F59CCAE7A92BC2
- Certificate Name: Apple Public EV Server RSA CA 2 - G1
SHA-256 Fingerprint: D6EF3E09EBE0D9370E51F5C09A532B3AC70D3CE822253F9FC84C28E9BFA550D5
Apple verified with our external WebTrust auditors that this was an unintended error of omission on the WTBR audit statement, that the completed audits correctly covered these omitted CAs, and that a corrected audit statement would be issued
2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.
- 2020-04-16: Audit Period Start Date
- 2021-04-15: Audit Period End Date
- 2021-06-09: Audit Reports finalized and signed
- 2021-06-09: Completed Audit Reports sent to CPA Canada
- 2021-06-15: Seal ID URLs received from CPA Canada
- 2021-07-01: Updated Seals and Audit Reports published to https://www.apple.com/certificateauthority/
- 2021-08-03 10:23 PT: Apple received notification of intermediate certificate records in the Common CA Database (CCADB) that have outdated audit statements
- 2021-08-03 11:03 PT: Apple communicated with the root vendor to perform analysis of the discovered issue
- 2021-08-03 13:43 PT: Apple verified that 3 EV Sub-CAs were not listed in the published WTBR audit statement, resulting in the above notification
- 2021-08-03 13:50 PT: Apple notified the external auditors of the discrepancy in the WTBR audit statement
- 2021-08-03 15:45 PT: Apple received verification from the external auditors that this omission was unintended and did not accurately represent the work completed in the audit; the 3 EV Sub-CAs were correctly scoped in to the WTBR audit and the audit statement was incorrect in not having these 3 EV Sub-CAs listed
- 2021-08-06: Received amended audit statements and distributed them to Mozilla and root vendor
3. Whether your CA has stopped, or has not yet stopped, certificate issuance or the process giving rise to the problem or incident. A statement that you have stopped will be considered a pledge to the community; a statement that you have not stopped requires an explanation.
Certificate issuance was not affected in this incident
4. In a case involving certificates, a summary of the problematic certificates. For each problem: the number of certificates, and the date the first and last certificates with that problem were issued. In other incidents that do not involve enumerating the affected certificates (e.g. OCSP failures, audit findings, delayed responses, etc.), please provide other similar statistics, aggregates, and a summary for each type of problem identified. This will help us measure the severity of each problem.
Three (3) EV Sub-CAs were erroneously omitted from our recently issued “WebTrust for Certification Authorities – SSL Baseline with Network Security” (WTBR) audit statement
5. In a case involving certificates, the complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem. In other cases not involving a review of affected certificates, please provide other similar, relevant specifics, if any.
6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.
- The WTBR audit was completed in the expected timeline with the correct scope of included Sub-CAs and its audit statement was published in a timely manner. The discovery of the erroneously omitted EV Sub-CAs was made from the CCADB report generated by Mozilla and shared with the root vendor
- Reviews of the published audit statements were manual and relied on visual inspection and comparison of the included Sub-CAs. This approach was prone to human error which is what occurred in this instance; notably, despite multiple independent people performing that review, this error was not caught
7. List of steps your CA is taking to resolve the situation and ensure that such situation or incident will not be repeated in the future, accompanied with a binding timeline of when your CA expects to accomplish each of these remediation steps.