1700809 - Microsoft PKI Services: Failure to disclose Unconstrained Intermediate within 7 Days

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[John Mason]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4429.0 Safari/537.36 Edg/90.0.818.0

Comment 1

[John Mason]

Attachment: microsoft ecc tls issuing eoc ca 02.crt

Comment 2

[John Mason]

Attachment: microsoft ecc tls issuing eoc ca 01.crt

Comment 3

[John Mason]

Attachment: microsoft ecc tls issuing aoc ca 01.crt

Comment 4

[John Mason]

Attachment: microsoft ecc tls issuing aoc ca 02.crt

Comment 5

[John Mason]

Attachment: microsoft rsa tls issuing aoc ca 01.crt

Comment 6

[John Mason]

Attachment: microsoft rsa tls issuing aoc ca 02.crt

Comment 7

[John Mason]

Attachment: microsoft rsa tls issuing eoc ca 01.crt

Comment 8

[John Mason]

Attachment: microsoft rsa tls issuing eoc ca 02.crt

Comment 9

[John Mason]

1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.

This was first reported to Microsoft by DigiCert at 02:36 pm Pacific time on Wednesday, 2021-03-24 via email informing us that two of our new Issuing CAs were reported in the “Unconstrained Trust: Disclosure is required!” report: https://crt.sh/mozilla-disclosures.

2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.

Note: Times are listed in Pacific time zone
• 24 March 2021 02:36 PM – Issue reported by DigiCert as an email
• 24 March 2021 02:39 PM – Internal investigation started
• 24 March 2021 03:30 PM – Confirmed that this is related to a recent deployment of new Issuing CAs
• 24 March 2021 05:15 PM – CCADB updated with the 8 newly created CA certificates (attached to this bug)

3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.

No subscriber certificates have been issued and there are no plans to issue subscriber certificates until an audit seal has been obtained. However, we have issued OCSP and Sample Certs off of the Microsoft RSA TLS Issuing AOC CA 01 and Microsoft ECC TLS Issuing CA 01. This was a process problem for a new Issuing CA certificate and we did not meet the disclosure timeline requirement from Section 5.3.2 of the Mozilla Root Store Policy. These 8 CA's were created on March 11 and added to CCADB on March 24 (13 days).

4. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.

The 8 new TLS issuing CA's are all attached to this bug.

Two of them are in CRT.SH already:

• 03/11/2021 Microsoft RSA TLS Issuing AOC CA 01 https://crt.sh/?sha256=13e295a5a6ecaf8391126e752b131b316675d187266bcf537892f839454b9480
• 03/11/2021 Microsoft ECC TLS Issuing AOC CA 01 https://crt.sh/?sha256=9d8bb150c07b81c25799b7945978b036e4b49c593c882be379a58d08e1c25074

The Other 6 are:

• 03/11/2021 Microsoft RSA TLS Issuing AOC CA 02 563c935011b3aa7638da69817a964245f8a6ba7c
• 03/11/2021 Microsoft RSA TLS Issuing EOC CA 01 0ee6aa1f759e18509f1edd02da49f1f212cbf8b2
• 03/11/2021 Microsoft RSA TLS Issuing EOC CA 02 a922d91efa12a71b7aeddb9665e80da6f846d875
• 03/11/2021 Microsoft ECC TLS Issuing AOC CA 02 f1cb1990f32697e1e90cb05975224ba78f191132
• 03/11/2021 Microsoft ECC TLS Issuing EOC CA 01 aec01b8bf645cad77f3be39ad1f01d928d52ceb8
• 03/11/2021 Microsoft ECC TLS Issuing EOC CA 02 f9593f17cade86c383e92ba9f38302018bf26724

5. The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem.

See above and or attached for the list of new Issuing CA details.

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

This was a process issue. It was the first time that we have deployed new Issuing CAs since being added to the Mozilla root programs and we mistakenly had the disclosure task set later in the project, which didn’t allow us to meet the 7 day disclosure requirement.

7. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.

We have been unable to determine an automated way to solve this problem. We have updated our ceremony documentation and moved the CCADB update (disclosure) immediately after the key generation ceremony with a note that it must be updated within 7 days.

Comment 10

[Ryan Sleevi]

This was a process issue. It was the first time that we have deployed new Issuing CAs since being added to the Mozilla root programs and we mistakenly had the disclosure task set later in the project, which didn’t allow us to meet the 7 day disclosure requirement.

I'm hoping you can provide more thorough details here. It's not clear exactly what the process issue was. This makes it difficult to properly assess the root cause; the answer to Question 7 gives a proximate proposed fix for the immediate issue, but I think it's useful to understand a bit more about the process issues.

Note that CCADB does support automated disclosures, as DigiCert themselves sponsored the API work. Given Microsoft's existing relationship, is this something you've also discussed with DigiCert?

Comment 11

[John Mason]

Our process was setup to post the CA’s to CCADB in concert with the Audit Letters that would initially include said CA’s in them (updating CCADB with the Audit Letters and the new CA’s at the same time). This is the process that we had followed prior to acceptance in the Mozilla Root Program.

With the telemetry from the “Unconstrained Trust” report we learned of our nonconformance and reported it via this bug. We also became familiar with many similar cases like it and have taken those into account in updating our processes.

As our answer to Question 7 states, we have changed our process to ensure we post the CA’s to CCADB as a part of completing the CA ceremony and documentation. This will allow us to meet the 7 day requirement.

In addition, we will work with Digicert to understand more about the API work they have done, and we intend to automate this process in the future.

Comment 12

[John Mason]

We have worked with both Digicert and Mozilla to gather more information about an API that has been developed to automate posting to CCADB. Here is the link for more information, https://github.com/mozilla/CCADB-Tools/tree/master/API_AddUpdateIntermediateCert .

We have updated our backlog with the information to add this automated API to our systems in the future. However, we believe the process improvements above will allow us to meet the requirements as implemented.

We ask that this issue be resolved, unless there are any further questions.

Comment 13

[Ben Wilson]

I'll close this on or about 30-June-2021 unless there are further questions.

Comment 14

[Dimitris Zacharopoulos]

@John Mason: You might find this information useful https://github.com/HARICA-official/ccadb-ca-tools. It would also be great if you could publish similar code that would help other CAs automate their intermediate CA Certificates disclosure processes.

Comment 15

[John Mason]

Thank you Dimitris this is appreciated and I have added the reference to our documentation. We would love to help provide similar resources to the community as we develop our processes to automate this process.