1886624 - certSIGN: Certificates with incorrect Subject attribute order

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Gabriel PETCU]

Attachment: List-certSIGN-serials-bad-order-20.03.2024.txt

Incident Report

Summary

certSIGN has issued 625 TLS certficates, since September 15 2023, with an incorrect relative order of Subject attributes, as defined in BR section 7.1.4.2.

Impact

Immediately after discovering the incorrect Subject attribute order in a set of issued certificates, on March 18, 2024, certSIGN stopped the issuance of all TLS certificates.

The Subject attribute order was then corrected and the issuance restarted.

certSIGN will revoke all affected certificates.

Timeline

All times are UTC.

2024-03-18:

• 13:07 an email was received with a Certificate Problem Report about a non-conformity on the Subject Attribute Encoding order
• 16:00 an analysis of the cause of the non-conformity started
• 16:10 the third party informing certSIGN on the Certificate Problem Report was acknowledged
• 16:30 certSIGN stopped the issuance of TLS certificates
• 16:30 a search started on all certificates issued since the effective date of the CA/Browser Forum version 2.0.0, that is September 15, 2023.
• 17:00 the affected main Subscribers were identified, and certSIGN started to notify them

2024-03-19:

• 17:30 the search of the certificates issued was finalized

2024-03-20:

• 08:00 certSIGN informed the WebTrust auditors about the incident
• 14:00 the new update was deployed in Production
• 15:30 certSIGN restarted the issuance of TLS certificates
• 22:30 the incident report was registered in Bugzilla

Root Cause Analysis

Our linter software was not catching the incorrect order due to a misconfiguration.

Lessons Learned

What went well

• Immediate actions were taken when the error was identified

What didn't go well

• The configuration of the certSIGN linter software

Where we got lucky

• N/A

Action Items

| Action Item | Kind | Due Date | Status |

| Check all certificates issued since September 15, 2023 | Analyze | 2024-03-19 | Done |

| Fix the linter configuration, test and deploy | Correct | 2024-03-20 | Done |

| Update of the software for issuing certificates in order to fix the correct order of the attributes | Correct | 2024-03-20 | Done |

| Revoke all the affected certificates| Correct | 2024-03-23 | In progress |

| Review the software update testing process | Prevent | 2024-03-26 | In progress |

Appendix

Details of affected certificates

In the attachment

Based on Incident Reporting Template v. 2.0

Comment 1

[Gabriel PETCU]

We revoked and reissued all affected certificates.

Comment 2

[Gabriel PETCU]

The software update testing process had been reviewed on 26.03.2024. The process is correct but the steps had not all been followed. An extra check was added in the internal procedure for the validator to explicitly verify the configuration files
All the action items are closed now.

Comment 3

[Gabriel PETCU]

We have no additional actions and consider the bug resolved unless there are further questions.

Comment 4

[Gabriel PETCU]

We have no additional actions and consider the bug resolved unless there are further questions.

Comment 5

[Ben Wilson]

Hi Gabriel,
Can you address what actions have been taken or will be taken to ensure that certSIGN linter(s) will be up-to-date when future certificate profile requirements become effective? Some categories of actions might be: continual monitoring of CA/B Forum ballots/guideline versions (and scheduling linter updates accordingly), regular periodic review of your linter(s) and performing code review of other linters, and monitoring Bugzilla for incidents involving mis-issuance by other CAs.
Thanks,
Ben

Comment 6

[Gabriel PETCU]

certSIGN continuously monitor CA/B Forum ballots/guideline versions and it is scheduling updates on the changes, but these will be improved through a periodic checking of the status of the tickets allocated for each change, at least every month, by the PKI Policies Manager. This improvement is added to the operational procedure.
The certSIGN responsible for the linters already have a period task of reviewing the linter's performance and fulfilment of the requirements, but this will be supplemented with the internal auditor review of linters used in certSIGN, on a quarterly basis.

Comment 7

[Ben Wilson]

I intend to close this bug next Wed. 2024-06-05, unless there are additional comments or questions.