1882256 - AGCE: Non-Compliant VPN Certificate Issuance

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[ance.certification.info]

Incident Report

Summary

The AGCE RA function notified CA stakeholders about the issuance of a non-compliant SSL/VPN certificate, the certificate had two unsupported attributes “unstructured name” and “unstructured address” and an IP address in the dNSName field under the SAN

Impact

One non-compliant SSL/VPN certificate is issued on February 4, 2024, 13:34 UTC+1

Timeline

February 6, 2024, 11:30 UTC+1 – The Incident was raised by the AGCE RA function to CA stakeholders, that a certificate was issued on February 4, 2024, 13:34 UTC+1 with two issues:

1. Two unsupported attributes “Unstructured Name” and “Unstructured Address”, and
2. An IP address SAN value under dNSName field.
   AGCE operations team start an internal investigation.
   February 6, 2024, 14:09 UTC+1 – AGCE confirms that the certificate was indeed misissued.
   February 6, 2024, 15:12 UTC+1 – AGCE notifies the subscriber informing them of the fact their certificate was misissued and is scheduled for revocation.
   February 6, 2024, 16:26 UTC+1 – The affected certificate is revoked.

Root Cause Analysis

The issued certificate had two issues as follow:

1. Two unsupported attributes “Unstructured Name” and “Unstructured Address”: as per our initial investigation this incident happened due to our RA system not supporting the two attributes in a way that they were not even displayed to the RA officer during the vetting process, this resulted in the successful vetting of the certificate request as no anomalies were detected by the RA Officer.
2. An IP address in the dNSName field under the SAN: this happened during the vetting process where the RA system only displayed one SAN field which is the dNSName with an IPaddress value, this vetting step is done manually by the RA Officer, due to a human error not noticing the IPaddress being under the DnsName field instead of the IPaddress field.

Lessons Learned

What went well

Timely revocation of the certificate by the CA.

What didn't go well

Our RA system did not show that the uploaded CSR contained unsupported attributes which made it seem valid for the RA officer during the vetting process.
Our RA officer did not detect during the vetting process that the uploaded CSR was not created properly as the dNSName field contained an IP address value.

Where we got lucky

The AGCE through the RA function maintain direct continuous human communications with the subscribers via phone calls or email. This allowed the RA officers to quickly detect the incident by checking the contents of certificates shortly after they are issued.

Action Items

| RA Officers have received the appropriate awareness as a lessons learned communication instructing them to be more vigilant when reviewing SSL/VPN certificate requests | Prevent | 12 February 2024 |
| AGCE Policy function is actively working on enhancing its RA processes to enforce the certificate vetting processes | Prevent | End of April 2024 |
| Technically the operational team is working with our partners to enhance the RA application | Prevent | End of March 2024 |
| The AGCE RA Officer will also communicate with subscribers on the importance of submitting CSRs that comply with our CPSs| Prevent | 12 February 2024 |

Appendix

Details of affected certificates

https://crt.sh/?sha256=[sha256 fingerprint of the certificate]

Based on Incident Reporting Template v. 2.0

Comment 1

[Dustin Hollenback]

Hello,

I have some initial questions/comments:

• Why did it take 21 days to file this bug?

• In the "What didn't go well" section, the details imply that all verification only occurs by the RA officer. I'm assuming the RA officer is a person. If I assume that correctly, are there any automated checks in place that do not rely on a human and if not, why not?

• Can you elaborate on this Action Item: "AGCE Policy function is actively working on enhancing its RA processes to enforce the certificate vetting processes"? What specific enhancements or enforcements are you committing to implement?

• Can you elaborate on this Action Item: "Technically the operational team is working with our partners to enhance the RA application"? What enhancements are you planning to implement?

• I have a comment about this Action Item: "The AGCE RA Officer will also communicate with subscribers on the importance of submitting CSRs that comply with our CPSs." From my experience, relying on subscribers to submit compliant CSRs is a futile exercise. This is why it is so important for the Certification Authority to have automated checks in place that perform validations/linting to capture problems from poor CSRs before signing with a publicly trusted key.

• Do you use any publicly available linting tools such as certlint or zlint?

• The Details of affected certificates has a placeholder instead of the real certificate link. Can you provide a direct crt.sh link to the certificate?

Thank you,

Dustin

Comment 2

[Malcolm D]

Will you also be checking your corpus of issued certificates to look for similar misissuances?

Comment 3

[ance.certification.info]

Hello Mr Dustin Hollenback,
Please find below the responses to your questions and comments:

"1.Why did it take 21 days to file this bug?
-We had to transfer the ownership of the Bugzilla reporter account due a role change which took a few days to sort out the access.

2. In the "What didn't go well" section, the details imply that all verification only occurs by the RA officer. I'm assuming the RA officer is a person. If I assume that correctly, are there any automated checks in place that do not rely on a human and if not, why not?
   -Our RA vetting processes are comprised of human (RA officers) and machine (RA system) verification levels. RA officers handle the verification of SDN information accuracy and correctness as well as domain ownership, while the RA system handles the verification of the CSR’s technical properties (verifying the private key ownership, verifying the public key contains valid public exponent and modules, verifying the public key is not already used, verifying the key length is allowed by the CA, verifying the debian weak keys are not used).

3. Can you elaborate on this Action Item: "AGCE Policy function is actively working on enhancing its RA processes to enforce the certificate vetting processes"? What specific enhancements or enforcements are you committing to implement?
   -We will introduce a new level of validation for the RA officer based on the use of separated tools than the deployed RA system to dump the CSR file and detect any anomalies.

4. Can you elaborate on this Action Item: "Technically the operational team is working with our partners to enhance the RA application"? What enhancements are you planning to implement?
   -This comprises two enhancements: the RA system to automatically reject CSRs with non-supported attributes, and to verify the correspondence of the CSR fields and the value set in them.

5. I have a comment about this Action Item: "The AGCE RA Officer will also communicate with subscribers on the importance of submitting CSRs that comply with our CPSs." From my experience, relying on subscribers to submit compliant CSRs is a futile exercise. This is why it is so important for the Certification Authority to have automated checks in place that perform validations/linting to capture problems from poor CSRs before signing with a publicly trusted key.
   -The action items above aim to indeed enforce the validation by both enhancing the human and automated levels.

6. Do you use any publicly available linting tools such as certlint or zlint?
   -RA Officers are familiar with these tools but they are not used as part of a process.

7. The Details of affected certificates has a placeholder instead of the real certificate link. Can you provide a direct crt.sh link to the certificate?
   -Our subordinate CA hasn’t integrated certificate transparency and thus certificates are not available on crt.sh"

Cordially

Comment 4

[ance.certification.info]

(In reply to Malcolm D from comment #2)

Will you also be checking your corpus of issued certificates to look for similar misissuances?

Hello,

Our subordinate CA in question delivers certificates to a limited number of subscribers with which they maintain constant and direct communication. Other Issued certificates other than this one were verified and were not misissued.

Cordially

Comment 5

[Ben Wilson]

Can you upload the revoked certificate here as an attachment?

Comment 6

[ance.certification.info]

Attachment: the revoked vpn certificate

Hello Mr Ben Willson,
please find attached the revoked vpn certificate
Cordially

Comment 7

[Rob Stradling]

You can also find this certificate at https://crt.sh/?id=12217695455.

Comment 8

[Seo Suchan]

BTW running other certificate on CT from that CA though zlint warns lifetime over 397 days: But this CA doesn't have chain to public root store so literally anything can parsed works I guess

Comment 9

[Andrew Ayer]

This CA is applying for inclusion in Bug 1736904 so reports of additional noncompliance are helpful.

Comment 10

[Seo Suchan]

(In reply to Andrew Ayer from comment #9)

This CA is applying for inclusion in Bug 1736904 so reports of additional noncompliance are helpful.

https://crt.sh/?id=10000096404&opt=zlint
and certificate this thread mentions (https://crt.sh/?id=12217695455) also has longer than 397 days (by a second like 1715455)

Comment 11

[Ben Wilson]

I believe this matter can be closed, so I close it on or about Wed. 28-Aug-2024 unless there are questions or issues to be discussed.