1888689 - Asseco DS / Certum: CRL non-conformance with the TLS BRs

Reporter

Description

•

8 months ago

Issue: Several of Asseco Data Systems S.A’s CRLs disclosed to the CCADB appear in violation of the TLS BRs by extension of violating RFC 5280. Specifically, these CRLs contain the revoked certificates field, however no revoked certificates appear present.

Relevant Policy Language:

RFC 5280: Section 5.1.2.6 states, “When there are no revoked certificates, the revoked certificates list MUST be absent.”
TLS BRs (v2.02): Section 7.2 states, “Effective 2024-03-15, the CA SHALL issue CRLs in accordance with the profile specified in these Requirements. If the CA asserts compliance with these Baseline Requirements, all CRLs that it issues MUST comply with the following CRL profile, which incorporates, and is derived from RFC 5280. Except as explicitly noted, all normative requirements imposed by RFC 5280 shall apply, in addition to the normative requirements imposed by this document.”

How was this detected?: pkilint (and manual investigation). The issue was also corroborated with OpenSSL (example below):

Certificate Revocation List (CRL):
        Version 2 (0x1)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: /C=CM/O=National Agency for Information and Communication Technologies/CN=ANTIC DV CA
        Last Update: Mar 29 15:02:31 2024 GMT
        Next Update: Apr  5 15:02:30 2024 GMT
        CRL extensions:
            X509v3 Authority Key Identifier: 
                keyid:4D:4D:1A:5D:45:92:91:28:55:70:AD:DA:96:57:E6:1F:D6:48:D8:5A

            X509v3 CRL Number: 
                1539735
No Revoked Certificates.
    Signature Algorithm: sha256WithRSAEncryption
         6a:df:05:a0:43:96:6a:6d:b9:70:2c:b7:8f:e0:2f:c1:d4:4f:
         98:96:a7:1a:b6:a5:07:cd:15:6f:e4:02:4d:c5:b2:07:84:67:
         f2:d9:39:37:35:1d:87:3d:b9:89:b0:93:51:03:2d:2d:9c:1d:
         ba:c0:eb:94:b2:cc:e9:5f:c0:89:cf:e5:9e:fe:81:c8:cf:4d:
         13:35:3e:9c:2a:48:be:27:4e:19:95:15:2e:5c:44:de:f0:d9:
         1a:ce:d2:a6:3f:1c:47:48:c2:97:16:f3:1b:b3:81:70:f8:c4:
         56:69:48:67:0f:36:ec:19:bf:eb:e5:c5:64:12:33:fb:97:9f:
         27:cc:af:ce:f7:62:fb:3b:88:ca:ff:60:80:bd:0f:7c:e4:92:
         de:01:64:67:c0:3d:65:d6:c2:a6:5c:07:de:ad:c2:06:54:ff:
         95:18:a9:7f:77:46:31:d9:a8:99:ef:a1:52:ab:1a:3c:10:34:
         1c:4d:5b:01:3d:d7:dc:53:ca:81:63:46:5c:62:46:5f:4c:bc:
         ba:f5:55:73:b2:21:da:e1:e1:80:94:31:35:79:2a:51:29:f6:
         b9:a4:bb:e3:7b:84:67:f3:23:9f:4f:f7:23:7b:c8:b7:b7:2d:
         53:5b:d7:52:e0:23:5d:a6:04:3f:f4:59:9e:e9:1f:cd:05:4e:
         f7:34:ea:fb

And…

{'tbsCertList': {'version': 1, 'signature': {'algorithm': '1.2.840.113549.1.1.11', 'parameters': bytearray(b'\x05\x00')}, 'issuer': ('rdnSequence', [[{'type': '2.5.4.6', 'value': bytearray(b'\x13\x02CM')}], [{'type': '2.5.4.10', 'value': bytearray(b'\x0c>National Agency for Information and Communication Technologies')}], [{'type': '2.5.4.3', 'value': bytearray(b'\x0c\x0bANTIC DV CA')}]]), 'thisUpdate': ('utcTime', datetime.datetime(2024, 3, 28, 23, 2, 31)), 'nextUpdate': ('utcTime', datetime.datetime(2024, 4, 4, 23, 2, 30)), 'revokedCertificates': [], 'crlExtensions': [{'extnID': '2.5.29.35', 'extnValue': b'0\x16\x80\x14MM\x1a]E\x92\x91(Up\xad\xda\x96W\xe6\x1f\xd6H\xd8Z', 'critical': False}, {'extnID': '2.5.29.20', 'extnValue': b'\x02\x03\x17}\xc2', 'critical': False}]}, 'signatureAlgorithm': {'algorithm': '1.2.840.113549.1.1.11', 'parameters': bytearray(b'\x05\x00')}, 'signatureValue': (b'x\x89sg\x1d|\xd7\xdf\xddv4\xfai\xd7%\x91\xd5\xbdm\x87\x9b\xf7\xd5\xe6\x8f\x99\xaf\xa5\xbd\x8b\x8d\x84pCB\xfczUH\xb4\xcbd\xcd\xb2\xa9v\x96\xc5\xed\xe2\xd0t\x9f\xb9*\x93.\x9c\x05\x83\xc6-,\xa8\'vV\xd9\x1ad\x95\r\xe7\xc8\xbd\xbbw]\xe4\xa4\xb4*\x8aE#3\xe2"sa1}\xcd\xf0\x01$\xba\xab\xec\x90y:+-\xd3\x9c5{\x10\xcb\x03\x0bC\xea$\x9eE\x8a\xea\x81[\xa8\xe3X@U\xa6\xe1/\x7f\xe4\r)\xc6\x07/\x80&9\xab*\x02\xad\xb5\xa6\xe3\x16\xd1\xcd\x85,\xc8\xb2\x9a\x8d\xc1\x9c\x1d\x17[m\xf5q\x82h|\xe7\xcf\x98#\xff\x183F!\xd9\xd6\xc3\xe9A\xc8I\xb8\xfa\xdd\xa1\x1e\x89\xa8\xd0\x8e\x16\xabE\xc2"\x07<S\x92\xc1C\x17z\x13\x8e\x8e\x900\xae\'\n%\x02\xc1n\xd5\x98\x04b|\xc8{\xaa\xfa(v\xcb.|\xd3\x1a\x19E\xbb\x86\xd9\xf6\xb5`*HK\xc0=\xe3*Y\x7f\xdbc0\xb3\xd4=\xec', 2048)}

URLs identified as serving CRLs matching this same issue:

Ryan Dickson

Reporter

Updated

•

8 months ago

Summary: Asseco Data Systems S.A. (Certum): CRL non-conformance with the TLS BRs Comment: → Asseco Data Systems S.A. (Certum): CRL non-conformance with the TLS BRs

Ryan Dickson

Reporter

Updated

•

8 months ago

Whiteboard: [crl-failure] [external]

Ben Wilson

Updated

•

8 months ago

Assignee: nobody → kateryna.aleksieieva

Status: NEW → ASSIGNED

Whiteboard: [crl-failure] [external] → [ca-compliance] [crl-failure] [external]

Kateryna Aleksieieva

Assignee

Comment 1

•

8 months ago

Ryan, thank you for bringing this to our attention.

Our team is starting to analyze the problem and is gathering the necessary information. We will submit the incident report by April 12th at the latest.

Kateryna Aleksieieva

Assignee

Comment 2

•

8 months ago

Incident Report

Summary

49 of Asseco Data Systems' CRLs violated TLS BRs (v2.02): Section 7.2 and RFC 5280. These CRLs contained the revoked certificates field, however no revoked certificates were present.

Impact

The problem occurred for any CRL that did not contain a revoked certificate. It affected 49 CRLs in total. No certificates were affected by the error.

Timeline

All times are CEST.

2023-08-17:

TLS BR 2.0.1 was published

2024-03-15:

TLS BR 2.0.1 came into effect

2024-03-29:

17:37 Incident was published

2024-03-30:

9:13 We acknowledged the incident. The compliance team has been informed

2024-04-02:

09:30 We started analysis.

2024-04-03

11:48 Analysis was finished.

2024-04-04

09:12 CRL module fix version was released.

2024-04-04

13:10 The fixed version was installed in the test environment and tested

2024-04-05

11:00 The tests in the test environment were completed and a fixed version was installed in the production environment.

2024-04-05

12:30 New CRLs were generated with correct structure.

Root Cause Analysis

During our testing process, we did not detect that some CRLs disclosed to the CCADB were in violation of the TLS Baseline Requirements (BRs) and, by extension, RFC 5280. Specifically, these CRLs contained the revoked certificates field, but no revoked certificates were present. This was because our testing was done on sample CRLs which did not include empty CRLs.

Our testing approach was to use pkilint on a limited number of samples, since we had configured the same profile for all the CRLs. We did not automate this testing process, but rather performed it manually. We made a wrong assumption that a few positive tests were enough to validate the CRLs. We overlooked the possibility of empty CRLs and did not include them as test scenarios.

Lessons Learned

What went well

The update was quick. The incident was published on 2024-03-29 and all incorrect CRLs were updated by 2024-04-05.

What didn't go well

Error was present in CRL module from the very beginning and was not detected during the testing process.

Where we got lucky

N/A

Action Items

Action Item	Kind	Due Date
CRL module update	Prevent	2024-04-05 (done)
Performed manual tests with pkilint for all CRLs	Detect	2024-04-05 (done)
Include linting for CRLs issuing process	Detect	2024-10-01

Appendix

Details of affected certificates

Kateryna Aleksieieva

Assignee

Comment 3

•

7 months ago

We have no updates on this bug.

amir

Comment 4

•

7 months ago

Six months to add linting to your CRL issuance process is unacceptable.

Not only should have this been here already - the lack of it is already raising questions. But telling the community you’re not going to get them for six months is a non starter.

There have been several bugs on Bugzilla committing to adding linting to their CRL pipeline.

Does Certum have a process for monitoring and triaging Bugzilla bugs for other CAs? If so, why did you wait until this incident to commit to making a plan for linting?

Flags: needinfo?(kateryna.aleksieieva)

Kateryna Aleksieieva

Assignee

Comment 5

•

7 months ago

(In reply to amir from comment #4)

Six months to add linting to your CRL issuance process is unacceptable.

Not only should have this been here already - the lack of it is already raising questions. But telling the community you’re not going to get them for six months is a non starter.

There have been several bugs on Bugzilla committing to adding linting to their CRL pipeline.

Does Certum have a process for monitoring and triaging Bugzilla bugs for other CAs? If so, why did you wait until this incident to commit to making a plan for linting?

Hi Amir, thank you for your questions.

Let me address your last question first. Our compliance team follows a procedure to track current bugs and check their impact on our CA. A good example of this from recent months is bug https://bugzilla.mozilla.org/show_bug.cgi?id=1865080, which we reported based on a bug from another CA.

We agree that linting is a powerful solution that can protect against many errors that may happen to CAs. We have been using zlint for several years now, and this year we were working on implementing pkilint. In results, in March we deployed and integrated pkilint in the TLS certificates issuing process.

As we wrote in the report above, we verified all CRLs manually using the pkilint tool and no error was returned for any of them. Taking into account the fact that changes in CRL profiles and in the application responsible for their generation are extremely rare, we assumed that the risk of recurrence of errors in CRLs that could be detected by pkilint is low. Until we have linting, if we make any changes, we can run manual check using the pkilint tool again.

The date for implementing the linting for CRL is associated with a date we have already committed to implement linting for S/MIME in the bug https://bugzilla.mozilla.org/show_bug.cgi?id=1879845 by October 2024.

Flags: needinfo?(kateryna.aleksieieva)

amir

Comment 6

•

7 months ago

Thanks for the information, I still think that the timeline provided is too late for a publicly trusted CA - but I leave that up to the root programs.

However, for https://bugzilla.mozilla.org/show_bug.cgi?id=1879845, I think these bugs generally remain open until all the action items have been done. Thoughts Ben?

Flags: needinfo?(bwilson)

Ben Wilson

Comment 7

•

7 months ago

I'll leave this bug open until the remediation is complete, and I urge Asseco to implement linting as soon as it can.

Flags: needinfo?(bwilson)

Kateryna Aleksieieva

Assignee

Comment 8

•

7 months ago

We will keep you updated on our progress and I'd like to ask for scheduling the next update for October 1st to avoid weekly updates.

Flags: needinfo?(bwilson)

Ben Wilson

Updated

•

7 months ago

Flags: needinfo?(bwilson)

Whiteboard: [ca-compliance] [crl-failure] [external] → [ca-compliance] [crl-failure] [external] Next update 2024-10-01

Clint Wilson

Updated

•

7 months ago

Summary: Asseco Data Systems S.A. (Certum): CRL non-conformance with the TLS BRs → Asseco DS / Certum: CRL non-conformance with the TLS BRs

Ben Wilson

Comment 9

•

3 months ago

What is the progress of your work to integrate linting into your CRL processes?

Flags: needinfo?(kateryna.aleksieieva)

Kateryna Aleksieieva

Assignee

Comment 10

•

3 months ago

The linting integration is almost complete, we've moved to the testing phase. If everything goes according to plan, we aim to deploy it on September 18th.

Flags: needinfo?(kateryna.aleksieieva)

Kateryna Aleksieieva

Assignee

Comment 11

•

2 months ago

I'm glad to inform, that we have implemented linting for for CRL issuing process.

Action Items

Action Item	Kind	Due Date
Include linting for CRLs issuing process	Detect	2024-09-18 (Completed)

Kateryna Aleksieieva

Assignee

Comment 12

•

2 months ago

We have no updates on this bug. Can it be closed if the community does not have any questions?

Flags: needinfo?(bwilson)

Ben Wilson

Comment 13

•

2 months ago

I intend to close this on or about next Wed. 2-Oct-2024 unless there are concerns from the community.

Ben Wilson

Updated

•

2 months ago

Whiteboard: [ca-compliance] [crl-failure] [external] Next update 2024-10-01 → [ca-compliance] [crl-failure] [external]

Ben Wilson

Updated

•

2 months ago

Status: ASSIGNED → RESOLVED

Closed: 2 months ago

Flags: needinfo?(bwilson)

Resolution: --- → FIXED