2004521 - TWCA: CA Certificate not published in DER Encoded Format

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[chtsai]

Preliminary Incident Report

Summary

• Incident description:
  The TWCA CYBER Root CA's SubCA has a root CA download link (http://sslserver.twca.com.tw/cacert/cyber_root_2022.crt) in its AIA field, and its certificate is PEM encoded, which is not the DER encoding required by RFC 5280.

• Relevant policies:
  RFC 5280 Section 4.2.2.1

• Source of incident disclosure:
  Third Party Reported

Comment 1

[chtsai]

Full Incident Report

Summary

• CA Owner CCADB unique ID: A000054
• Incident description:
  • The TWCA CYBER Root CA's SubCAs have a root CA download link (http://sslserver.twca.com.tw/cacert/cyber_root_2022.crt) in its AIA field, and its certificate is PEM encoded, which is not the DER encoding required by RFC 5280.
• Timeline summary:
  • Non-compliance start date: 2023-02-23
  • Non-compliance identified date: 2025-12-06
  • Non-compliance end date: 2025-12-06
• Relevant policies:
  • RFC 5280 Section 4.2.2.1
• Source of incident disclosure: Third Party Reported

Impact

• Total number of certificates: N/A

• Total number of "remaining valid" certificates: N/A

• Affected certificate types: N/A

• Incident heuristic:

  • The potential impact of this incident may affect application clients that perform operations by utilizing the caIssuers within the certificate's authorityInfoAccess information.

  • The TWCA CYBER Root CA has currently issued a total of 4 SubCAs, which are as follows:

    crt.sh ID	Not Before	Not After	Subject Name
    14487601692	2024-09-06	2034-09-06	C=TW, O=TAIWAN-CA, CN=TWCA EVSSL Certification Authority
    14487601691	2024-09-06	2034-09-06	C=TW, O=TAIWAN-CA, CN=TWCA SSL Certification Authority
    9946963269	2023-02-23	2033-02-23	C=TW, O=TAIWAN-CA, OU=EVSSL Sub-CA, CN=TWCA EVSSL Certification Authority
    10256433881	2023-02-23	2033-02-23	C=TW, O=TAIWAN-CA, OU=SSL Sub-CA, CN=TWCA SSL Certification Authority

  • Since the AIA of these SubCAs all point to the same TWCA CYBER Root CA path, the potential scope of impact includes these SubCAs and all the TLS certificates they have issued.

• Was issuance stopped in response to this incident, and why or why not?:
  We have not stopped issuing any certificates because these certificates themselves do not have a format error; rather, the error lies in the format of the remote file they point to.

• Analysis: N/A

• Additional considerations: N/A

Timeline

All times are UTC+8.

• 2025-12-06 09:15: Received a notification from a third party that the CA certificate encoding format was incorrect.
• 2025-12-06 13:23: Identified the issue, confirmed it was valid, and escalated it internally.
• 2025-12-06 13:50: Preliminary clarification determined the scope is limited to the incorrect certificate format at the TWCA CYBER Root CA certificate download location.
• 2025-12-06 14:30: The Preliminary Incident Report was disclosed on Bugzilla.
• 2025-12-06 14:36: Operations personnel were informed of the issue status and scope.
• 2025-12-06 15:03: The file replacement was completed and verified to meet expectations.

---

Updates will continue.

Comment 2

[chtsai]

Full Incident Report

Summary

• CA Owner CCADB unique ID: A000054
• Incident description:
  • The TWCA CYBER Root CA's SubCAs have a root CA download link (http://sslserver.twca.com.tw/cacert/cyber_root_2022.crt) in its AIA field, and its certificate is PEM encoded, which is not the DER encoding required by RFC 5280.
• Timeline summary:
  • Non-compliance start date: 2023-02-23
  • Non-compliance identified date: 2025-12-06
  • Non-compliance end date: 2025-12-06
• Relevant policies:
  • RFC 5280 Section 4.2.2.1
• Source of incident disclosure: Third Party Reported

Impact

• Total number of certificates: N/A

• Total number of "remaining valid" certificates: N/A

• Affected certificate types: N/A

• Incident heuristic:

  • The potential impact of this incident may affect application clients that perform operations by utilizing the caIssuers within the certificate's authorityInfoAccess information.

  • The TWCA CYBER Root CA has currently issued a total of 4 SubCAs, which are as follows:

    crt.sh ID	Not Before	Not After	Subject Name
    14487601692	2024-09-06	2034-09-06	C=TW, O=TAIWAN-CA, CN=TWCA EVSSL Certification Authority
    14487601691	2024-09-06	2034-09-06	C=TW, O=TAIWAN-CA, CN=TWCA SSL Certification Authority
    9946963269	2023-02-23	2033-02-23	C=TW, O=TAIWAN-CA, OU=EVSSL Sub-CA, CN=TWCA EVSSL Certification Authority
    10256433881	2023-02-23	2033-02-23	C=TW, O=TAIWAN-CA, OU=SSL Sub-CA, CN=TWCA SSL Certification Authority

  • Since the AIA of these SubCAs all point to the same TWCA CYBER Root CA path, the potential scope of impact includes these SubCAs and all the TLS certificates they have issued.

• Was issuance stopped in response to this incident, and why or why not?:

  • We have not stopped issuing any certificates because these certificates themselves do not have a format error; rather, the error lies in the format of the remote file they point to.

• Analysis: N/A

• Additional considerations: N/A

Timeline

All times are UTC+8.

• 2024-09-02: Inspired by Incident 1914466, TWCA's automated verification tool(AutoWorker) has been updated to include a check for the AIA in subscriber certificates. This check includes validating the certificate format, but the scope does not cover CA certificates.
• 2025-02-23: TWCA CYBER Root CA issues two SubCAs, which are used for SSL and EV SSL certificates, respectively.
• 2025-09-06: TWCA CYBER Root CA issues two SubCAs, which are used for SSL and EV SSL certificates, respectively.
• 2025-12-06:
  • 09:15: Received a notification from a third party that the CA certificate encoding format was incorrect.
  • 13:23: Identified the issue, confirmed it was valid, and escalated it internally.
  • 13:50: Preliminary clarification determined the scope is limited to the incorrect certificate format at the TWCA CYBER Root CA certificate download location.
  • 14:30: The Preliminary Incident Report was disclosed on Bugzilla.
  • 14:36: Operations personnel were informed of the issue status and scope.
  • 15:03: The file replacement was completed and verified to meet expectations.
• 2025-12-08 12:43: Completed check of all CA files; no encoding errors found.
• 2025-12-08 18:21: Completed check of all CRL files; no encoding errors found.

Related Incidents

Bug	Date	Description
Bug 1884461	2024-03-08	CA Certificate not published in DER Encoded Format
Bug 1914466	2024-08-22	CA Certificate not published in DER Encoded Format
Bug 1914893	2024-08-26	CRL not published in DER Encoded Format
Bug 1938167	2024-12-18	CRL not published in DER Encoded Format
Bug 2004492	2025-12-05	CA Certificate not published in DER Encoded Format

Root Cause Analysis

Contributing Factor #1: Lack of Comprehensive File Placement SOP

• Description:
  • The SOP for the CA certificate issuance process lacks a check on the file encoding/format for externally downloadable files.
    • We have SOPs for CA certificate issuance and for setting subscriber certificate profiles, both containing checklists for each step, but there has never been a checklist item for the certificate download file encoding.
• Timeline:
  • The problem has existed all along.
• Detection:
  • Notified by a third party.
• Interaction with other factors: 
  • Files are currently placed by experienced operations personnel (who are aware of the encoding requirements). Without an SOP checklist or other automated assistance, this leads to an uncontrollable outcome (encoding may or may not be correct).
  • The CRL is published automatically by the system with no manual intervention, and the publishing system's encoding is correct.

Contributing Factor #2: Incomplete Coverage of Automated Verification Tool

• Description:
  • Our self-developed automated verification tool (hereinafter referred to as AutoWorker) only checks the AIA in Subscriber Certificates, and does not check the AIA in CA Certificates, thus achieving only partial verification.
• Timeline:
  • 2024-01-30: AutoWorker v1.0.0.0 initial version created and deployed.
  • 2024-09-02: AutoWorker v6.0.0.0 added AIA and file encoding checks (Test Case No. 41).
  • 2025-12-03: AutoWorker v14.28.0.0 released (version at the time of this incident).
• Detection:
  • After this incident, we examined the AutoWorker configuration file and found that the configuration was incomplete, as the check scope did not cover all file download locations.
• Interaction with other factors:
  • The person who wrote the tool lacked sufficient understanding or had ambiguous requirements, believing that checking the subscriber certificate profile was enough and neglecting the CA certificate component.

Contributing Factor #3: Lack of Synchronization Mechanism with Automated Verification Tool

• Description:
  • Currently, there is no procedure or mechanism to notify AutoWorker to adjust its configuration file in response to the addition or deletion of certificate download files.
        - Certificate operations and AutoWorker run independently of each other.
• Timeline: 
  • Since the initial tool release on 2024-01-30.
• Detection:
  • Discovered during the post-mortem review.
• Interaction with other factors:
  • Related to Contributing Factor #1: The SOPs for CA certificate issuance or subscriber certificate profile changes lack a synchronization procedure.

Lessons Learned

• What went well:
  • Although the CPR was received on a holiday, the robust alert mechanism ensured that the incident handling was not significantly delayed.
  • Regular weekly tracking of Bugzilla incidents and self-review were implemented, and encoding checks were integrated into the automated verification mechanism, thus limiting the scope of impact.
• What didn’t go well:
  • Certificate operation-related SOPs are still incomplete.
  • The person who wrote the AutoWorker lacked sufficient understanding or had ambiguous requirements, resulting in insufficient coverage of the verification scope.
  • A lack of synchronization mechanism between operations and AutoWorker.
• Where we got lucky: N/A 
• Additional:
  • We have made the AutoWorker source code publicly available on GitHub for community reference, all sensitive information in the project has been removed or masked.

Action Items

Action Item	Kind	Corresponding Root Cause(s)	Evaluation Criteria	Due Date	Status
Fix the known files with encoding errors	Mitigate	Root Cause # 1	Manually download and check files	2025-12-06	Complete
Manually check all CA certificate download locations for correct file encoding	Mitigate	Root Cause # 1	Manually download and check files	2025-12-08	Complete
Manually check all CRL download locations for correct file encoding	Mitigate	Root Cause # 1	Manually download and check files	2025-12-08	Complete
Adjust AutoWorker configuration file to cover all verification checklists	Detect	Root Cause # 2	Tool version update	2025-12-12	Complete
Add certificate file encoding checks to Certificate Operation SOPs	Prevent	Root Cause # 1	SOP document version update	2026-01-31	Ongoing
Add an item to Certificate Operation SOPs to notify AutoWorker of changes	Prevent	Root Cause # 3	SOP document version update	2026-01-31	Ongoing

Appendix

N/A

Comment 3

[chtsai]

Action Items are currently in progress. We request the next update to be set for 2026/1/15. Thank you.

Comment 4

[Dustin Hollenback (Apple)]

Hello. I noticed that this bug is Third‑Party Reported. Since similar issues were disclosed by several other CAs last year, it’s unclear whether you are monitoring incident reports from the broader community. Public disclosures are intended to help all CA operators learn from each other’s findings.

Could you confirm whether you review other CAs’ incident reports as part of your internal processes? If so, a brief overview of how you track these reports and incorporate relevant lessons would be helpful. It would also be useful to understand how this issue was ultimately identified through a third‑party report rather than your internal monitoring.

Comment 5

[chtsai]

(In reply to Dustin Hollenback (Apple) from comment #4)

Hello. I noticed that this bug is Third‑Party Reported. Since similar issues were disclosed by several other CAs last year, it’s unclear whether you are monitoring incident reports from the broader community. Public disclosures are intended to help all CA operators learn from each other’s findings.

Could you confirm whether you review other CAs’ incident reports as part of your internal processes? If so, a brief overview of how you track these reports and incorporate relevant lessons would be helpful. It would also be useful to understand how this issue was ultimately identified through a third‑party report rather than your internal monitoring.

Thank you for your feedback.

We have established mechanisms and processes for monitoring community activities, which have been formalized and documented as part of TWCA's internal ISMS. This documentation outlines the scope, methods, and specific procedures for tracking such incidents (as this is an internal document, we will provide it to you privately).

As described in the timeline, following a similar incident on September 2, 2024, we performed an internal analysis and integrated new check items into our automated validation tools. However, as detailed in "Factor #2," our validation scope was not comprehensive enough, which resulted in our inability to detect this specific error internally.

As noted in "Factor #3," we recognize that there is still room for improvement. Ultimately, an incident has occurred, and we acknowledge that certain synchronization mechanisms remain incomplete or suboptimal.

Comment 6

[Dustin Hollenback (Apple)]

I appreciate the extra detail. Thank you. That answered my questions.

Comment 7

[Sean Huang (TWCA)]

We have incorporated two key work items—A. Certificate Encoding Validation and B. Notification Automation Tool Monitoring—into our Certificate Operational SOP (an internal ISMS control document). Accordingly, we have marked the final two action items as 'Complete.' All action items have now been finalized.

Action Items

Action Item	Kind	Corresponding Root Cause(s)	Evaluation Criteria	Due Date	Status
Fix the known files with encoding errors	Mitigate	Root Cause # 1	Manually download and check files	2025-12-06	Complete
Manually check all CA certificate download locations for correct file encoding	Mitigate	Root Cause # 1	Manually download and check files	2025-12-08	Complete
Manually check all CRL download locations for correct file encoding	Mitigate	Root Cause # 1	Manually download and check files	2025-12-08	Complete
Adjust AutoWorker configuration file to cover all verification checklists	Detect	Root Cause # 2	Tool version update	2025-12-12	Complete
Add certificate file encoding checks to Certificate Operation SOPs	Prevent	Root Cause # 1	SOP document version update	2026-01-31	Complete
Add an item to Certificate Operation SOPs to notify AutoWorker of changes	Prevent	Root Cause # 3	SOP document version update	2026-01-31	Complete

Comment 8

[Sean Huang (TWCA)]

Report Closure Summary

• Incident description: The incident involved the publication of a CA certificate referenced in the AIA (caIssuers) field that was not encoded in DER format as required by RFC 5280. The issue was limited to the externally downloadable CA certificate file and did not affect the encoding or validity of issued certificates themselves.
• Incident Root Cause(s): The root cause was the absence of explicit procedural checks for externally published CA certificate file encoding, combined with incomplete coverage and synchronization between certificate operations and the automated verification tool (AutoWorker).
• Remediation description: The incorrectly encoded certificate file was promptly replaced with a correctly DER-encoded version. All CA certificate and CRL download locations were manually reviewed and verified to ensure correct encoding. In addition, AutoWorker was updated to expand verification coverage, and certificate operation SOPs were revised to include file encoding validation and synchronization requirements with automated verification mechanisms.
• Commitment summary: TWCA commits to ongoing maintenance of updated certificate operation SOPs and continued use and improvement of automated verification tools to ensure that externally published certificate artifacts remain compliant with applicable standards and requirements.

All Action Items disclosed in this report have been completed as described, and we request its closure.

Comment 9

[incident-reporting]

This is a final call for comments or questions on this Incident Report.

Otherwise, it will be closed on approximately 2026-01-13.