Open Bug 1964167 Opened 14 days ago Updated 1 day ago

VikingCloud: Missing CRL in CCADB

Categories

(CA Program :: CA Certificate Compliance, task)

Product:
CA Program
Component:
CA Certificate Compliance
Type:
task

Tracking

(Not tracked)

Status:
ASSIGNED

People

(Reporter: vikingcloud_ca_bugzilla, Assigned: vikingcloud_ca_bugzilla)

Details

(Whiteboard: [ca-compliance] [disclosure-failure])

Preliminary Incident Report

This is a preliminary report and VikingCloud will provide a full report no later than May 15, 2025.

Summary

  • Incident description:

On May 2, 2025 at 15:04 UTC, VikingCloud received a certificate problem report regarding a missing Certificate Revocation List (CRL) Distribution Point disclosure in CCADB for the Unique ID A012218. On May 2, 2025 at 18:43 UTC, this issue was rectified in CCADB.

  • Relevant policies:

This incident violates section 6 of the Chrome Root Program Policy, version 1.6:

Disclose either the Certificate Revocation List (CRL) Distribution Point or a JSON Array of Partitioned CRLs on root and subordinate CA certificate records in the CCADB within 7 days of the corresponding CA issuing its first certificate. This applies to each included CA certificate and each CA certificate chaining up to a certificate included in the Chrome Root Store.

  • Source of incident disclosure:

A Certificate Problem Report was filed by Chrome Root Program (CRP) Team.

Assignee: nobody → vikingcloud_ca_bugzilla
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Whiteboard: [ca-compliance] [disclosure-failure]

Full Incident Report

Summary

  • CA Owner CCADB unique ID: A000058

  • Incident description: On May 2, 2025 at 15:04 UTC, VikingCloud received a Certificate Problem Report regarding a missing Certificate Revocation List (CRL) Distribution Point disclosure in CCADB for the Unique ID A012218. This issue was resolved in CCADB on May 2, 2025 at 18:43 UTC.

  • Timeline summary:

    • Non-compliance start date: 2025-03-31
    • Non-compliance identified date: 2025-05-02 15:04 UTC
    • Non-compliance end date: 2025-05-02 18:43 UTC

  • Relevant policies:
    This incident violates section 6 of the Chrome Root Program Policy, version 1.6:

    Disclose either the Certificate Revocation List (CRL) Distribution Point or a JSON Array of Partitioned CRLs on root and subordinate CA certificate records in the CCADB within 7 days of the corresponding CA issuing its first certificate. This applies to each included CA certificate and each CA certificate chaining up to a certificate included in the Chrome Root Store.

  • Source of incident disclosure:
    A Certificate Problem Report was filed by Chrome Root Program (CRP) Team.

Impact

  • Total number of certificates: N/A
  • Total number of "remaining valid" certificates: N/A
  • Affected certificate types: N/A
  • Incident heuristic: N/A
  • Was issuance stopped in response to this incident, and why or why not?: N/A
  • Analysis: N/A
  • Additional considerations: N/A

Timeline

  • 2025-03-24: VikingCloud added certificate ID A012218 to CCADB.
  • 2025-04-08 10:03 UTC: CCADB sent an email regarding missing full CRL.
  • 2025-05-02 15:04 UTC: VikingCloud is informed of the failed disclosure by Chrome Root Program (CRP) Team.
  • 2025-05-02 18:43 UTC: VikingCloud updated the CCADB record.

Related Incidents

Bug Date Description
1818833 2023-02-24 14:53 PST Inaccurate CRL details were entered into CCADB. The process improvement was the addition of a second person to validate entry before submission into CCADB.

Root Cause Analysis

Contributing Factor #1: CCADB initial entry interrupted

  • Description: Initial entry of certificate into CCADB was interrupted due to equipment malfunction.
  • Timeline:
    • 2025-03-24: VikingCloud added certificate ID A012218 to CCADB
  • Detection: CCADB update failure was identified while investigating the Certificate Problem Report.
  • Interaction with other factors: No
  • Root Cause Analysis methodology used: 5-Whys

Contributing Factor #2: Missed alerts and normal checks

  • Description: CCADB correspondence is managed by two directly responsible individuals, with at least one additional person monitoring. At the time of the incident, we were in the process of transferring CCADB roles and responsibilities.
  • Timeline:
    • 2025-04-08 10:03 UTC: CCADB sent an email regarding missing full CRL
    • 2025-05-02 15:04 UTC: VikingCloud is informed of the failed disclosure by Chrome Root Program (CRP) Team.
    • 2025-05-02 18:43 UTC: VikingCloud updated the CCADB record. 
  • Detection: Process failure identified while investigating the Certificate Problem Report.
  • Interaction with other factors: Factor interaction increased impact time.
  • Root Cause Analysis methodology used: 5-Whys

Lessons Learned

  • What went well: Once notified, we rapidly updated the CCADB record.

  • What didn’t go well: During our transition, a designated owner was not maintained to ensure ongoing responsibility for all CCADB updates.

  • Where we got lucky: The full CRL was properly disclosed in another CCADB record.

  • Additional:

Action Items

Action Item Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Modify our protocol to include a secondary check of CCADB records 24 hours after any update. Prevent Root Cause # 1 This will verify that the CCADB record changes have been properly recorded. 2025-05-30 Ongoing
Define a regular cadence to check on CCADB and assign a designated owner during any transition. Prevent Root Cause # 2 Transition ownership will add protection against a future occurrence. 2025-05-30 Ongoing

Appendix

You need to log in before you can comment on or make changes to this bug.