Open Bug 2081595 Opened 10 days ago Updated 3 days ago

GlobalSign: EV Test Certificates missing for 2 roots

Categories

(CA Program :: CA Certificate Compliance, task)

Tracking

(Not tracked)

ASSIGNED

People

(Reporter: pki-notifications, Assigned: pki-notifications)

Details

(Whiteboard: [ca-compliance] [policy-failure])

Preliminary Incident Report

Summary

  • Incident description: EV Test Certificates are missing for 2 of our EV enabled TLS Roots
  • Relevant policies: Guidelines for the Issuance and Management of Extended Validation Certificates: Appendix A
  • Source of incident disclosure: Third Party Reported

We are currently investigating and will provide a full incident report by 2026-10-20

Assignee: nobody → pki-notifications
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Whiteboard: [ca-compliance] [policy-failure]

Full Incident Report

Summary

  • CA Owner CCADB unique ID: A000027
  • Incident description: On 2026-10-12 we received a Certificate Problem Report for two of our Roots (R3 and R5) Test Sites. EV Guidelines require these Test Site certificates to be EV whilst we have OV certificates installed.
  • Timeline summary:
    • Non-compliance start date: 2023-07-11
    • Non-compliance identified date: 2026-10-12
    • Non-compliance end date: ongoing
  • Relevant policies: Guidelines for the Issuance and Management of Extended Validation Certificates: Appendix A
  • Source of incident disclosure: Third Party Reported

Impact

  • Total number of certificates: No subscriber certificates were impacted.
  • Total number of "remaining valid" certificates: Not applicable, the incident is related to missing EV test certificates.
  • Affected certificate types: Not applicable, the incident is related to missing EV test certificates.
  • Incident heuristic: Certificates installed on the following Test Sites have OV certificates instead of the required EV:
    • R3 Root: valid.r3.roots.globalsign.com, revoked.r3.roots.globalsign.com, expired.r3.roots.globalsign.com
    • R5 Root: valid.r5.roots.globalsign.com, revoked.r5.roots.globalsign.com, expired.r5.roots.globalsign.com
  • Was issuance stopped in response to this incident, and why or why not?: Issuance was not stopped due to not being an issue with certificates issued to subscribers
  • Analysis: The issue is limited to the Test Sites for the R3 Root and R5 Root, internal review of all other Test Sites confirmed they have the required EV certificates installed.
  • Additional considerations: The two impacted Roots are multiple purpose Roots and have been marked for deprecation since July. Therefore we stopped issuance of EV TLS certificates since July 27.

Timeline

Date/Time (UTC) Description
2025-09-19 Deploy monitoring script to verify our Test Sites certificates are all in correct state (valid, revoked, expired)
2023-07-11 - 2023-07-17 Ticket and discussion for review and creation of EV CAs for issuance of certificates for Test Site from our new platform
2026-07-27 Disabled EV TLS Issuance from R3 and R5 Roots as part of changeover to single purpose Roots
2026-10-12 13:51 Report from Third Party
2026-10-12 14:32 Investigated and confirmed the issue
2026-10-13 12:01 Response to Third Party
2026-10-13 12:41 Issued EV certificate for Valid R3 Root Test Sites
2026-10-13 12:42 Issued EV certificate for Revoked R3 Root Test Sites
2026-10-13 13:28 Identified that configuration for variable lifetimes had been removed from R3 EV CA for short term certificate issuance
2026-10-13 13:41 Identified that configuration for R5 EV certificate issuance had been removed from our issuance system
2026-10-14 07:25 Determined one of the old R5 issued certificates from 2021 can be used for R5 Expired Test Site
2026-10-14 10:37 Confirmed that issuance from R5 CA would require development or a new CA
2026-10-15 05:40 Request created for new R5 EV issuer and scheduled creation for November
2026-10-15 07:34 Identified solution for issuing short term certificate from R3 Root
2026-10-20 08:45 Implemented solution for issuing short term certificate from R3 Root
2026-10-21 05:16 Issued 1 hour EV certificate for R3 Root Expired Test Site

Related Incidents

Bug Date Description
2081155 2026-10-12 Disclosed test pages served OV TLS certificates rather than EV TLS certificates
2081411 2026-10-13 Missing EV TLS test certificates.
2081961 2026-10-14 Disclosed test pages served DV TLS certificates rather than EV TLS certificates
2082334 2026-10-14 Disclosed test pages served OV TLS certificates rather than EV TLS certificates

Root Cause Analysis

Contributing Factor #1: Failed to flag that the identified R3 and R5 CAs were OV instead of the required EV.

  • Description: In July 2023 we performed a review of available CAs for issuance of certificates for our Test Sites via our new platform, we concluded that we only needed a single EV CA from our R1 Root for its Test Sites in order that we would have issuers available from each Root for this purpose. The review listed the EV requirement and the CAs to be used from each Root and we did create an EV CA for R1 Root as a result of this review; however, we failed to flag that the two CAs identified for R3 Root and R5 Root issuance were actually OV enabled CAs instead of the required EV.
  • Timeline: 2023-07-11 - 2023-07-17
  • Detection: Root cause analysis of this incident.
  • Interaction with other factors: Because the review did not explicitly validate the certificate type for these two Roots, the gap in the monitoring script was not compensated for by this review.
  • Root Cause Analysis methodology used: 5-Whys

Contributing Factor #2: Our testing script did not verify the Test Sites certificate types

  • Description: We have a monitoring script in place which checks our Test Sites certificates are all in correct state (valid, revoked, expired) at any given time however this monitor does not verify the actual certificate types are EV for EV enabled roots.
  • Timeline: 2025-09-19
  • Detection: Root cause analysis of this incident.
  • Interaction with other factors: The absence of a check for certificate types in the monitoring script meant that detection depended on the review.
  • Root Cause Analysis methodology used: 5-Whys

Lessons Learned

  • What went well: We were able to re-enable our R3 Root EV Issuer for issuing replacement certificates, so no new Issuing CA had to be created.
  • What didn’t go well: For R5 Root EV, because the configuration was already removed from our production platform, this has introduced delay in issuing replacement certificates.
  • Where we got lucky: N/A
  • Additional: N/A

Action Items

Action Item Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Create and deploy new certificates for R3 Test sites Mitigate Root Cause # 1 R3 test-site certificates are generated, deployed to all required R3 test endpoints, and pass validation for chain, expiry, hostname and EV policy/OID. 2026-10-21 Complete
Create and deploy new EV CA for issuance of certificates for R5 Test Sites Mitigate Root Cause # 1 The new EV CA is created and configured for R5 test-site certificate issuance, with CA properties, issuance profile, chain, and audit evidence reviewed and approved before use. 2027-01-07 Ongoing
Create and deploy new certificates for R5 Test sites Mitigate Root Cause # 1 R5 test-site certificates are issued from the approved CA, deployed to all required R5 test endpoints, and pass validation for chain, expiry, hostname and EV policy/OID. 2027-01-07 Ongoing
Update monitoring script to verify certificate type and Root Detect Root Cause # 2 The monitoring script correctly identifies certificate type and root chain across all Test Site samples and reports mismatches or unsupported types clearly. 2026-12-02 Ongoing
You need to log in before you can comment on or make changes to this bug.