Closed Bug 1979475 Opened 1 year ago Closed 6 months ago

Microsoft PKI Services: End Entity Certificate Mis-issuance against CPS (BasicConstraints)

Categories

(CA Program :: CA Certificate Compliance, task)

Product:
CA Program
Component:
CA Certificate Compliance
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: CentralPKI, Assigned: CentralPKI)

Details

(Whiteboard: [ca-compliance] [policy-failure] [ov-misissuance])

Attachments

(2 files)

BasicConstraint_Certs.txt
68.05 KB, text/plain
Details
BasicConstraint_Impacted Certificates.xlsx
394.50 KB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
Details

Preliminary Incident Report

Summary

Incident Description:

Microsoft PKI Services identified a compliance issue involving the issuance of Public TLS end entity certificates that did not conform with our Microsoft Certificate Practice Statement (CPS). On 2025-07-24 we discovered that 784 certificates were issued without the Basic Constraint extension. Even though the BRs specify this extension as optional, our CPS requires this field.

The issue was discovered through enhanced monitoring that is being added as part of remediations for Bugzilla 1962829. All impacted certificates were revoked on 2025-07-25.

Relevant policies:

  • TLS Baseline Requirements v2.1.5 – Section 7.1.2.7.6 Subscriber Certificate Extensions

  • Microsoft PKI Services Public TLS CPS v3.3.2 – Appendix B: Organization Validated TLS Subscriber Certificate (Related to Section 7.1.2.7)

Source of incident disclosure:

Microsoft PKI Services identified the issue internally through internal enhanced monitoring on 2025-07-24 06:53:31 PDT.

Assignee: nobody → CentralPKI
Whiteboard: [ca-compliance] [policy-failure]
Whiteboard: [ca-compliance] [policy-failure] → [ca-compliance] [policy-failure] [ov-misissuance]

Weekly Status Update

We are actively working on the full incident report and will have completed and shared no later than 8/8.

Full Incident Report

Summary

  • CA Owner CCADB unique ID:
    A002577

  • Incident description:
    Microsoft PKI Services identified a compliance issue involving the issuance of Public TLS end entity certificates that did not conform with our Microsoft Certificate Practice Statement (CPS). On 2025-07-24 we discovered that 784 certificates were issued without the Basic Constraint extension. Even though the BRs specify this extension as optional, our CPS requires this field. There were no security concerns with the issued certificates.

    The issue was discovered through enhanced monitoring that is being added as part of remediations for Bugzilla 1962829. All impacted certificates were revoked on 2025-07-25.

  • Timeline summary:

    • Non-compliance start date:
      2025-04-21 14:25 PDT
    • Non-compliance identified date:
      2025-07-24 06:53 PDT
    • Non-compliance end date:
      2025-07-25 18:00 PDT

  • Relevant policies:

  • Source of incident disclosure:
    Self-Reported, Microsoft PKI Services identified the issue internally through internal enhanced monitoring on 2025-07-24 06:53 PDT.

Impact

  • Total number of certificates:
    784
  • Total number of "remaining valid" certificates:
    0 - All certificates were revoked on 7/25/2025
  • Affected certificate types:
    Organization Validated TLS Subscriber Certificates
  • Incident heuristic:
    This incident impacts all OV Subscriber certificates that were issued with Server Authentication Only profiles
  • Was issuance stopped in response to this incident, and why or why not?:
    Yes. Once the issue was confirmed, all affected profiles were disabled, effectively halting the issuance and renewal of certificates until profiles were fixed.
  • Analysis:
    Not applicable. The Whiteboard field does not contain revocation-delay, and no delay occurred. All affected certificates were revoked as part of the incident.
  • Additional considerations:
    N/A

Timeline

  • 2020-07-23 18:24 PDT – Creation of profiles
  • 2020-07-23 18:24 PDT – Profiles were put into production rotation – no certificates were issued at the time
  • 2025-02-10 (approx.) - Subscribers granted permissions to use older Certificate Profiles configured for Server Authentication only
  • 2025-04-21 14:25 PDT - Certificate was issued with non-compliance
  • 2025-07-24 06:53 PDT – Issue was reported via enhanced monitoring mechanisms. Alerting incident was created.
  • 2025-07-24 07:53 PDT – Internal investigation of the incident began
  • 2025-07-24 14:11 PDT – Impacted certificate profiles were disabled, halting issuance/renewals for certificates using these profiles
  • 2025-07-24 15:17 PDT – Notifications sent to impacted subscriber regarding incident
  • 2025-07-24 18:12 PDT – Fixes applied to profiles and re-enabled into production
  • 2025-07-25 18:00 PDT – Certificate revocation complete for all impacted certificates (770/784 – 14 expired)
  • 2025-07-25 18:00 PDT – Incident ended
  • 2025-07-25 18:24 PDT – Preliminary Incident Report posted

Related Incidents

Bug Date Opened Description
1711147 2021-05-13 Microsoft PKI Services issued eight Intermediate CA certificates without the required certificatePolicies extension due to a misconfigured CA profile. While this was a BR violation, it highlights the importance of enforcing controls at the CA issuance profile level—similar to subject incidents involving profile misconfiguration or nonconformance with requirements.
1885132 2024-03-13 TWCA issued 16,481 OV TLS certificates with a non-critical basicConstraints extension, violating BR §7.1.2.6. The root cause was determined to be a failure to properly configure their certificate profiles prior to issuance.
1962829 2025-04-25 Microsoft PKI Services identified that their CPS incorrectly stated keyEncipherment was not included in RSA subscriber certificates. The repair item associated to this bug (enhanced monitoring) is what caught this bug.
1972887 2025-06-18 A Trust discovered multiple end-entity certificates that violated Baseline Requirements, including non-critical basicConstraints, missing SAN, and improper subject attributes. This was caused by misconfiguration in the issuance process, similar to the subject incident if related to misconfigured certificate profiles or missing lint validation.

Root Cause Analysis

Contributing Factor #1: Outdated profile configuration

  • Description:
    The certificate profiles involved in this incident were originally created in July 2020 for server authentication-only use cases. At the time of creation, the omission of the Basic Constraints extension was consistent with both the Baseline Requirements (BRs) and the Microsoft CPS in effect. These profiles were not actively used upon creation and remained dormant until 2025, when a relevant use case emerged.

    Upon reuse, the profiles were not reviewed against current CPS requirements. Post issuance validation later flagged the absence of the Basic Constraints extension, which is now required by the updated CPS, resulting in the issuance of 784 end-entity certificates that were non-compliant with the CPS.

  • Timeline:

    • 2020-07-23 18:24 PDT – Creation of Profiles
    • 2025-02-10 (approx) – Subscribers granted permissions to use older Certificate Profiles configured for Server Authentication only
    • 2025-04-21 14:25 PDT – Issuance of non-compliant certificates
    • 2025-07-24 06:53 PDT – Identification of non-compliance
    • 2025-07-25 18:00 PDT – Revocation of non-compliant certificates

  • Detection:
    The issue was identified through enhanced monitoring implemented as part of remediation efforts for Bugzilla 1962829. This monitoring flagged the absence of the Basic Constraints extension shortly after the certificates were issued.

  • Interaction with other factors:
    The legacy nature of the profiles allowed them to be reintroduced into production. Additionally, the misalignment between linting tools and CPS enforcement (see Factor #3) meant that the profiles passed pre-validation checks despite being non-compliant with the CPS.

  • Root Cause Analysis methodology used:
    5 Whys

Contributing Factor #2: Profile Validation Oversight

  • Description:
    Before reusing the legacy certificate profiles, the associated issuance profiles were not validated to ensure compliance against the most current Microsoft CPS requirements.

  • Timeline:

    • 2025-04-21 14:25 PDT – Issuance of non-compliant certificates
    • 2025-07-24 06:53 PDT – Identification of non-compliance
    • 2025-07-24 14:11 PDT – Impacted profiles were disabled to halt further issuance
    • 2025-07-24 18:12 PDT – Impacted profiles were fixed to include BasicConstraint extension and re-enabled in production

  • Detection:
    The issue was identified after issuance through enhanced monitoring. Internal investigation revealed that the profiles were not subjected to a compliance checklist and linting process aligned with current CPS requirements for reuse.

  • Interaction with other factors:
    This condition amplified the effects of the legacy profile misconfiguration noted in Contributing Factor #1. While pre- and post-issuance linting were in place, the absence of a process to reassess profile conformance to the CPS prior to reuse, combined with gaps in linting enforcement described in Contributing Factor #3, resulted in a missed detection of non-compliant certificates until surfaced by monitoring systems.

  • Root Cause Analysis methodology used:
    5 Whys

Contributing Factor #3: Nonconformance Between Linting and CPS Requirements

  • Description:
    Microsoft PKI Services relied on external linting tools to validate certificate profiles and issued certificates. These tools are designed to enforce compliance with the CAB Forum Baseline Requirements (BRs), which they do effectively. However, they do not account for additional constraints or stricter requirements defined in Microsoft’s internal Certificate Practice Statement (CPS). In this incident, the Basic Constraints extension, optional under BRs but required by the CPS, was not flagged as missing by the external linting tools. This gap allowed certificates to pass validation and be issued despite being non-compliant with internal policy.

  • Timeline:

    • 2021-07-25 14:32 PDT Linting tool implemented
    • 2025-07-24 06:53 PDT Incident revealed gap between linting enforcement and CPS requirements.

  • Detection:
    Similar to the above, the issue was not caught by linting tools but was instead identified through enhanced monitoring implemented as part of Bugzilla 1962829 remediation efforts. Post-incident analysis confirmed that the linting tools did not enforce CPS-specific rules.

  • Interaction with other factors:
    This factor amplified the impact of both the legacy profile misconfiguration (Factor #1) and the missed validation against our CPS (Factor #2). Because linting tools conform against the BR and not our CPS, the misconfigured profiles were able to pass validation and be used for issuance without triggering alerts.

  • Root Cause Analysis methodology used:
    5 whys

Lessons Learned

  • What went well:

    • Detection via Enhanced Monitoring: Microsoft PKI Services identified this issue internally through our enhanced monitoring implemented as part of remediations for Bugzilla 1962829. This proactive detection allowed the team to act before customer impact escalated.

  • What didn’t go well:

    • Profile Misconfiguration: The root cause was an older certificate profile that became obsolete and failed to enforce the required Basic Constraints extension, violating CPS policy.
    • Linting and CPS nonconformance: The issue revealed that linting tools did not fully enforce CPS requirements, allowing non-compliant certs to be issued.
    • Ambiguity in the current CPS: The requirement for the Basic Constraints extension was not clearly and unambiguously stated, leaving room for interpretation and contributing to our inability to catch this issue during validation.

  • Where we got lucky:

    • Impacted certificates in NPE: The impacted certificates were utilized in Non-Production Environment testing minimizing production impact.
    • Low usage of impacted profiles: The profiles in question had very low utilization (784 certs) reducing the blast radius for potential impact.
    • No SLA or Security Breach: Despite the policy violation, this was a non-compliance against our CPS, not a security related issue.

  • Additional:
    N/A

Action Items

Action Item Description Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Establish a mandatory validation checklist for issuance profiles before activation or reuse, ensuring conformance with current CPS requirements. Prevent Root Cause #1 Effectiveness will be measured by the inclusion of the checklist in change management workflows and verified through internal audit logs. Public impact can be observed via reduced incidents of CPS non-compliance in future incident reports. TBD Proposed
Implement internal linting rules that enforce CPS-specific requirements, including Basic Constraints, to supplement external BR-based linting tools. Prevent Root Cause #3 Effectiveness will be demonstrated by internal test certificates flagged for CPS violations and tracked in internal validation logs. Public verification possible via Certificate Transparency logs showing compliant certificate profiles post-implementation. TBD Proposed
Update CPS documentation to clarify the requirement for Basic Constraints in Subscriber certificates and remove ambiguity. Mitigate Root Cause #3 Effectiveness will be measured by publication of CPS v3.3.X with updated language. Public can verify via the published CPS on the Microsoft PKI Services website and future audit statements referencing the updated CPS. TBD Proposed
Conduct a cleanup of legacy profiles and profiles that are no longer in use or fail to meet current CPS standards. Mitigate Root Cause #2 Effectiveness will be measured by the reduction in unused or non-compliant profiles, documented in internal inventory logs. Public impact can be inferred from the absence of similar profile-related incidents in future reports. TBD Proposed

Appendix

Weekly Status Update

We’re actively addressing all action items and have updated due date for action item #2 and are working to establish dates for the rest.

Action Item Description Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Establish a mandatory validation checklist for issuance profiles before activation or reuse, ensuring conformance with current CPS requirements. Corrective Root Cause #1 Effectiveness will be measured by the inclusion of the checklist in change management workflows and verified through internal audit logs. Public impact can be observed via reduced incidents of CPS non-compliance in future incident reports. TBD New
Implement internal linting rules that enforce CPS-specific requirements, including Basic Constraints, to supplement external BR-based linting tools. Preventative Root Cause #3 Effectiveness will be demonstrated by internal test certificates flagged for CPS violations and tracked in internal validation logs. Public verification possible via Certificate Transparency logs showing compliant certificate profiles post-implementation. 2025-09-30 New
Update CPS documentation to clarify the requirement for Basic Constraints in Subscriber certificates and remove ambiguity. Preventative Root Cause #3 Effectiveness will be measured by publication of CPS v3.3.X with updated language. Public can verify via the published CPS on the Microsoft PKI Services website and future audit statements referencing the updated CPS. TBD New
Conduct a cleanup of legacy profiles and profiles that are no longer in use or fail to meet current CPS standards. Corrective Root Cause #2 Effectiveness will be measured by the reduction in unused or non-compliant profiles, documented in internal inventory logs. Public impact can be inferred from the absence of similar profile-related incidents in future reports. TBD New
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true

Weekly Status Report

We are actively progressing through the action items below. We have provided due dates for the remaining action items.

Here’s your table converted to raw Markdown:

Action Item Description Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Establish a mandatory validation checklist for issuance profiles before activation or reuse, ensuring conformance with current CPS requirements. Corrective Root Cause #1 Effectiveness will be measured by the inclusion of the checklist in change management workflows and verified through internal audit logs. Public impact can be observed via reduced incidents of CPS non-compliance in future incident reports. 10/15/2025 In Progress
Implement internal linting rules that enforce CPS-specific requirements, including Basic Constraints, to supplement external BR-based linting tools. Preventative Root Cause #3 Effectiveness will be demonstrated by internal test certificates flagged for CPS violations and tracked in internal validation logs. Public verification possible via Certificate Transparency logs showing compliant certificate profiles post-implementation. 9/30/2025 In Progress
Update CPS documentation to clarify the requirement for Basic Constraints in Subscriber certificates and remove ambiguity. Preventative Root Cause #3 Effectiveness will be measured by publication of CPS v3.3.X with updated language. Public can verify via the published CPS on the Microsoft PKI Services website and future audit statements referencing the updated CPS. 9/15/2025 In Progress
Conduct a cleanup of legacy profiles and profiles that are no longer in use or fail to meet current CPS standards. Corrective Root Cause #2 Effectiveness will be measured by the reduction in unused or non-compliant profiles, documented in internal inventory logs. Public impact can be inferred from the absence of similar profile-related incidents in future reports. 9/15/2025 In Progress

Weekly Status Report

We are actively progressing through the action items below. We remain on track to meet the expected due dates outlined in the full incident report

Action Item Description Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Establish a mandatory validation checklist for issuance profiles before activation or reuse, ensuring conformance with current CPS requirements. Corrective Root Cause #1 Effectiveness will be measured by the inclusion of the checklist in change management workflows and verified through internal audit logs. Public impact can be observed via reduced incidents of CPS non-compliance in future incident reports. 10/15/2025 In Progress
Implement internal linting rules that enforce CPS-specific requirements, including Basic Constraints, to supplement external BR-based linting tools. Preventative Root Cause #3 Effectiveness will be demonstrated by internal test certificates flagged for CPS violations and tracked in internal validation logs. Public verification possible via Certificate Transparency logs showing compliant certificate profiles post-implementation. 9/30/2025 In Progress
Update CPS documentation to clarify the requirement for Basic Constraints in Subscriber certificates and remove ambiguity. Preventative Root Cause #3 Effectiveness will be measured by publication of CPS v3.3.X with updated language. Public can verify via the published CPS on the Microsoft PKI Services website and future audit statements referencing the updated CPS. 9/15/2025 In Progress
Conduct a cleanup of legacy profiles and profiles that are no longer in use or fail to meet current CPS standards. Corrective Root Cause #2 Effectiveness will be measured by the reduction in unused or non-compliant profiles, documented in internal inventory logs. Public impact can be inferred from the absence of similar profile-related incidents in future reports. 9/15/2025 In Progress

Weekly Status Update

We are actively progressing through the action items below. Please see updated due date for action item #3.

Action Item Description Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Establish a mandatory validation checklist for issuance profiles before activation or reuse, ensuring conformance with current CPS requirements. Corrective Root Cause #1 Effectiveness will be measured by the inclusion of the checklist in change management workflows and verified through internal audit logs. Public impact can be observed via reduced incidents of CPS non-compliance in future incident reports. 10/15/2025 New
Implement internal linting rules that enforce CPS-specific requirements, including Basic Constraints, to supplement external BR-based linting tools. Preventative Root Cause #3 Effectiveness will be demonstrated by internal test certificates flagged for CPS violations and tracked in internal validation logs. Public verification possible via Certificate Transparency logs showing compliant certificate profiles post-implementation. 9/30/2025 In Progress
Update CPS documentation to clarify the requirement for Basic Constraints in Subscriber certificates and remove ambiguity. Preventative Root Cause #3 Effectiveness will be measured by publication of CPS v3.3.X with updated language. Public can verify via the published CPS on the Microsoft PKI Services website and future audit statements referencing the updated CPS. 9/20/2025 In Progress
Conduct a cleanup of legacy profiles and profiles that are no longer in use or fail to meet current CPS standards. Corrective Root Cause #2 Effectiveness will be measured by the reduction in unused or non-compliant profiles, documented in internal inventory logs. Public impact can be inferred from the absence of similar profile-related incidents in future reports. 9/15/2025 New

Weekly Status Update

We are actively progressing through the action items below. Please see update due date for action item #2 & #4.

Action Item Description Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Establish a mandatory validation checklist for issuance profiles, ensuring conformance with current CPS requirements. Corrective Root Cause #1 Effectiveness will be measured by the inclusion of the checklist in change management workflows and verified through internal audit logs. Public impact can be observed via reduced incidents of CPS non-compliance in future incident reports. 2025-10-15 New
Implement internal linting rules that enforce CPS-specific requirements, including Basic Constraints, to supplement external BR-based linting tools. Preventative Root Cause #3 Effectiveness will be demonstrated by internal test certificates flagged for CPS violations and tracked in internal validation logs. Public verification possible via Certificate Transparency logs showing compliant certificate profiles post-implementation. 2025-10-30 In Progress
Update CPS documentation to clarify the requirement for Basic Constraints in Subscriber certificates and remove ambiguity. Preventative Root Cause #3 Effectiveness will be measured by publication of CPS v3.3.X with updated language. Public can verify via the published CPS on the Microsoft PKI Services website and future audit statements referencing the updated CPS. 2025-09-20 In Progress
Conduct a cleanup of legacy profiles and profiles that are no longer in use or fail to meet current CPS standards. Corrective Root Cause #2 Effectiveness will be measured by the reduction in unused or non-compliant profiles, documented in internal inventory logs. Public impact can be inferred from the absence of similar profile-related incidents in future reports. 2025-09-30 In Progress

Weekly Status Update

We are actively progressing on the full set of action items outlined in the incident report. Please note the update action #8. We have also closed action item #5, based on investigation with subscribers who account for 90+% of the certificate volume we have determined that the usage is valid use of public TLS certificates, but there are opportunities to reduce certificate lifetimes, which is now tracked as part of action item #7.

Action Item Description Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Revoke impacted certificates (in batches beginning 5/28/2025) Mitigate Root Cause 1 Percent of impacted certificates revoked will be tracked and published monthly. Verification possible via Certificate Transparency (CT) logs and serial number disclosure via Microsoft’s CRL. 2025-11-15 In Progress
Migrate cert issuance to use partitioned CRLs Prevent Root Cause 1 Percentage of newly issued certificates appearing in CT logs with updated CDP endpoints pointing to partitioned CRLs. Logs and CRL URLs can be independently verified by the public. 2025-11-15 In Progress
Standup cross-signed warm standby CAs. We are currently in planning stages. We will have the plan ready before 06/14/2025 Prevent Root Cause 1 Standby ICAs will be disclosed in CT logs with test certificates. Public can verify issuance and presence of standby ICAs through CT logs and Microsoft’s published CA repository. 2025-09-30 In Progress
Create training and TSG Documentation to educate team on revocation expectations Prevent Root Cause 1 Training completion rates will be tracked internally. Effectiveness will be evaluated through internal audits and inclusion of the training materials in external audit reviews. 2025-07-31 Complete
Reduce usage of public PKI Prevent Root Cause 1 Publish a monthly percentage reduction of unexpired, publicly trusted certificates issued from impacted hierarchies. Public can track progress using CT log data filtered for affected intermediates. 2025-09-30 Complete
Exercise and refine the mass revocation playbook Prevent Root Cause 1 Effectiveness will be assessed through internal tracking of simulated revocation scenarios, including coverage and execution timing. The results of these exercises will inform iterative improvements to the playbook. While objective external metrics are limited, Microsoft will evaluate the impact through internal reviews and incorporate this action into relevant audit scopes. 2025-09-01 Complete
Publish a phased plan to reduce the default certificate validity period, with the long-term goal of transitioning to short-lived certificates. Preventive Root Cause 1 Effectiveness will be measured by publication of the plan by 2025-08-22. Public can verify via the published plan and future CPS updates reflecting the proposed changes. 2025-08-22 Complete
Begin implementation of the phased certificate lifecycle reduction plan, including updates to issuance systems and CPS. Preventive Root Cause 1 Effectiveness will be measured by issuance of certificates with reduced validity periods, visible in CT logs, and updated CPS language. Public can verify through CT data and CPS version history. 2025-09-22 In-Progress
Complete migration of all customers to G2 ICAs with CRL partitioning and eliminate issuance from non-partitioned ICAs. Mitigate Root Cause 1 Effectiveness will be measured by the percentage of certificates issued from G2 ICAs with partitioned CRLs, visible in CT logs. Public can verify through CCADB hierarchy updates and issuance patterns in CT. Internal tracking will confirm deprecation of non-partitioned ICAs. We will provide regular updates on the burndown for G1 to G2 transition. 2026-02-28 New
Develop and publish a plan for regular ICA rotations to maintain operational readiness and crypto agility. Preventive Root Cause 1 Effectiveness will be measured by publication of the ICA rotation plan. ICA rotation can be publicly verified through CCADB and CT logs as we execute the plan. 2025-10-17 New

Revocation Delay Status Update

  • Total certificates revoked (planned to date):
    9,808,144 (10,558,644)

  • Remaining active certificates (total affected):
    13,845,195 (72,070,777)

  • Total certificates expired and not revoked (to date):
    47,295,464

  • Estimate for remaining revocations:
    We will continue to revoke certificates in batches until 11/15/2025

Please disregard Comment 10 and Comment 11 as these are meant for Bug 1965612

Weekly Status Update

We are actively progressing through the action items below. Please note that action item #3 is complete.

Action Item Description Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Establish a mandatory validation checklist for issuance profiles, ensuring conformance with current CPS requirements. Corrective Root Cause #1 Effectiveness will be measured by the inclusion of the checklist in change management workflows and verified through internal audit logs. Public impact can be observed via reduced incidents of CPS non-compliance in future incident reports. 2025-10-15 In Progress
Implement internal linting rules that enforce CPS-specific requirements, including Basic Constraints, to supplement external BR-based linting tools. Preventative Root Cause #3 Effectiveness will be demonstrated by internal test certificates flagged for CPS violations and tracked in internal validation logs. Public verification possible via Certificate Transparency logs showing compliant certificate profiles post-implementation. 2025-10-30 In Progress
Update CPS documentation to clarify the requirement for Basic Constraints in Subscriber certificates and remove ambiguity. Preventative Root Cause #3 Effectiveness will be measured by publication of CPS v3.3.X with updated language. Public can verify via the published CPS on the Microsoft PKI Services website and future audit statements referencing the updated CPS. 2025-09-20 Done
Conduct a cleanup of legacy profiles and profiles that are no longer in use or fail to meet current CPS standards. Corrective Root Cause #2 Effectiveness will be measured by the reduction in unused or non-compliant profiles, documented in internal inventory logs. Public impact can be inferred from the absence of similar profile-related incidents in future reports. 2025-09-30 In Progress

Weekly Status Update

We are actively progressing through the action items identified in the full incident report. No significant changes to report at this time.

Weekly Status Report

We are actively progressing through the action items below. Please note that action item #4 is complete.

Action Item Description Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Establish a mandatory validation checklist for issuance profiles, ensuring conformance with current CPS requirements. Corrective Root Cause #1 Effectiveness will be measured by the inclusion of the checklist in change management workflows and verified through internal audit logs. Public impact can be observed via reduced incidents of CPS non-compliance in future incident reports. 10/15/2025 In Progress
Implement internal linting rules that enforce CPS-specific requirements, including Basic Constraints, to supplement external BR-based linting tools. Preventative Root Cause #3 Effectiveness will be demonstrated by internal test certificates flagged for CPS violations and tracked in internal validation logs. Public verification possible via Certificate Transparency logs showing compliant certificate profiles post-implementation. 10/30/2025 In Progress
Update CPS documentation to clarify the requirement for Basic Constraints in Subscriber certificates and remove ambiguity. Preventative Root Cause #3 Effectiveness will be measured by publication of CPS v3.3.X with updated language. Public can verify via the published CPS on the Microsoft PKI Services website and future audit statements referencing the updated CPS. 9/20/2025 Complete
Conduct a cleanup of legacy profiles and profiles that are no longer in use or fail to meet current CPS standards. Corrective Root Cause #2 Effectiveness will be measured by the reduction in unused or non-compliant profiles, documented in internal inventory logs. Public impact can be inferred from the absence of similar profile-related incidents in future reports. 9/30/2025 Complete

Weekly Status Report

We are actively progressing through the action items identified in the full incident report. No significant changes to report at this time.

Weekly Status Update

We are actively progressing through the action items below. Please note that action item #1 is complete.

Action Item Description Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Establish a mandatory validation checklist for issuance profiles, ensuring conformance with current CPS requirements. Corrective Root Cause #1 Effectiveness will be measured by the inclusion of the checklist in change management workflows and verified through internal audit logs. Public impact can be observed via reduced incidents of CPS non-compliance in future incident reports. 10/15/2025 Complete
Implement internal linting rules that enforce CPS-specific requirements, including Basic Constraints, to supplement external BR-based linting tools. Preventative Root Cause #3 Effectiveness will be demonstrated by internal test certificates flagged for CPS violations and tracked in internal validation logs. Public verification possible via Certificate Transparency logs showing compliant certificate profiles post-implementation. 10/30/2025 In Progress
Update CPS documentation to clarify the requirement for Basic Constraints in Subscriber certificates and remove ambiguity. Preventative Root Cause #3 Effectiveness will be measured by publication of CPS v3.3.X with updated language. Public can verify via the published CPS on the Microsoft PKI Services website and future audit statements referencing the updated CPS. 9/20/2025 Complete
Conduct a cleanup of legacy profiles and profiles that are no longer in use or fail to meet current CPS standards. Corrective Root Cause #2 Effectiveness will be measured by the reduction in unused or non-compliant profiles, documented in internal inventory logs. Public impact can be inferred from the absence of similar profile-related incidents in future reports. 9/30/2025 Complete

Weekly Status Update

We are actively progressing through the action items below. No changes at this time.

Weekly Status Report

We are actively progressing through the action items below. We have updated the due date for action item #2.

Action Item Description Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Establish a mandatory validation checklist for issuance profiles, ensuring conformance with current CPS requirements. Corrective Root Cause #1 Effectiveness will be measured by the inclusion of the checklist in change management workflows and verified through internal audit logs. Public impact can be observed via reduced incidents of CPS non-compliance in future incident reports. 10/15/2025 Complete
Implement internal linting rules that enforce CPS-specific requirements, including Basic Constraints, to supplement external BR-based linting tools. Preventative Root Cause #3 Effectiveness will be demonstrated by internal test certificates flagged for CPS violations and tracked in internal validation logs. Public verification possible via Certificate Transparency logs showing compliant certificate profiles post-implementation. 11/7/2025 In Progress
Update CPS documentation to clarify the requirement for Basic Constraints in Subscriber certificates and remove ambiguity. Preventative Root Cause #3 Effectiveness will be measured by publication of CPS v3.3.X with updated language. Public can verify via the published CPS on the Microsoft PKI Services website and future audit statements referencing the updated CPS. 9/20/2025 Complete
Conduct a cleanup of legacy profiles and profiles that are no longer in use or fail to meet current CPS standards. Corrective Root Cause #2 Effectiveness will be measured by the reduction in unused or non-compliant profiles, documented in internal inventory logs. Public impact can be inferred from the absence of similar profile-related incidents in future reports. 9/30/2025 Complete

Weekly Status Update

We are actively progressing through the action items below. We have updated the due date for action item #2.

Action Item Description Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Establish a mandatory validation checklist for issuance profiles, ensuring conformance with current CPS requirements. Corrective Root Cause #1 Effectiveness will be measured by the inclusion of the checklist in change management workflows and verified through internal audit logs. Public impact can be observed via reduced incidents of CPS non-compliance in future incident reports. 10/15/2025 Complete
Implement internal linting rules that enforce CPS-specific requirements, including Basic Constraints, to supplement external BR-based linting tools. Preventative Root Cause #3 Effectiveness will be demonstrated by internal test certificates flagged for CPS violations and tracked in internal validation logs. Public verification possible via Certificate Transparency logs showing compliant certificate profiles post-implementation. 11/14/2025 In Progress
Update CPS documentation to clarify the requirement for Basic Constraints in Subscriber certificates and remove ambiguity. Preventative Root Cause #3 Effectiveness will be measured by publication of CPS v3.3.X with updated language. Public can verify via the published CPS on the Microsoft PKI Services website and future audit statements referencing the updated CPS. 9/20/2025 Complete
Conduct a cleanup of legacy profiles and profiles that are no longer in use or fail to meet current CPS standards. Corrective Root Cause #2 Effectiveness will be measured by the reduction in unused or non-compliant profiles, documented in internal inventory logs. Public impact can be inferred from the absence of similar profile-related incidents in future reports. 9/30/2025 Complete

Weekly Status Update

We are actively progressing through the action items below. We have updated the due date for action item #2.

Action Item Description Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Establish a mandatory validation checklist for issuance profiles, ensuring conformance with current CPS requirements. Corrective Root Cause #1 Effectiveness will be measured by the inclusion of the checklist in change management workflows and verified through internal audit logs. Public impact can be observed via reduced incidents of CPS non-compliance in future incident reports. 10/15/2025 Complete
Implement internal linting rules that enforce CPS-specific requirements, including Basic Constraints, to supplement external BR-based linting tools. Preventative Root Cause #3 Effectiveness will be demonstrated by internal test certificates flagged for CPS violations and tracked in internal validation logs. Public verification possible via Certificate Transparency logs showing compliant certificate profiles post-implementation. 11/24/2025 In Progress
Update CPS documentation to clarify the requirement for Basic Constraints in Subscriber certificates and remove ambiguity. Preventative Root Cause #3 Effectiveness will be measured by publication of CPS v3.3.X with updated language. Public can verify via the published CPS on the Microsoft PKI Services website and future audit statements referencing the updated CPS. 9/20/2025 Complete
Conduct a cleanup of legacy profiles and profiles that are no longer in use or fail to meet current CPS standards. Corrective Root Cause #2 Effectiveness will be measured by the reduction in unused or non-compliant profiles, documented in internal inventory logs. Public impact can be inferred from the absence of similar profile-related incidents in future reports. 9/30/2025 Complete

Weekly Status Update

We are actively progressing through all repair items identified in the incident report. No major changes at this time.

Weekly Status Update

We are actively progressing through the action items below. We have updated the due date for action item #2.

Action Item Description Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Establish a mandatory validation checklist for issuance profiles, ensuring conformance with current CPS requirements. Corrective Root Cause #1 Effectiveness will be measured by the inclusion of the checklist in change management workflows and verified through internal audit logs. Public impact can be observed via reduced incidents of CPS non-compliance in future incident reports. 10/15/2025 Complete
Implement internal linting rules that enforce CPS-specific requirements, including Basic Constraints, to supplement external BR-based linting tools. Preventative Root Cause #3 Effectiveness will be demonstrated by internal test certificates flagged for CPS violations and tracked in internal validation logs. Public verification possible via Certificate Transparency logs showing compliant certificate profiles post-implementation. 12/10/2025 In Progress
Update CPS documentation to clarify the requirement for Basic Constraints in Subscriber certificates and remove ambiguity. Preventative Root Cause #3 Effectiveness will be measured by publication of CPS v3.3.X with updated language. Public can verify via the published CPS on the Microsoft PKI Services website and future audit statements referencing the updated CPS. 9/20/2025 Complete
Conduct a cleanup of legacy profiles and profiles that are no longer in use or fail to meet current CPS standards. Corrective Root Cause #2 Effectiveness will be measured by the reduction in unused or non-compliant profiles, documented in internal inventory logs. Public impact can be inferred from the absence of similar profile-related incidents in future reports. 9/30/2025 Complete

Weekly Status Update

We are actively progressing through all repair items identified in the incident report. No major changes at this time.

Weekly Status Update

We are actively progressing through all repair items identified in the incident report. Please note the due date for Action Item #2 was updated to align with safe deployment and approval requirements.

Action Item Description Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Establish a mandatory validation checklist for issuance profiles, ensuring conformance with current CPS requirements. Corrective Root Cause #1 Effectiveness will be measured by the inclusion of the checklist in change management workflows and verified through internal audit logs. Public impact can be observed via reduced incidents of CPS non-compliance in future incident reports. 10/15/2025 Complete
Implement internal linting rules that enforce CPS-specific requirements, including Basic Constraints, to supplement external BR-based linting tools. Preventative Root Cause #3 Effectiveness will be demonstrated by internal test certificates flagged for CPS violations and tracked in internal validation logs. Public verification possible via Certificate Transparency logs showing compliant certificate profiles post-implementation. 1/9/2026 In Progress
Update CPS documentation to clarify the requirement for Basic Constraints in Subscriber certificates and remove ambiguity. Preventative Root Cause #3 Effectiveness will be measured by publication of CPS v3.3.X with updated language. Public can verify via the published CPS on the Microsoft PKI Services website and future audit statements referencing the updated CPS. 9/20/2025 Complete
Conduct a cleanup of legacy profiles and profiles that are no longer in use or fail to meet current CPS standards. Corrective Root Cause #2 Effectiveness will be measured by the reduction in unused or non-compliant profiles, documented in internal inventory logs. Public impact can be inferred from the absence of similar profile-related incidents in future reports. 9/30/2025 Complete

Weekly Status Report

We are actively progressing through the action items below. No changes at this time.

Weekly Status Update

We are actively working through the full set of action items outlined in the incident report. No changes at this time.

Weekly Status Report

We are actively progressing through the action items below. No changes at this time.

Weekly Status Update

We have completed all action items associated with this bug and will be posting our closure report soon.

Action Item Description Kind Corresponding Root Cause(s) Evaluation Criteria Due Date Status
Establish a mandatory validation checklist for issuance profiles, ensuring conformance with current CPS requirements. Corrective Root Cause #1 Effectiveness will be measured by the inclusion of the checklist in change management workflows and verified through internal audit logs. Public impact can be observed via reduced incidents of CPS non-compliance in future incident reports. 10/15/2025 Complete
Implement internal linting rules that enforce CPS-specific requirements, including Basic Constraints, to supplement external BR-based linting tools. Preventative Root Cause #3 Effectiveness will be demonstrated by internal test certificates flagged for CPS violations and tracked in internal validation logs. Public verification possible via Certificate Transparency logs showing compliant certificate profiles post-implementation. 1/9/2026 Complete
Update CPS documentation to clarify the requirement for Basic Constraints in Subscriber certificates and remove ambiguity. Preventative Root Cause #3 Effectiveness will be measured by publication of CPS v3.3.X with updated language. Public can verify via the published CPS on the Microsoft PKI Services website and future audit statements referencing the updated CPS. 9/20/2025 Complete
Conduct a cleanup of legacy profiles and profiles that are no longer in use or fail to meet current CPS standards. Corrective Root Cause #2 Effectiveness will be measured by the reduction in unused or non-compliant profiles, documented in internal inventory logs. Public impact can be inferred from the absence of similar profile-related incidents in future reports. 9/30/2025 Complete

Report Closure Summary

  • Incident description:
    Microsoft PKI Services identified a compliance issue involving the issuance of Public TLS end entity certificates that did not conform with our Microsoft Certificate Practice Statement (CPS). On 2025-07-24 we discovered that 784 certificates were issued without the Basic Constraint extension. Even though the BRs specify this extension as optional, our CPS requires this field. There were no other correctness or compliance issues with the issued certificates.
    The issue was discovered through enhanced monitoring that is being added as part of remediations for 1962829 - Microsoft PKI Services: Policy document bug. All impacted certificates were revoked on 2025-07-25.

  • Incident Root Cause(s):
    The incident occurred because legacy certificate profiles created in July 2020 for server authentication lacked the Basic Constraints extension, which was acceptable under the Baseline Requirements and Microsoft CPS at that time. When these profiles were reused in 2025 without reassessment against updated CPS requirements, 784 non-compliant certificates were issued. This was compounded by the absence of a formal validation process before the certificate profiles were reused and reliance on external linting tools that enforce CAB Forum BRs but not Microsoft CPS, allowing misconfigured profiles to pass checks. The issue was ultimately detected through enhanced monitoring introduced during Bugzilla 1962829 remediation.

  • Remediation description:
    To address the incident, four key actions were initiated: (1) establishing a mandatory validation checklist for issuance profiles before activation or reuse to ensure compliance with current CPS requirements, (2) implementing internal linting rules that enforce CPS-specific constraints, supplementing external BR-based tools, (3) updating CPS documentation to explicitly require Basic Constraints in subscriber certificates and eliminate ambiguity, and (4) performing a cleanup of legacy or unused profiles that fail to meet current CPS standards. These measures aim to prevent recurrence, strengthen compliance, and improve transparency through audits, validation logs, and updated public documentation.

  • Commitment summary:
    All identified action items were completed. Beyond these action items, we remain committed to continuous improvements in our tooling to minimize opportunities for human errors.
    All Action Items disclosed in this report have been completed as described, and we request its closure.

This is a final call for comments or questions on this Incident Report.

Otherwise, it will be closed on approximately 2026-01-20.

Flags: needinfo?(incident-reporting)
Whiteboard: [ca-compliance] [policy-failure] [ov-misissuance] → [close on 2026-01-20] [ca-compliance] [policy-failure] [ov-misissuance]

Weekly Status Update

The closure report associated to this bug has been submitted. Please close if no other comments are provided.

Status: ASSIGNED → RESOLVED
Closed: 6 months ago
Flags: needinfo?(incident-reporting)
Resolution: --- → FIXED
Whiteboard: [close on 2026-01-20] [ca-compliance] [policy-failure] [ov-misissuance] → [ca-compliance] [policy-failure] [ov-misissuance]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: