2063842 - NETLOCK: Failure to file a preliminary incident report within 72 hours (OCSP responder incident, Bug 2051459)

Status: UNCONFIRMED

(CA Program :: CA Certificate Compliance, task)

Description

[[:Levente@Netlock]]

Full Incident Report — Failure to file a preliminary incident report within 72 hours

Scope note (please read first). This report addresses a single compliance
failure: NETLOCK's failure to publicly disclose a known compliance incident
within the 72-hour window set by the CCADB Incident Reporting Guidelines. The
underlying technical incident (an OCSP responder returning "unknown" for a
validly-issued certificate) is tracked in Bug 2051459, and the related
failure to acknowledge a Certificate Problem Report within 24 hours is tracked
in Bug 2052541. Those have distinct root causes and remediations. This
report is filed separately, as requested in Bug 2051459 Comments #5 and #13,
because the disclosure-timeliness failure has its own distinct root cause and
remediation.

Summary

• CA Owner CCADB unique ID: A000039
• Incident description: NETLOCK became aware, through its own monitoring, of a
  compliance incident (the OCSP responder condition in Bug 2051459) on
  2026-06-16. Under the CCADB Incident Reporting Guidelines a preliminary
  incident report was due within 72 hours — i.e. by 2026-06-19. NETLOCK did
  not file any public report by that deadline. No public disclosure of the
  incident occurred until 2026-07-03, when NETLOCK posted its report in Bug
  2051459 — and that bug was itself opened by a third party on 2026-06-29, not by
  NETLOCK. The 72-hour preliminary-report obligation was therefore missed, and the
  first disclosure of any kind was approximately 14 days late relative to the
  point at which NETLOCK became aware of the incident.
• Timeline summary:
  • Awareness of the underlying incident: 2026-06-16 20:12 UTC
  • 72-hour preliminary-report deadline: 2026-06-19 20:12 UTC
  • Non-compliance start (deadline missed): 2026-06-19 20:12 UTC
  • Non-compliance end (first public disclosure, Bug 2051459 Comment #1): 2026-07-03 14:34 UTC
  • Non-compliance identified: 2026-07-06 (raised by a third party in Bug 2051459 Comment #5)
• Relevant policies:
  • CCADB Incident Reporting Guidelines — requirement to file a preliminary
    incident report within 72 hours of becoming aware of an incident.
  • Mozilla Root Store Policy §2.4 ("Incidents"), which requires incident
    handling in accordance with the CCADB Incident Reporting Guidelines.
• Source of incident disclosure: Third Party Reported

Impact

• Total number of certificates: N/A. This is a disclosure-timeliness failure,
  not a mis-issuance. No certificate was mis-issued and none requires revocation as
  a consequence of this incident.
• Total number of "remaining valid" certificates: N/A.
• Affected certificate types: N/A.
• Incident heuristic: N/A.
• Was issuance stopped in response to this incident, and why or why not?: No.
  The failure is in our disclosure process, not in certificate issuance; halting
  issuance would not have remediated it.
• Analysis: For the duration of the non-compliance (2026-06-19 20:12 UTC to
  2026-07-03 14:34 UTC, approximately 13 days and 18 hours) the ecosystem had no
  public disclosure from NETLOCK of an incident NETLOCK already knew about. The
  practical harm of a late disclosure is that root-program members and relying
  parties are deprived of timely information; in this case the underlying
  condition had already been remediated on 2026-06-17, but that does not reduce
  the disclosure obligation, and we do not treat the outcome as harmless.

Timeline

All times are in UTC.

Date/Time (UTC)	Event
2026-06-16 20:12	NETLOCK identifies the OCSP responder non-compliance through its own monitoring-analysis activities (Bug 2051459). Awareness — the 72-hour clock starts.
2026-06-17 02:41	The underlying OCSP condition is remediated (Bug 2051459). The incident is handled internally as an operational fix; it is not recognised as a reportable incident, and no disclosure clock is started.
2026-06-19 20:12	72-hour preliminary-report deadline passes with no public report. Non-compliance begins.
2026-06-29 00:29	A third party opens Bug 2051459 concerning the OCSP incident. NETLOCK locates it through its own periodic review of Bugzilla activity rather than through an assignment or notification.
2026-07-03 14:34	NETLOCK posts its first public report in Bug 2051459 Comment #1. First disclosure — non-compliance ends (~14 days after awareness).
2026-07-06	A third party (Bug 2051459 Comment #5) identifies that the 72-hour preliminary-report obligation was missed and that it must be accounted for as its own matter.
2026-07-15 / 2026-07-27	The point is reiterated (Comment #13); NETLOCK acknowledges it will file a separate report (Comment #14).

Related Incidents

Bug	Date	Relationship
Bug 2013400	resolved FIXED 2026-04-17	Directly on point. A prior NETLOCK late-disclosure incident whose remediation included "automated 72-hour countdown alerts with escalation" computed from time of first awareness. The recurrence documented here shows that control starts its countdown only once an incident is logged in our tracking system, and this condition was never entered there — see Root Cause Analysis.
Bug 2051459	2026-06-29	The underlying OCSP responder incident. This disclosure-timeliness failure is the failure to report that incident on time.
Bug 2052541	2026-06-30	The related failure to acknowledge a Certificate Problem Report within 24 hours, arising from the same event. Tracked separately (distinct root cause: intake spam-filtering and routing).

Root Cause Analysis

Contributing Factor 1 (primary): our incident-disclosure process is triggered by inbound external reports, not by our own detections.

• Description: When NETLOCK detects a compliance-relevant condition through its
  own monitoring and remediates it, there is no defined step that classifies the
  condition as a reportable incident and starts the 72-hour disclosure clock.
  The condition on 2026-06-16 was detected, remediated on 2026-06-17, and closed
  as an operational fix. Disclosure was, in practice, only triggered when a third
  party opened Bug 2051459 on 2026-06-29.
• Detection: The gap was surfaced when a third party pointed out the missed
  deadline (2026-07-06), not by an internal control.
• 5 Whys: report late → no report was started at awareness → awareness of the
  technical issue did not translate into recognition of a reportable incident →
  our process only recognises incidents that arrive as external reports/tickets →
  self-detected findings are handled as operational work and never enter the
  disclosure workflow.

Contributing Factor 2: the Bug 2013400 control only starts once an incident is logged, and this condition was never logged.

• Description: The control built after Bug 2013400 computes the 72-hour
  countdown from the "time of first awareness" once an incident is entered into
  our incident-tracking system, and was deployed to production on 2026-03-23.
  Its limitation is upstream of the countdown: it depends on a condition first
  being recognised and entered as a reportable incident. This condition was
  detected on 2026-06-16, remediated on 2026-06-17, and handled as an operational
  fix — it was never entered into the incident-tracking system, so the
  awareness-based countdown never started.
• Interaction: Contributing Factor 2 shares the root cause of Contributing
  Factor 1: a self-detected condition was never converted into a logged, reportable
  incident. The 2013400 control operates only after that conversion step, so it
  could not catch this case.

Contributing Factor 3: no interim-update discipline.

• Description: Once the reporting obligation was in motion, our communications
  on this matter also slipped past the cadence we had committed to. We did not post
  interim "in progress / ETA" updates when a full answer was not yet ready, which
  compounds the appearance and the substance of delayed disclosure.

Lessons Learned

• What went well: The underlying technical condition was remediated quickly
  (within ~6.5 hours of detection). Once the disclosure obligation was raised, we
  accepted it without disputing that it was owed.
• What didn't go well: We treated a self-detected, compliance-relevant
  condition as routine operational work and never entered it into a disclosure
  workflow (Contributing Factor 1). The control from Bug 2013400 that should have
  prevented a repeat did not cover self-detected incidents (Contributing Factor 2).
  We then compounded the delay by not posting interim updates (Contributing Factor 3).
• Where we got lucky: A third party opened Bug 2051459, which is ultimately
  what caused us to disclose. We cannot rely on external parties to trigger our own
  disclosure obligations.

Action Items

#	Action Item	Kind	Corresponding Root Cause	Evaluation Criteria	Target	Status
1	Establish a reportability gate at the point of internal detection: every internally-detected, compliance-relevant condition is assessed for reportability and, if reportable, immediately starts the 72-hour disclosure countdown — the same workflow used for externally-filed tickets.	Prevent	CF1	A self-detected incident injected in a test/tabletop exercise starts the 72-hour countdown without an external ticket.	2026-09-30	In Progress
2	Extend the Bug 2013400 automated 72-hour countdown/escalation control so it triggers on internal incident detections, not only on externally-filed Bugzilla tickets assigned to our component.	Prevent	CF2	The countdown fires for a self-detected incident in test; alerting/escalation verified.	2026-09-30	In Progress
3	Adopt an interim-update discipline: whenever a report or a requested answer will not be ready on the committed cadence, post an interim status with an ETA rather than going silent.	Mitigate	CF3	Documented in our incident-handling procedure and applied on all open incident bugs.	2026-08-31	In Progress

(Action Item 1 is the same preventive control tracked as Action Item 5 in Bug
2051459; it is repeated here because it is the primary remediation for this
incident. Owners: NETLOCK Compliance team, with PKI Operations.)

Appendix

N/A. This incident did not result in any mis-issued or revocable certificate.