2031551 - Mozdetect CDF error preventing alerting: RuntimeWarning: invalid value encountered in scalar divide

Status: RESOLVED FIXED

(Testing :: Performance, defect, P1)

Description

[Greg Mierzwinski [:sparky]]

There's an issue that intermittently pops up when run change detections in mozdetect with the CDF squared technique:

/home/sparky/mozilla-source/mozdetect/src/mozdetect/detectors/cdf_squared.py:31: RuntimeWarning: invalid value encountered in scalar divide
  * ((merged_hist["diff"].sum()) / abs(merged_hist["diff"].sum()))

Specifically from this line: https://github.com/gmierz/mozdetect/blob/master/src/mozdetect/detectors/cdf_squared.py#L30

For some reason the diff sum there is 0. So far, I've found that it seems to happen when there's a day with no data near the start of the data. I'm able to reproduce it with this script:

import mozdetect

from mozdetect.telemetry_query import get_metric_table

ts_detectors = mozdetect.get_timeseries_detectors()
cdf_ts_detector = ts_detectors["cdf_squared"]

print("Querying for data...")
data = get_metric_table(
    "network_tls_handshake", "Windows", process="content", use_fog=True, from_build_date="2025-09-13")
timeseries = mozdetect.TelemetryTimeSeries(data)

print("Running detections...")
ts_detector = cdf_ts_detector(timeseries)
detections = ts_detector.detect_changes()

for detection in detections:
    print(detection.location)

From 09-05 to 09-13, I'm able to reproduce it. The date 09-14 has no data, and using 09-14 or greater works fine here. It looks like there's an issue in this function because when I change the number of days to average it starts to work with dates like 09-10 for a multiday average of 3: https://github.com/gmierz/mozdetect/blob/2882f7a2695b752ce16d3c05bd02e3587d22bc52/src/mozdetect/timeseries_detectors/cdf_squared.py#L260

This issue is severe because if someone gets unlucky and adds a probe near a date that has this issue of missing data, it will not produce alerts.

Comment 1

[Greg Mierzwinski [:sparky]]

Fixed here: https://github.com/gmierz/mozdetect/commit/c7783cddfa6879bc6fcf943a8de85df2fafa555f