1606645 - Errors related to experiments info are recorded only on the metrics ping

Status: NEW

(Data Platform and Tools :: Glean: SDK, defect, P4)

Description

[Michael Droettboom [:mdroettboom]]

As :raphael noticed when developing integration tests, when experiment info has an error (such as a too-long experiment name), that error is recorded in the metrics ping, but not in other ping types, even though the experiment info itself is recorded in every ping.

This, I believe, comes from an implementation detail, rather than a conscious decision. Experiments are recorded in the database on a "special" ping (glean_internal_info) that is never actually sent. Errors are always recorded to the metric's ping(s) and the metrics ping. [sorry about that sentence]. Therefore, we arrive at the visible behavior which is that these errors are only on the metrics ping.

Since the Python bindings don't currently have a way to send a metrics ping, there is no way to verify (externally) that experiments with errors record those errors somewhere.

I think we need to either:

1. Record experiment errors on every ping
2. Provide a way for the Python bindings to send and/or obtain a metrics ping

Comment 1

[Alessio Placitelli [:Dexter]]

(In reply to Michael Droettboom [:mdroettboom] from comment #0)

As :raphael noticed when developing integration tests, when experiment info has an error (such as a too-long experiment name), that error is recorded in the metrics ping, but not in other ping types, even though the experiment info itself is recorded in every ping.

This, I believe, comes from an implementation detail, rather than a conscious decision. Experiments are recorded in the database on a "special" ping (glean_internal_info) that is never actually sent. Errors are always recorded to the metric's ping(s) and the metrics ping. [sorry about that sentence]. Therefore, we arrive at the visible behavior which is that these errors are only on the metrics ping.

Yes, I think you're right. We should add the 'experiments' errors to all the pings, since experiments are reported in all the pings.