1822132 - Look into spike of Glean.js glean.restarted errors

Status: RESOLVED FIXED

(Data Platform and Tools Graveyard :: Glean Platform, task, P3)

Description

[Bruno Rosa [:brosa]]

We are seeing a spike of glean.restarted errors in the Glean.js monitoring dashboard. We want to determine the cause of this issue and see what we need to do to fix it.

Dashboard - https://sql.telemetry.mozilla.org/dashboard/glean-js-monitoring

Comment 1

[Bruno Rosa [:brosa]]

Findings

VPN is currently using both Glean and Glean.js (info courtesy of :brizental). I tweaked the query that the Glean.js dashboard is using to pull data to group the rows by telemetry_sdk_build and all of the versions being shown are 52.0.0+. Since Glean.js hasn't had a version over 1.0.0, this means that these errors are coming from Glean.

The error we are seeing looks like it could only come from Glean, as Glean.js doesn't report this same error. More specifically in event_database/mod.rs.
Glean - https://github.com/search?q=repo%3Amozilla%2Fglean%20ErrorType%3A%3AInvalidState&type=code
Glean.js - https://github.com/search?q=repo%3Amozilla%2Fglean.js%20ErrorType.InvalidState&type=code

Follow up

We should find a way to remove the Glean versions from the Glean.js dashboard. There are only 6 different Glean versions that show up in the table, so if we update the Glean.js dashboard query to filter any versions > 52.0.0 it should fix it since Glean.js versions are at max 1.x.x.

We also should determine the cause of these errors and see what we can do to stop them.

Comment 2

[Bruno Rosa [:brosa]]

I created a new ticket for updating the queries since the work for this ticket is done.

https://bugzilla.mozilla.org/show_bug.cgi?id=1836797