1690253 - Implement the 'events' ping

Status: RESOLVED FIXED

(Data Platform and Tools Graveyard :: Glean.js, task, P1)

Comment 1

[Beatriz Rizental [:brizental]]

Upon implementation let's not forget to make sure this ping never contains the glean.restarted event.

See discussion: https://github.com/mozilla/glean.js/pull/448

Comment 2

[Alessio Placitelli [:Dexter]]

(In reply to Beatriz Rizental [:brizental] from comment #1)

Upon implementation let's not forget to make sure this ping never contains the glean.restarted event.

See discussion: https://github.com/mozilla/glean.js/pull/448

Note for our future selves: the specific discussion is here.

Comment 3

[Alessio Placitelli [:Dexter]]

Untaking, shifting priorities

Comment 4

[Beatriz Rizental [:brizental]]

The Glean JavaScript SDK is built to work on multiple JavaScript environment: websites, web extensions, Node.js and even QML apps.

Each of these environments have different requirements when it comes to ping scheduling and data collection. Even when looking at applications built for the same environment, the requirements can be very different e.g. Node.js applications can be both command line interfaces that run for seconds and web servers which run for days and months non-stop.

I do not believe the model implemented on the Rust based SDKs is best suited for the JavaScript SDK, where Glean's core defines scheduling of some out-of-the box pings that just work for all users. That is not flexible enough for the possibilities we want to serve on JavaScript-land.

With this in mind, I do not think there is space for an 'events' ping implementation on Glean.js.

Once the Glean.js is more widely adopted we may start to notice patterns of usage that we can then use as prompts to create some plugins for Glean.js that define metrics and pings for specific use cases e.g. Google Analytic's autotrack plugin.

We could also help users on scheduling pings, by allowing scheduling to be defined on the YAML files e.g. "sent every 30 minutes" or "sent once 300 events are collected". Then again, I believe it does not make sense to look into such possibilities until the library is more widely used and we understand user requirements better.

Until then I propose we close this as WONTFIX.

WDYT [jan-erik] and [:Dexter]?

Comment 5

[Alessio Placitelli [:Dexter]]

I think there's some room for implementing the events ping based on at least one of the criteria used to trigger the submission in the other Glean SDKs: the number of collected events.

For events going to the 'events' ping, we could provide a minimal 5-10 events buffer after which the events ping is submitted.

If we don't do at least that, then we need to disable the possibility of assigning things to the 'events' ping for the Glean JS SDK , since this would lead to data loss.

Comment 6

[Beatriz Rizental [:brizental]]

I think there's some room for implementing the events ping based on at least one of the criteria used to trigger the submission in the other Glean SDKs: the number of collected events.

I agree with that. However the number of events would vary per platform. 5-10 make sense for websites I guess, but not for web extensions.

...thinking...

I just remembered maxEvents is actually configurable is Kotlin, isn't it? In that case we could potentally implement the events ping in Glean.js and add maxEvents as a configuration option. WDYT?

Comment 7

[Alessio Placitelli [:Dexter]]

(In reply to Beatriz Rizental [:brizental] from comment #6)

I think there's some room for implementing the events ping based on at least one of the criteria used to trigger the submission in the other Glean SDKs: the number of collected events.

I agree with that. However the number of events would vary per platform. 5-10 make sense for websites I guess, but not for web extensions.

...thinking...

I just remembered maxEvents is actually configurable is Kotlin, isn't it? In that case we could potentally implement the events ping in Glean.js and add maxEvents as a configuration option. WDYT?

Yes, I remember us having something like that. I don't know if it's still around, but the reason why we had it is exactly the one we're talking about :) If we feel like this is a path forward, then yes, making the threshold configurable is the way to go IMHO.

Comment 8

[Beatriz Rizental [:brizental]]

Looks configurable to me: https://github.com/mozilla/glean/blob/main/glean-core/android/src/main/java/mozilla/telemetry/glean/config/Configuration.kt#L72. I'll wait for [jan-erik]'s confirmation though.

I agree that this is a better way to go, specially because users cannot rely on this "number of events" trigger even if they want to for a custom ping and it is a useful trigger for it.

Comment 9

[Jan-Erik Rediger [:janerik]]

It is configurable across all SDKs, you are right. We just never really advertised it actively and no one ever complained about the 500-limit either (then again: it's not hit that often I guess).

The other way the events ping is sent right now is on client inactivity, which I agree doesn't make sense for websites or web extensions.
Luckily that's nicely behind the client activity API, so even if we would want it it's up to the application to use it.
(And on startup but that's a solved thing on Glean.js already).

While initially I agreed with Bea of not building it at all, I think I'm convinced that the effort to provide the maxEvents-based events ping is low enough that it's worth doing now.

Comment 10

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/glean.js/pull/1267

Comment 11

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/glean/pull/1988