1597325 - Python bindings: Provide a way to send "built-in" pings for command line tools

Status: NEW

(Data Platform and Tools :: Glean: SDK, defect, P3)

Description

[Michael Droettboom [:mdroettboom]]

There is currently no way to send "built-in" pings (baseline, metrics, events) from the Python bindings. This will be required to automated end-to-end tests.

There are a few alternatives:

1. Make Glean.send_pings_by_name public. This will allow any ping to be sent on command.

2. Add APIs that "simulate" app-level events, e.g. Glean.handle_background_event, which would trigger the sending of the baseline and events pings as if the user went to background. This means the tests are written as if automated an app, and which pings get sent remains an implementation detail.

3. Add a send_all_pings method, which would send all registered pings immediately.

Comment 1

[Alessio Placitelli [:Dexter]]

(In reply to Michael Droettboom [:mdroettboom] from comment #0)

There is currently no way to send "built-in" pings (baseline, metrics, events) from the Python bindings. This will be required to automated end-to-end tests.

I don't think this should be required. End to end testing could just use and submit custom pings for the use case highlighted in the design document. All in all, this is for testing that Glean SDK-generated data can be queried/answer questions at the end of the pipeline: doesn't really need a Glean-defined ping.

If application behaviour is important or required, then I think we should probably find a way to puppet existing sample applications or real product.

Comment 2

[Raphael Aurich [:raphael] UTC+01:00]

1. Would require knowledge of the Glean built-in pings from the consumers of the Python SDK. This is currently not required for existing products using Glean, correct?

2. This would be OK for the proof-of-concept for the end-to-end tests, but probably not idiomatic for other Python applications, which don't have a concept of going-to-background as for example command-line tools

3. This API would be ideal for the proof-of-concept for the end-to-end tests since it doesn't require knowledge of the Glean built-in pings, but makes sure we execute a critical code path in glean-core without faking events

Comment 3

[Michael Droettboom [:mdroettboom]]

I think where we ended up is to not do this for MVP, but to reinvestigate it when these bindings will be used for end-user applications.

Comment 4

[Michael Droettboom [:mdroettboom]]

In light of inclusion in mozregression, mozphab and soon mach, we should probably have a plan for this.

I think it's important to note that unlike Kotlin/Android and Swift/iOS, we anticipate Python being used in a few different very different contexts -- command line tools, web servers, integration tests. The naming proposal introduces the concept of "application lifecycle type" for these different things. I'm proposing limiting the scope of this bug to just the "command line tool" application type, since that is the most pressing need at the moment.

Comment 5

[Michael Droettboom [:mdroettboom]]

Attachment: Proposal for non-mobile application lifecycle types

Comment 6

[Chris H-C :chutten]

Comment on attachment 9133182 [details]
Proposal for non-mobile application lifecycle types

The one concrete section (CLI) seems like a good approach for now. I think I'd like to see two additions:
* an explanation in the same style of the current mobile gui app lifecycle type for comparison with the CLI and server ones
* a list of the lifecycle types we know of at present and speculate about for the future.

Comment 7

[Jan-Erik Rediger [:janerik]]

Comment on attachment 9133182 [details]
Proposal for non-mobile application lifecycle types

Left a longer comment in the doc. tl;dr: I'm not sold on the pre-defined types and the application telling us. I'm not outright rejecting the proposal though, I just like that point being discussed.

Comment 8

[Travis Long [:travis_]]

Comment on attachment 9133182 [details]
Proposal for non-mobile application lifecycle types

Since we really only have two events that Glean cares about right now, I think I lean more towards the flexible approach proposed by Jan-Erik: if we expose the right hooks, then applications are free to define their own lifecycle start and stop events and we aren't stuck with the burden of wiring in every lifecycle we can think of now, and every new one that pops up.

Comment 9

[Chris H-C :chutten]

(In reply to Travis Long [:travis_] from comment #8)

Comment on attachment 9133182 [details]
Proposal for non-mobile application lifecycle types

Since we really only have two events that Glean cares about right now, I
think I lean more towards the flexible approach proposed by Jan-Erik: if we
expose the right hooks, then applications are free to define their own
lifecycle start and stop events and we aren't stuck with the burden of
wiring in every lifecycle we can think of now, and every new one that pops
up.

It does provide yet another way for embedding applications to fail to do this "properly", though. Are we okay if, e.g., applications opt out of hooking these up at all? Call them in the 'wrong' order?

Comment 10

[Alessio Placitelli [:Dexter]]

Comment on attachment 9133182 [details]
Proposal for non-mobile application lifecycle types

Clearing ni? - please do flag me again once new efforts will be poured in this area