1626086 - Python: Provide a way to make sure pings are sent before application exits

Status: RESOLVED FIXED

(Data Platform and Tools :: Glean: SDK, defect, P1)

Description

[William Lachance (:wlach)]

I noticed last Friday that if the application exits before the Glean thread in a python application has a chance to complete, a ping might not get sent. This can easily happen in mozregression or any command-line application where the ping is sent near the end of a process lifetime.

We should probably provide some mechanism to allow blocking application exit until things are sent (not sure if it should be enabled automatically or not).

We discussed this a bit on the Glean channel last Friday, one current hack you can try to get this effect is:

glean._dispatcher.Dispatcher._task_worker._queue.join()

... but that seems quite liable to break.

Comment 1

[Alessio Placitelli [:Dexter]]

(In reply to William Lachance (:wlach) (use needinfo!) from comment #0)

We should probably provide some mechanism to allow blocking application exit until things are sent (not sure if it should be enabled automatically or not).

The next iteration of the SDK will provide a bit more flexibility about the upload scheduling mechanism. This problem could be tackled as part of that bigger effort.

I'm not sure blocking on network automatically is a good idea, but maybe the python bindings could do something as the pingsender on Windows? i.e. when the application process is about to die (if we can detect that) we spin up an upload process.

Comment 2

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/glean/pull/798

Comment 3

[Michael Droettboom [:mdroettboom]]

I think this should be tackled in two parts:

• Add an atexit handler to just wait a while for the worker thread to complete. This will get handle most cases when things are working well and the network is up, etc. That's basically what PR 798 does.

• Move ping uploading (just the networking part) to its own process. This is the part that's most likely to have long delays. This could then be fired off at shutdown allowing the main process to exit quickly. I think this is mildly blocked by bug 1605078 to move the networking handling to Rust -- not truly blocked, but it will end up looking very different after that and maybe best not to build it twice.