1579146 - Provide a Rust API for the Glean SDK

Status: RESOLVED FIXED

(Data Platform and Tools :: Glean: SDK, enhancement, P3)

Description

[Alessio Placitelli [:Dexter]]

With more Mozilla components being written in Rust and with the Glean SDK itself being in Rust we should consider providing a Rust API.

Today, from the #glean Slack channel:

so, it turns out application-services might like to call into glean from rust code in the forseeable future, mostly for non-sync telemetry metrics (e.g. how long autocomplete_match takes, how long we spend waiting for various mutices, various counters, etc. obviously these would all go through data-review before being recorded).very broadly i see the following options:1. have our rust code dynamically load the libglean.so (or whatever you call it) and call it's FFI functions

• this is less work for us, but probably less desirable for you, depending on how churny your FFI layer is likely to be, potentially completely off the table -- and we'd need some way of detecting ABI-breaking changes to error out rather than hit memory unsafety even if your FFI is not churny (a version counter that gets incremented on ABI-incompatible changes would suffice here).2. a callback-based approach over the FFI, similar to how we proxy logging, network requests, etc.
• we'd probably make it so that just a single callback over the FFI was exposed, which passed a protobuf containing an enum, which the kotlin side would unpack, switch on, and call relevant glean functionality (having just one callback over the FFI for something like this isn't so bad, but a larger API gets painful fast because of how JNA's Callback class works).
• This is pretty tedious, but maybe some of the code could be autogenerated.3. reconsider megazording them together so that our rust code can talk directly

There are a few things we need to consider:

• handles for Glean objects are now stored on the Rust FFI layer;
• when using Glean in the same product from multiple languages, there must be a single Glean handle shared across all the languages.

Comment 1

[Alessio Placitelli [:Dexter]]

We had a meeting with the application-services team about this (notes here). We need to kick off a design document about the "nice rust API" and all the challenges connected about collecting Glean metrics across different languages to the same instance. Jan-Erik, any chance you could handle this?

Comment 2

[Jan-Erik Rediger [:janerik]]

As discussed we're gonna follow up with a design document and plans on how to get there. I'm going to lead that discussion.

Comment 3

[Jan-Erik Rediger [:janerik]]

Document: https://docs.google.com/document/d/1cY6Aa4jjp27NU2SB_BwXVKj2rscZKCKbEQIodpvM6Gw/edit
Not fully done yet.

Comment 4

[Jan-Erik Rediger [:janerik]]

Above document got a lot of feedback and improvements.
I'm still working on a small example, but with glean-preview out and review cycle closed, I consider this bug done!
Going to file a few follow up bugs for the implementation next.