1650759 - Investigate memory usage of Glean

Status: NEW

(Data Platform and Tools :: Glean: SDK, defect, P4)

Description

[Jan-Erik Rediger [:janerik]]

So far we haven't paid much attention to Glean's memory usage.
We always considered the overhead of converting/copying back and forth between Rust and Java/Kotlin/Swift/Python/C# larger than any internal allocation anyway.

There's a couple of parts in Glean that definitely allocate:

• Common metric data (category, name, send_in_pings: all are heap-allocated strings)
• Ping payload
• Ping HTTP headers
• Test APIs (through JSON-formatted data)
• the database itself

Some of these are more or less static at compile time/load time (common metric data never changes[^1]).

It might be worth to:

• measure memory usage over the span of a normal application usage (across different platforms)
• determine alternative representations of quasi-static data
  • inlined strings or string-interning ([^2])

None of this requires immediate attention (as no one complained yet).

[^1]: we built labeled metrics dynamically on top of these though.
[^2]: Inspiration: https://fasterthanli.me/articles/small-strings-in-rust