1651107 - Establish a best practice for metrics.yaml and pings.yaml in FOG

Status: RESOLVED FIXED

(Toolkit :: Telemetry, task, P1)

Description

[Chris H-C :chutten]

glean_parser and probe-scraper both would happily consume multiple metrics.yaml and pings.yaml files in mozilla-central. But what should we recommend? Should each subfolder of toolkit/components have its own? Should we put everything in a single place? Should we allow the Wild West and document what needs to be done to add it to the build system and probe-scraper?

This probably doesn't need a proposal as much as it needs this bug as a space for discussion, and to track the work to add the resulting decision to the documentation.

Comment 1

[Chris H-C :chutten]

A conversation with Raphael forced me to expand some thoughts about why we might prefer components to have their own .yaml files:

At present, ownership and bug filing is determined by the location in the tree of the files being changed (see for instance FOG's moz.build). Tooling like Phabricator's "Herald" are also based on file location.

So if we manage to keep non-fog-metrics' metrics.yamls outside of toolkit/components/glean, these processes, conventions, and tools will work more smoothly because toolkit/components/glean/metrics.yaml won't be shared across the entire product. We could require Firefox Telemetry Team review to any files changed within toolkit/components/glean without being included in every new metric in every component within the project.

Downsides of course include:

• What do people do if they're the first one adding FOG instrumentation to a component? How will they know there's a guide to follow? They'll need to add their metrics.yaml to toolkit/components/glean's build step somehow, which will require a Firefox Telemetry Peer review anyway.
  • We can't do much about the guide except putting a nastygram atop toolkit/components/glean's metrics.yaml and just socializing the practice. Plus it's self-reducing.
• Never having a single place to find where all metrics live
  • That's what the Probe Dictionary's for.

Comment 2

[Chris H-C :chutten]

I made it a document anyway: https://docs.google.com/document/d/1jEmPVLVxeJudxNaLd4_O2b-dtpj7JxPzy7Ik04NKc3o/edit#

Don't worry, it's a short one (1.5 pages)

Please provide your feedback by next Friday, Aug 7. Feedback I'm especially interested in includes:

• More arguments for/against
• More possibilities other than "one, like Histograms.json" and "many, like android-components' metrics.yaml"
• Technical implementation challenges
• Non-technical implementation challenges

Comment 3

[Alessio Placitelli [:Dexter]]

(In reply to Chris H-C :chutten from comment #2)

I made it a document anyway: https://docs.google.com/document/d/1jEmPVLVxeJudxNaLd4_O2b-dtpj7JxPzy7Ik04NKc3o/edit#

I like the proposal and the idea. The one concern I have relates to the ecosystem level integration with the "probe-scraper". See my embedded comments.

Comment 4

[Jan-Erik Rediger [:janerik]]

Multiple files sounds like a good organizational structure. I left some comments in the doc.

Comment 5

[Michael Droettboom [:mdroettboom]]

Comments left in the doc. Happy where the consensus is trending (multiple files, single index) right now.

Comment 6

[Chris H-C :chutten]

On #build :froydnj confirmed that "many metrics.yaml in the tree" should be fine. He also mentioned an even more ergonomic implementation for this where each component would identify their metrics.yaml by adding a directive to their moz.build file (maybe GLEAN_METRICS += [...]). This would require some glue code akin to XPIDL_SOURCES to ensure it runs the resulting task at the right time with the full list of files, but it would make adding, removing, and moving definitions files truly self-serve.

(( It would complicate dealing with probe-scraper who might need to learn how to parse moz.build files or grep the tree for GLEAN_METRICS directives, but it could be magic ))

Something to think about for the future. For now, if we pursue multiple metrics.yaml files we can stick to a list in t/c/glean/moz.build or an flat-file index that it reads at build time.

Comment 7

[Chris H-C :chutten]

The proposal of "many metrics.yaml" is accepted. This bug is now about documenting the particulars in-tree.

For now it will be quite manual: new yamls will need to be manually added to t/c/glean/moz.build and probe-scraper's repositories.yaml. Future improvements like index files can be evaluated when/if they become necessary.

Comment 8

[Chris H-C :chutten]

Attachment: Bug 1651107 - Document best practices for Glean definitions files in-tree r?janerik!

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d94b928a953c
Document best practices for Glean definitions files in-tree r=janerik DONTBUILD

Comment 10

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/d94b928a953c