1734011 - Support tags in Glean metrics and pings

Status: RESOLVED FIXED

(Data Platform and Tools :: Glean: SDK, enhancement, P3)

Description

[William Lachance (:wlach)]

Preliminary feedback from Firefox for Android product management and data science is that the "tags" we added to glean-annotations (original proposal) are "useful". They have a couple of advantages over the existing metric categories:

1. Easy to change as you go (since they aren't represented in BigQuery).
2. Easy to represent in a visually prominent way in tools like the Glean Dictionary (we could also add them to GLAM).
3. Where it makes sense, you can have more than one of them attached to a single metric.
4. You can document what each of them means (for Firefox for Android, we linked back to the GitHub label associated with them).

However, adding these tags is currently a somewhat high friction process, involving a manual pull request against the glean annotations repository for every metric:

https://github.com/mozilla-mobile/fenix/wiki/Add-a-Glean-Annotation-for-an-event

In the original proposal for adding these annotations, we considered the possibility of moving the data back into the metrics files once we've evaluated that the tags are useful and we have a way forward that makes sense. I believe this time has arrived to take a serious look at this.

In the current implementation of glean annotations, there are tags with both a label (e.g. Autofill) and a markdown-based description (e.g. Address and Credit Card autofill. Corresponds to the [Feature:Autofill](https://github.com/mozilla-mobile/fenix/issues?q=label%3AFeature%3AAutofill) label on GitHub.). You can see a full list of the tags for Fenix on GitHub.

The simplest way forward is probably to add a tags.yaml file (opt-in) to allow Glean applications to define tags that make sense, and then to allow each metric to define the tags that apply to it.

Before implementing, let's do a quick survey to verify how useful this feature has been to various people throughout the organization and see if there's anything more we can do.

Comment 1

[Jan-Erik Rediger [:janerik]]

Thanks wlach for this experiment and the writeup.
Moving it into Glean seems reasonable if there's actual use for it.
I think the tags.yaml approach seems good, we can discuss details when we get to it (e.g. whether tags: is directly under the metric definition or we introduce a metatdata: key, as we might also have other metadata outside consumers might want in the future).

You said there should be a survey to verify usefulness.
Will you take this on and if so do you already have a timeframe in mind?

Comment 2

[Chris H-C :chutten]

Will the PR-based flow be replaced? There might be organizational tags that don't become apparent until analysis, so it might be worth considering post-instrumentation flows for adding tags.

Comment 3

[William Lachance (:wlach)]

(In reply to Chris H-C :chutten from comment #2)

Will the PR-based flow be replaced? There might be organizational tags that don't become apparent until analysis, so it might be worth considering post-instrumentation flows for adding tags.

It should be possible to augment the set of tags provided in the application with ones in glean-annotations. Note that there has not been much activity on adding them from people outside of Fenix engineering (and myself, but I don't think I count)

Comment 4

[William Lachance (:wlach)]

Survey results:

• 9 responses: 4 from Data Science, 1 from Fenix product management, 4 from Fenix software/product engineering
• 6 said "very useful", 3 said "somewhat useful" (2 of the "somewhat useful" answers were from Fenix software engineering, who I think do less analysis currently)
• The Fenix engineers don't seem to love the current process of manually adding annotations to glean-annotations, which seems to be an argument for implementing this bug.
• Full results here: https://docs.google.com/spreadsheets/d/1dOBjQ-YMM5POZHVc33UelywFFoS6cOerhilNHY2-Sjg/edit?resourcekey#gid=1164012614

Freeform feedback from survey:

• I will be great if we could reduce the developer time, for adding new tags, as at the moment we have to add the ping on the metrics.yaml and them in separate repository we have to add the tag
• Tags are already perfect but I'd love more clear descriptions of events/metrics.
• It is hard to remember to add them (since it is a new part of our process and lives in a separate repository). I have been trying to think of ways to make it clear for the engineers implementing new telemetry that this step can't be skipped..
• This works as best as I can imagine it could right now, the set of categories available so far has contained everything that I have tried to look for.
• I don't use Glean dictionary a lot. When I do I usually search for the category (ex: bookmarks_management) since I work on the metrics.yml file. This might be something more useful for Data Science team.

Comment 5

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/glean_parser/pull/406

Comment 6

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/probe-scraper/pull/358

Comment 7

[William Lachance (:wlach)]

I wrote up a quick proposal on my implementation strategy here: https://docs.google.com/document/d/1jtshKEf6sAS_6LA-_dBXg7OcW8-M5xc6M_MiyAr0LEw/edit#

Just want to make sure we get the details right before asking for review, since this is something that we're going to be stuck with for a while.

Comment 8

[William Lachance (:wlach)]

I think we're good here for the initial implementation. Anything else can be done as followups.