1745283 - Support "global" tags in metrics.yaml files

Status: RESOLVED FIXED

(Data Platform and Tools :: Glean: SDK, task, P1)

Description

[William Lachance (:wlach)]

(based on discussions with :chutten)

For Firefox on Glean, we'd like to be able to specify tags for the bugzilla product/component that a metric belongs to (which is conveniently specified by the location of the metrics.yaml file). To avoid having to specify this information metric-by-metric, we'd like to be able to specify tags inside a metrics.yaml file. E.g. for gfx/metrics.yaml we might want:

$schema: moz://mozilla.org/schemas/glean/metrics/2-0-0

$tags:
  - "Core :: Graphics"

paint:
  build_displaylist_time:
    type: timing_distribution
    description: >
      The time to build a Gecko display list.
    bugs:
      - https://bugzilla.mozilla.org/show_bug.cgi?id=1728423
    data_reviews:
      - https://bugzilla.mozilla.org/show_bug.cgi?id=1728423
    data_sensitivity:
      - technical
    notification_emails:
      - [e-mail]
      - [e-mail]
    expires: never
    telemetry_mirror: PAINT_BUILD_DISPLAYLIST_TIME

This way we can avoid setting product/component information for every metric in m-c, which is likely to be tedious and prone to user error.

I'll take this one. This bug will be about adding support for the concept to glean_parser: we can figure out the Firefox pieces after that's done.

Comment 1

[Chris H-C :chutten]

In conversation with :mdboom I learned that $key-named properties at the file level is a namespace staked out by JSONSchema (See rfc for their uses of $schema, $ref, $vocabularies, and more).

So we may wish to, for bits and bobs that are added by us, use a different pattern to avoid stomping on toes. Maybe $$key or some other ascii symbol?

Comment 2

[Jan-Erik Rediger [:janerik]]

Hm, I wonder if instead of coming up with our own way of doing that we can use already-existing YAML features such as anchors?
E.g. I use YAML anchors in a test file to reduce the amount of typing.
It still requires manual annotation of each metric though and because we treat every top/second-level key as a category/metric this needs to be either on the first metric or a dummy metric needs to be used.

Comment 3

[Alessio Placitelli [:Dexter]]

(In reply to Jan-Erik Rediger [:janerik] from comment #2)

Hm, I wonder if instead of coming up with our own way of doing that we can use already-existing YAML features such as anchors?
E.g. I use YAML anchors in a test file to reduce the amount of typing.
It still requires manual annotation of each metric though and because we treat every top/second-level key as a category/metric this needs to be either on the first metric or a dummy metric needs to be used.

Anchors seem to be a useful path also used by regrets-reporter

Comment 4

[Chris H-C :chutten]

I don't know that anchors will encourage folks to add their own tags, though, as if we

fake.category:
  fake.name: &base
    tags: [ "Data Platform and Tools :: Glean: SDK"]

Then later

real.category:
  real.name:
    >>: *base
    tags:
      - "My Tag"

Will override &base's tags and replace it.

(( Took me a while to gather all the specs together, so here they are: YAML anchors are specced here (for the ref), and here (for the alias). The >> spreading operator is a language-independent type called a "Merge Key" and is specced here (it's optional to the spec). ))

We could enforce this as part of the build to ensure that the tags of individual metrics all contain the correct tag given their file's location. But the ergo for using tags starts to suck.

I still prefer the file-level annotation.

Comment 5

[William Lachance (:wlach)]

I think I agree with :chutten that a glean_parser wide solution is better, but using YAML's built in features is a definite improvement over manually specifying the tags each time. e.g.:

test:
  run:
    type: uuid
    description: >
      ID of the current test run.
    metadata:
      tags: &default_tags
        - "Data Platform and Tools :: Glean: SDK"

  name:
    type: string
    description: >
      Name of the current test.
    metadata:
      tags: *default_tags

It might be worth considering using that for an initial implementation and then tackling this bug as a followup to improve the ergonomics later.

Comment 6

[Jan-Erik Rediger [:janerik]]

Setting a low-priority (to move it out of our incoming list)

Comment 7

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/glean_parser/pull/435

Comment 8

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/glean/pull/1927

Comment 9

[Chris H-C :chutten]

chutten merged PR #435: "bug 1745283 Support file-level tags" in 8e2c38b.

Comment 10

[Jan-Erik Rediger [:janerik]]

badboy merged PR #1927: "bug 1745283 - Document file-level tags" in a4a5f14.