1654074 - Expose Glean telemetry to mach subcommands

Status: RESOLVED FIXED

(Firefox Build System :: General, enhancement)

Description

[Mitchell Hentges [:mhentges] 🦀]

• Add ability for subcommands to define and report custom metrics
• Add docs for how to setup and use telemetry as a subcommand
• Update Glean docs generation to use all registered metrics files

Comment 1

[Mitchell Hentges [:mhentges] 🦀]

Fortunately, it appears like Glean allows us to pick and choose which set of metrics we want to report at runtime.
So, this allows us to expose a correctly-scoped set of metrics to each subcommand. For example:

| mach (top-level)
| ================
| mach_metrics = glean.load_metrics("<mach's metrics.yaml>")
| report_mach_metrics(mach_metrics)
|-------------------------------------------------------------------
|>>| mach python-test (subcommand)
|>>| =============================
|>>| pt_metrics = glean.load_metrics("<python-test's metrics.yaml>")
|>>| report_python_test_metrics(pt_metrics)
|>>| ...
|-------------------------------------------------------------------
| mach_metrics.command.duration.stop()
| mach_metrics.command.success.set(<True|False>)
| pings.usage.submit() # Submits both mach_metrics and pt_metrics

Comment 2

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1654074: Make @CommandProvider validation more specific

@CommandProvider does parameter validation and collects information (such
as "pass_context") that will be needed by Registrar.

However, rather than just checking parameter length, we can make it more
specific and assert that the specific expected parameter ("context") exists.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4eaef350187e
Make @CommandProvider validation more specific r=firefox-build-system-reviewers,rstewart

Comment 4

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/4eaef350187e

Comment 5

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1654074: Publish glean handle to mach commands

Allows mach commands to define their own glean metrics with the metrics_path @CommandProvider parameter.
When metrics_path is defined:

• A metrics kwarg is provided to the decorated class. This metrics handle is a Glean instance,
  so Glean documentation should be consulted for usage
• When mach doc telemetry is run, metrics docs will be generated from all the registered metrics files.

Note: there was some consideration between making metrics_path a @CommandProvider or @Command parameter.
In the end, @CommandProvider seemed like a better fit because:

• Metrics seem to be more associated with the entire class than a specific command/method. This is because
  a class represents a "domain", and that domain may have different commands that have overlapping metrics.
  Accordingly, all the metrics should be defined once as available to the entire class.
• Currently, @Command methods only take parameters that map one-to-one with CLI arguments. It could seem
  inconsistent to have one exception: the metrics handle

Depends on D83572

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/92cacd2f7294
Publish glean handle to mach commands r=firefox-build-system-reviewers,rstewart

Comment 7

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/92cacd2f7294

Comment 8

[Mitchell Hentges [:mhentges] 🦀]

Comment on attachment 9167159 [details]
Bug 1654074: Make @CommandProvider validation more specific

ESR Uplift Approval Request

• If this is not a sec:{high,crit} bug, please state case for ESR consideration: Needed to cleanly uplift https://phabricator.services.mozilla.com/D86255
• User impact if declined: None
• Fix Landed on Version: 81
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): It's a build-only patch
• String or UUID changes made by this patch:

Comment 9

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9167159 [details]
Bug 1654074: Make @CommandProvider validation more specific

Needed to better support running mach on newer macOS releases. Approved for 78.8esr.

Comment 10

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr78/rev/d90ed108cfc9