1651424 - Gather build telemetry using Glean

Status: RESOLVED FIXED

(Firefox Build System :: General, task)

Description

[Mitchell Hentges [:mhentges] 🦀]

Rather than using hand-rolled telemetry, we want to piggyback off of the good work of the glean folks.

Note that glean requires Python 3, so this will remove our visibility into mach invocations for Python 2 commands, like ./mach test.

Comment 1

[Mitchell Hentges [:mhentges] 🦀]

As discussed with Ricky, this ticket won't include the deprecation of the old build telemetry, but rather this telemetry will be gathered in addition to the existing system.

Comment 2

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1651424: Install glean in bootstrap

Installs glean to the currently-active python installation during bootstrap, whether
that be a virtualenv or the system instance of python.

Comment 3

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1651424: Report build telemetry using Glean

In addition to the existing build telemetry, also gather the stats and
report with Glean. This new telemetry is reported in tandem with the existing
telemetry to allow testing and confidence before a full roll-out.

Additionally, Glean isn't compatible with Python 2, so the new telemetry only runs
on Python 3 mach commands.

Depends on D83012

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/cc6516a63bb5
Install glean in bootstrap r=firefox-build-system-reviewers,rstewart

Comment 5

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/cc6516a63bb5

Comment 6

[Mitchell Hentges [:mhentges] 🦀]

Glean is now installed, but we still need to report the data - more work to do yet!

Comment 7

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1651424: Bump glean version to 32.3.0

Improves glean performance.
Prior to this change, using Glean adds ~500ms to each mach run.
After this change, using Glean adds ~200ms to each mach run.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/398eb90cd742
Bump glean version to 32.3.0 r=firefox-build-system-reviewers,rstewart

Comment 9

[Mitchell Hentges [:mhentges] 🦀]

Depending on 1662130 so that try runs aren't clouded with unrelated test failures.

Comment 10

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/398eb90cd742

Comment 11

[BugBot [:suhaib / :marco/ :calixte]]

Since the status are different for nightly and release, what's the status for beta?
For more information, please visit auto_nag documentation.

Comment 12

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1651424: Bump glean version to 32.3.1

32.3.0 was missing Python wheels for macOS.

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6170aee18eb4
Bump glean version to 32.3.1 r=firefox-build-system-reviewers,rstewart

Comment 14

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/6170aee18eb4

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5ecacfbe4d0d
Report build telemetry using Glean r=firefox-build-system-reviewers,Dexter,rstewart

Comment 16

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/5ecacfbe4d0d

Comment 17

[Mitchell Hentges [:mhentges] 🦀]

Telemetry is being gathered with glean.
Though there's follow-up work, this is complete.