1858640 - Generating Glean files takes a large part of the time in no-op builds

Status: RESOLVED FIXED

(Toolkit :: Telemetry, defect)

Description

[Florian Quèze [:florian]]

Here's a profile of a no-op build on my M1 Max macbook pro: https://share.firefox.dev/3PUojyb

Here's the list of glean files that were generated during that no-op build and how long it took to generate them: https://share.firefox.dev/3RTC3fm

Note that the GleanMetrics.h and GleanPings.h files are generated twice, once during the export phase, and one during the compile phase.

Comment 1

[Chris H-C :chutten]

If it's a no-op build, then the GeneratedFile's deps shouldn't have been changed so the script to generate e.g. GleanPings.h should never have been called, right? (Asking :nalexander, who's shown some facility with this directive in the recent past)

Florian: Is this new (since, say, bug 1852869)?

Comment 2

[Mike Hommey [:glandium]]

REBUILD_CHECK=1 ./mach build shows:

 0:00.72 Rebuilding toolkit/components/glean/.deps/EventGIFFTMap.h.stub because metrics_index.py was removed
 0:00.74 Rebuilding toolkit/components/glean/.deps/HistogramGIFFTMap.h.stub because metrics_index.py was removed
 0:00.76 Rebuilding toolkit/components/glean/.deps/ScalarGIFFTMap.h.stub because metrics_index.py was removed
 0:00.79 Rebuilding toolkit/components/glean/api/src/.deps/factory.rs.stub because metrics_index.py was removed

This comes from https://searchfox.org/mozilla-central/rev/7d77ff808f8407a3e4fc0911779da446c050f9ee/toolkit/components/glean/build_scripts/glean_parser_ext/run_glean_parser.py#27

Comment 3

[Mike Hommey [:glandium]]

Attachment: Bug 1858640 - Remove metrics_index.py from the manual deps.

It is imported by the script, so will be picked automatically, while the
manual dep doesn't currently contain the full path, and actually points
to a file that doesn't exist, making the script run every time.

Comment 4

[Florian Quèze [:florian]]

(In reply to Chris H-C :chutten from comment #1)

Florian: Is this new (since, say, bug 1852869)?

Yes, here's a profile with the patch from bug 1852869 reverted: https://share.firefox.dev/3FlMFvT

(FYI, if you want to generate similar profiles to verify a patch, you'll need the patches from bug 1850647, currently on autoland)

Comment 5

[BugBot [:suhaib / :marco]]

Set release status flags based on info from the regressing bug 1852869

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7aa62c4f81e6
Remove metrics_index.py from the manual deps. r=nalexander

Comment 7

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/7aa62c4f81e6