1907991 - Use the vendored `glean_parser` for `mach` and `build`

Status: RESOLVED FIXED

(Toolkit :: Telemetry, task, P1)

Description

[Chris H-C :chutten]

We're currently at v14.1.2. The upgrade to v14.2.0 will bring us fixed Kotlin (and Swift, but mozilla-central doesn't need that as badly) codegen for object metrics, as well as set us up for labeled_{custom|memory|timing}_distribution metric type support coming in bug 1907945

Comment 1

[Chris H-C :chutten]

Attachment: Bug 1907991 - Vendor glean_parser v14.2.0 r?janerik!

Also instruct mach to use the vendored glean_parser instead of the one it
would otherwise pull down from pip as part of its glean_sdk dep.

This unfortunately ties updates of mach's glean_sdk to any updates m-c gets
for its glean_parser. It's required to ensure ./mach build always uses the
vendored glean_parser (see bug 1908020).

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4f5fd55945a5
Vendor glean_parser v14.2.0 r=janerik,mach-reviewers

Comment 3

[Sandor Molnar[:smolnar]]

Backed out for causing build bustage

Backout link: https://hg.mozilla.org/integration/autoland/rev/93af9692fbff6336519b0e8ee2102caa40856b9a

Push with failures

Failure log -> Exception: According to "pip check", the current Python environment has package-compatibility issues

Comment 4

[Chris H-C :chutten]

Alex, got any ideas? Maybe it's that glean-parser 14.2 requires appdirs, diskcache, and pyyaml which aren't in the system python? I'll say that glean_parser's setup.py hasn't changed in seven months so these aren't new deps or anything.

Comment 5

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

We just need to add them the mach.txt as well, they're vendored already alongside glean_parser. This didn't occur for us locally since glean-sdk installs those deps in the mach site, but I'm guessing that's not happening in CI because glean-sdk isn't being installed.

vendored:third_party/python/appdirs
vendored:third_party/python/diskcache
vendored:third_party/python/PyYAML/lib/

Just add that and it should work.

I'll check to make sure there aren't any chained dependencies in those 3 that you'd also need to add.

Comment 6

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

I don't see any chaining dependencies in the lockfile, so it should be good after you add those three to mach.txt. Do a try run before pushing though, just in case.

Comment 7

[Chris H-C :chutten]

Will do. But first, I apparently need to fix rust codegen in glean_parser and cut a v14.2.1 which we'll need instead because as of the 16th we now have a direct rust consumer in-tree (hello crashreporter client).

Comment 8

[Chris H-C :chutten]

Ah, this places a hard dep on a Glean version with LabeledMetricData in it, so we'll not be able to bring it in ahead of time. It'll have to land with bug 1907945

Comment 9

[Chris H-C :chutten]

Attachment: Bug 1907991 - Require mach to use the vendored glean_parser r?TravisLong!

Comment 10

[Chris H-C :chutten]

With the glean_parser upgrade happening in bug 1907945, this is now just a "fix glean_parser deps to be sourced from the right place" bug

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/51ece15f6b4e
Require mach to use the vendored glean_parser r=TravisLong,mach-reviewers,ahochheiden

Comment 12

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/51ece15f6b4e

Comment 13

[Chris H-C :chutten]

Attachment: Bug 1907991 - Upgrade to glean_parser v14.3.0 r?TravisLong!