2068624 - Remove unused Legacy Telemetry Environment fields

Status: RESOLVED FIXED

(Toolkit :: Telemetry, task, P1)

Description

[Chris H-C :chutten]

Per my investigation in bug 2068267, quite a lot of the Legacy Telemetry Environment is unused by SQL or dataset derivation. If we remove them then we can ensure that no one suddenly starts using them and simplify somewhat the later tasks of working with owning teams to migrate the pieces that are in use.

Fields not in use include:

• build.applicationId (probably because it's shadowed by the common ping format's application.buildId)
• build.applicationName (probably because it's shadowed by the common ping format's application.name)
• build.vendor
• build.version (probably because it's shadowed by the common ping format's application.version)
• build.displayVersion (probably because it's shadowed by the common ping format's application.displayVersion)
• build.platformVersion (probably because it hasn't been different from the other versions for Quite Some Time)
• build.updaterAvailable (long superseded by environment.settings.update.enabled)
• settings.addonCompatibilityCheckEnabled (should've been removed along with active addons, come to think on it)
• settings.blocklistEnabled (ditto)
• settings.defaultPrivateSearchEngine
• settings.launcherProcessState
• settings.e10sEnabled (one of the first things I worked on at Moz back in 2015)
• settings.e10sMultiProcesses (2016)
• settings.fissionEnabled (long since enabled)
• settings.telemetryEnabled (you'd think it was a weird thing to communicate via telemetry, but this reported the pref which was about extended telemetry. If you don't know what that is, good, you don't need to.)
• settings.defaultPrivateSearchEngineData.*
• settings.intl.*
• settings.update.{autoDownload,background}
• settings.sandbox.* (well, it was used in January. But not since.)
• settings.services.* (there are other, better ways to figure out when and how accounts and sync are enabled)
• profile.{resetDate,firstUseDate} (honestly, profile.creationDate should also be unused because it is very flawed but folks are still using it as of May, so it gets to stick around)
• partner.{distributionVersion,partnerNames}
• system.{virtualMaxMB,isWowARM64,hasWinPackageId,winPackageFamilyName,appleModelId} (memoryMB and isWow64 have been used semi-recently, otherwise we could be rid of the whole lot)
• system.cpu.{pcount,mcount,ecount} (This one I'm confused about as I remember the efforts we exerted to add them)
• system.cpu.{l2cacheKB,l3cacheKB,speedMHz,extensions}
• system.os.{windowsUBR,installYear,locale,hasPrefetch,hasSuperfetch}
• system.hdd.{profile|binary|system}.*
• system.sec.*

(( system.gfx.* shall be removed as part of the broader GFX team's removal of legacy telemetry which can proceed now that https://mozilla-hub.atlassian.net/browse/DENG-10458 has been completed as part of https://mozilla-hub.atlassian.net/browse/DENG-11194. ))

Removing all these ought to make the remaining pieces a lot easier to keep in mind at once.

Comment 1

[Chris H-C :chutten]

Hey, :mstriemer, remember when we added all that intl stuff to the Legacy Telemetry Environment back in bug 1502921? I'm removing it! The data's all been in Glean metrics for a year (e.g. intl.available_locales), so it's time.

My question: is there a place perhaps inside LocaleService where we could put the calls setting these Glean metrics? The Glean calls will probably do better if they live closer to the code responsible for the data, and eventually we'll want to be rid of ol' TelemetryEnvironment.sys.mjs.

Comment 2

[Chris H-C :chutten]

Attachment: Bug 2068624 - Remove unused environment.build fields r?TravisLong

environment.build has the honour of being the only section of the
Environment that has required fields. And some of those required fields
must be of specific types and/or specific patterns.

This means that "removing" the seven unused fields results in:

• 2 set to "" to satisfy required strings
• 2 set to "00." to satisfy required strings with version patterns
• 1 set to null as vendor is required but can be null or string
• 2 actually removed (displayVersion and updaterAvailable)

And it only results in one decrement of the count, for updaterAvailable,
as the rest weren't ever mirrored in Glean inside the Environment
(because they were either already instrumented in client_info, or they
weren't useful data to mirror).

Comment 3

[Chris H-C :chutten]

Attachment: Bug 2068624 - Remove unused Legacy Telemetry environment.settings pieces r?TravisLong

Comment 4

[Chris H-C :chutten]

Attachment: Bug 2068624 - Remove unused Legacy Telemetry environment.services r?TravisLong

Comment 5

[Chris H-C :chutten]

In an abundance of caution I'll be leaving partner.{distributionVersion,partnerNames} alone. I'm still very certain that the values are unused, but these seem like the sort of distribution data that might have a data need sneak up on us unexpectedly. They can wait.

Comment 6

[Chris H-C :chutten]

Attachment: Bug 2068624 - Remove unused Legacy Telemetry environment.system fields r?TravisLong

Comment 7

[Chris H-C :chutten]

Attachment: Bug 2068624 - Remove unused Legacy Telemetry environment.system.cpu fields r?TravisLong

Comment 8

[Chris H-C :chutten]

Attachment: Bug 2068624 - Remove unused Legacy Telemetry environment.system.os fields r?TravisLong

Comment 9

[Chris H-C :chutten]

Attachment: Bug 2068624 - Remove unused Legacy Telemetry environment.system.hdd fields r?TravisLong

Comment 10

[Chris H-C :chutten]

Attachment: Bug 2068624 - Remove unused Legacy Telemetry environment.system.sec section r?TravisLong

Comment 11

[Chris H-C :chutten]

Attachment: Bug 2068624 - Remove unused Legacy Telemetry environment.profile fields r?TravisLong

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/96ff9e8d845e
https://hg.mozilla.org/integration/autoland/rev/773f60467832
Remove unused environment.build fields r=TravisLong
https://github.com/mozilla-firefox/firefox/commit/a34a04ddefbf
https://hg.mozilla.org/integration/autoland/rev/bed92fadb1f6
Remove unused Legacy Telemetry environment.settings pieces r=TravisLong
https://github.com/mozilla-firefox/firefox/commit/4486568ab6d4
https://hg.mozilla.org/integration/autoland/rev/f0d6502a83a1
Remove unused Legacy Telemetry environment.services r=TravisLong
https://github.com/mozilla-firefox/firefox/commit/2878a668e9d4
https://hg.mozilla.org/integration/autoland/rev/1985b7bdde64
Remove unused Legacy Telemetry environment.system fields r=TravisLong
https://github.com/mozilla-firefox/firefox/commit/f18845100acf
https://hg.mozilla.org/integration/autoland/rev/0277d85bada4
Remove unused Legacy Telemetry environment.system.cpu fields r=TravisLong
https://github.com/mozilla-firefox/firefox/commit/f08f25d1dea5
https://hg.mozilla.org/integration/autoland/rev/eb214e188fd4
Remove unused Legacy Telemetry environment.system.os fields r=TravisLong
https://github.com/mozilla-firefox/firefox/commit/aae565df3a9f
https://hg.mozilla.org/integration/autoland/rev/7abb0ca49d4e
Remove unused Legacy Telemetry environment.system.hdd fields r=TravisLong
https://github.com/mozilla-firefox/firefox/commit/af16f91c517d
https://hg.mozilla.org/integration/autoland/rev/1e04216fbbcc
Remove unused Legacy Telemetry environment.system.sec section r=TravisLong
https://github.com/mozilla-firefox/firefox/commit/9e4c0ceb5f3d
https://hg.mozilla.org/integration/autoland/rev/3057d1c37c80
Remove unused Legacy Telemetry environment.profile fields r=TravisLong

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/c467928188ba
https://hg.mozilla.org/integration/autoland/rev/02ad9ed63162
Revert "Bug 2068624 - Remove unused Legacy Telemetry environment.profile fields r=TravisLong" for causing xpcshells and TV failures
https://github.com/mozilla-firefox/firefox/commit/70c19d552196
https://hg.mozilla.org/integration/autoland/rev/e52c140b7e06
Revert "Bug 2068624 - Remove unused environment.build fields r=TravisLong"

Comment 14

[Silaghi Andreea]

Backed out for causing xpcshells and TV failures
Backout Link
Push with failures
Failure Log
Failure line TEST-UNEXPECTED-FAIL | toolkit/components/utils/test/unit/test_ClientEnvironment.js | testLiveTelemetry - [testLiveTelemetry : 119] Telemetry data is available - "undefined" == "158.0a1"

Comment 15

[Chris H-C :chutten]

Attachment: Bug 2068624 - Don't test now-removed pieces of the Legacy Environment r?beth

More generally, perhaps unconstrained access to the Legacy Telemetry
internals via ClientEnvironment should be discouraged.

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/6b9b468ae144
https://hg.mozilla.org/integration/autoland/rev/7cd63cd12484
Remove unused environment.build fields r=TravisLong
https://github.com/mozilla-firefox/firefox/commit/f60e9688f1b4
https://hg.mozilla.org/integration/autoland/rev/8fbb594d6368
Remove unused Legacy Telemetry environment.settings pieces r=TravisLong
https://github.com/mozilla-firefox/firefox/commit/c3c0e110c871
https://hg.mozilla.org/integration/autoland/rev/49139d3a40ec
Remove unused Legacy Telemetry environment.services r=TravisLong
https://github.com/mozilla-firefox/firefox/commit/31f2aaf23e78
https://hg.mozilla.org/integration/autoland/rev/fa6015b5b5d2
Remove unused Legacy Telemetry environment.system fields r=TravisLong
https://github.com/mozilla-firefox/firefox/commit/a2c7d0884a5f
https://hg.mozilla.org/integration/autoland/rev/4c70718ee403
Remove unused Legacy Telemetry environment.system.cpu fields r=TravisLong
https://github.com/mozilla-firefox/firefox/commit/e80e9b8ffb81
https://hg.mozilla.org/integration/autoland/rev/6eca17103fd8
Remove unused Legacy Telemetry environment.system.os fields r=TravisLong
https://github.com/mozilla-firefox/firefox/commit/beb2cef02eee
https://hg.mozilla.org/integration/autoland/rev/df6f26abe88a
Remove unused Legacy Telemetry environment.system.hdd fields r=TravisLong
https://github.com/mozilla-firefox/firefox/commit/0b04cfe8ac12
https://hg.mozilla.org/integration/autoland/rev/97d3b5cbcfd7
Remove unused Legacy Telemetry environment.system.sec section r=TravisLong
https://github.com/mozilla-firefox/firefox/commit/b386c7380608
https://hg.mozilla.org/integration/autoland/rev/077bdc111666
Remove unused Legacy Telemetry environment.profile fields r=TravisLong
https://github.com/mozilla-firefox/firefox/commit/3f8126deed98
https://hg.mozilla.org/integration/autoland/rev/b754b02c4bed
Don't test now-removed pieces of the Legacy Environment r=beth

Comment 17

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/7cd63cd12484
https://hg.mozilla.org/mozilla-central/rev/8fbb594d6368
https://hg.mozilla.org/mozilla-central/rev/49139d3a40ec
https://hg.mozilla.org/mozilla-central/rev/fa6015b5b5d2
https://hg.mozilla.org/mozilla-central/rev/4c70718ee403
https://hg.mozilla.org/mozilla-central/rev/6eca17103fd8
https://hg.mozilla.org/mozilla-central/rev/df6f26abe88a
https://hg.mozilla.org/mozilla-central/rev/97d3b5cbcfd7
https://hg.mozilla.org/mozilla-central/rev/077bdc111666
https://hg.mozilla.org/mozilla-central/rev/b754b02c4bed