1889123 - Measure potential device migrations for Mozilla Account holders

Status: RESOLVED FIXED

(Firefox :: Firefox Accounts, enhancement, P3)

Description

[Mark Reid [:mreid]]

One of the high-priority initiatives for Firefox in 2024 is: “Device Migration: Retain users on legacy desktop operating systems as they migrate to modern operating systems.” One way we can understand device migration is by looking to Mozilla Account users in Firefox Desktop.

In order to get a better understanding there, including the ability to run experiments, we can introduce a new ping containing the Mozilla Account ID, the Glean Client ID, and the Legacy Telemetry Client ID (plus experiment metadata).

Comment 1

[Mike Conley (:mconley) (:⚙️)]

Attachment: Bug 1889123 - Add a ping / metrics to help measure accounts usage. data-review=mreid, r?markh!,chutten!

Comment 2

[Mike Conley (:mconley) (:⚙️)]

Attachment: Data collection request

Comment 3

[Mike Conley (:mconley) (:⚙️)]

So this was working with my local copies of Glean and Glean Parser with the ping_schedule patches applied... but now it's not. And I don't know why?

When I apply the attached patch on the tip of mozilla-central (which has the Glean / Glean Parser versions with the ping_schedule support), I get this:

mikeconley@MacBook-Pro mozilla-unified % ./mach build
 0:00.39 W Clobber not needed.
 /Users/mikeconley/Projects/mozilla-unified/obj-artifact/backend.FasterMake+RecursiveMakeBackend is out of date with respect to /Users/mikeconley/Projects/mozilla-unified/toolkit/components/glean/metrics_index.py
Build configuration changed. Regenerating backend.
 0:00.57 /Users/mikeconley/.mozbuild/srcdirs/mozilla-unified-3bc3d5e0b760/_virtualenvs/build/bin/python /Users/mikeconley/Projects/mozilla-unified/obj-artifact/config.status
Reticulating splines...
Finished reading 2102 moz.build files in 2.12s
Read 0 gyp files in parallel contributing 0.00s to total wall time
Processed into 11837 build config descriptors in 1.21s
FasterMake+RecursiveMake backend executed in 0.74s
  3427 total backend files; 0 created; 3 updated; 3424 unchanged; 0 deleted
Total wall time: 4.64s; CPU time: 4.04s; Efficiency: 87%; Untracked: 0.57s
 0:05.42 W Adding make options from /Users/mikeconley/.mozconfigs/artifact
    AUTOCLOBBER=1
    MOZ_OBJDIR=/Users/mikeconley/Projects/mozilla-unified/obj-artifact
    OBJDIR=/Users/mikeconley/Projects/mozilla-unified/obj-artifact
    FOUND_MOZCONFIG=/Users/mikeconley/.mozconfigs/artifact
    export FOUND_MOZCONFIG
  Parallelism determined by memory: using 12 jobs for 12 cores based on 36.0 GiB RAM and estimated job size of 1.0 GiB
 0:05.42 /usr/bin/make -f client.mk -j12 -s
 0:14.38 Using artifact from local cache: /Users/mikeconley/.mozbuild/package-frontend/5f809040dca3db1d-target.common.tests.tar.gz
 0:14.44 Using artifact from local cache: /Users/mikeconley/.mozbuild/package-frontend/a8153bb8b2a3b136-target.dmg
 0:15.17 Using artifact from local cache: /Users/mikeconley/.mozbuild/package-frontend/63ae97aa19222ae3-target.update_framework_artifacts.zip
 0:15.17 Using artifact from local cache: /Users/mikeconley/.mozbuild/package-frontend/c8c1e0259adee62e-target.xpt_artifacts.zip
 0:15.41 Elapsed: 0.01s; From dist/xpi-stage: Kept 91 existing; Added/updated 0; Removed 0 files and 0 directories.
 0:15.41 Elapsed: 0.01s; From /Users/mikeconley/Projects/mozilla-unified/obj-artifact/dist/bin/browser/features/[e-mail]: Kept 28 existing; Added/updated 2; Removed 0 files and 0 directories.
 0:15.41 Elapsed: 0.00s; From /Users/mikeconley/Projects/mozilla-unified/obj-artifact/dist/bin/browser/features/[e-mail]: Kept 1 existing; Added/updated 2; Removed 0 files and 0 directories.
 0:15.41 Elapsed: 0.00s; From dist/private: Kept 0 existing; Added/updated 0; Removed 0 files and 0 directories.
 0:15.42 Elapsed: 0.00s; From dist/public: Kept 0 existing; Added/updated 0; Removed 0 files and 0 directories.
 0:15.42 Elapsed: 0.02s; From /Users/mikeconley/Projects/mozilla-unified/obj-artifact/dist/bin/browser/localization: Kept 118 existing; Added/updated 0; Removed 0 files and 0 directories.
 0:15.42 Elapsed: 0.02s; From dist/include: Kept 781 existing; Added/updated 0; Removed 0 files and 0 directories.
 0:15.44 Elapsed: 0.03s; From /Users/mikeconley/Projects/mozilla-unified/obj-artifact/dist/bin/gmp-clearkey/0.1: Kept 0 existing; Added/updated 1; Removed 0 files and 0 directories.
 0:15.49 Elapsed: 0.00s; From /Users/mikeconley/Projects/mozilla-unified/obj-artifact/dist/bin/gmp-fakeopenh264/1.0: Kept 1 existing; Added/updated 0; Removed 0 files and 0 directories.
 0:15.49 Elapsed: 0.00s; From /Users/mikeconley/Projects/mozilla-unified/obj-artifact/dist/bin/gmp-fake/1.0: Kept 1 existing; Added/updated 0; Removed 0 files and 0 directories.
 0:15.50 Elapsed: 0.01s; From /Users/mikeconley/Projects/mozilla-unified/obj-artifact/dist/bin/localization: Kept 88 existing; Added/updated 0; Removed 0 files and 0 directories.
 0:15.52 Elapsed: 0.13s; From _tests: Kept 1493 existing; Added/updated 0; Removed 0 files and 0 directories.
 0:15.60 Traceback (most recent call last):
 0:15.60   File "/Users/mikeconley/.pyenv/versions/3.9.19/lib/python3.9/runpy.py", line 197, in _run_module_as_main
 0:15.60     return _run_code(code, main_globals, None,
 0:15.60   File "/Users/mikeconley/.pyenv/versions/3.9.19/lib/python3.9/runpy.py", line 87, in _run_code
 0:15.60     exec(code, run_globals)
 0:15.60   File "/Users/mikeconley/Projects/mozilla-unified/python/mozbuild/mozbuild/action/file_generate.py", line 154, in <module>
 0:15.60     sys.exit(main(sys.argv[1:]))
 0:15.60   File "/Users/mikeconley/Projects/mozilla-unified/python/mozbuild/mozbuild/action/file_generate.py", line 98, in main
 0:15.60     ret = module.__dict__[method](
 0:15.60   File "/Users/mikeconley/Projects/mozilla-unified/toolkit/components/glean/build_scripts/glean_parser_ext/run_glean_parser.py", line 241, in jog_file
 0:15.60     all_objs, options = parse(args)
 0:15.60   File "/Users/mikeconley/Projects/mozilla-unified/toolkit/components/glean/build_scripts/glean_parser_ext/run_glean_parser.py", line 80, in parse
 0:15.60     return parse_with_options(input_files, options)
 0:15.60   File "/Users/mikeconley/Projects/mozilla-unified/toolkit/components/glean/build_scripts/glean_parser_ext/run_glean_parser.py", line 88, in parse_with_options
 0:15.61     if util.report_validation_errors(all_objs):
 0:15.61   File "/Users/mikeconley/.mozbuild/srcdirs/mozilla-unified-3bc3d5e0b760/_virtualenvs/mach/lib/python3.9/site-packages/glean_parser/util.py", line 473, in report_validation_errors
 0:15.61     for error in all_objects:
 0:15.61   File "/Users/mikeconley/.mozbuild/srcdirs/mozilla-unified-3bc3d5e0b760/_virtualenvs/mach/lib/python3.9/site-packages/glean_parser/util.py", line 239, in __iter__
 0:15.61     self.value = yield from self.g
 0:15.61   File "/Users/mikeconley/.mozbuild/srcdirs/mozilla-unified-3bc3d5e0b760/_virtualenvs/mach/lib/python3.9/site-packages/glean_parser/parser.py", line 461, in parse_objects
 0:15.61     yield from _instantiate_pings(
 0:15.61   File "/Users/mikeconley/.mozbuild/srcdirs/mozilla-unified-3bc3d5e0b760/_virtualenvs/mach/lib/python3.9/site-packages/glean_parser/parser.py", line 334, in _instantiate_pings
 0:15.61     if isinstance(all_objects["pings"][scheduler], Ping):
 0:15.61 KeyError: 'baseline'
 0:15.62 make[4]: *** [/Users/mikeconley/Projects/mozilla-unified/obj-artifact/toolkit/components/glean/.deps/jogfile.json.stub] Error 1
 0:15.62 make[4]: *** Waiting for unfinished jobs....
 0:15.79 Elapsed: 0.39s; From /Users/mikeconley/Projects/mozilla-unified/obj-artifact/dist/bin/browser: Kept 4124 existing; Added/updated 26; Removed 0 files and 0 directories.
 0:15.80 make[3]: *** [faster] Error 2
 0:15.80 make[2]: *** [pre-export] Error 2
 0:15.80 make[1]: *** [default] Error 2
 0:15.80 make: *** [build] Error 2
 0:15.80 W 0 compiler warnings present.

When I inspect the all_objects["pings"] dict with PDB when I hit this, I see this:

{'pings': {'messaging-system': <glean_parser.pings.Ping object at 0x105f34460>, 'newtab': <glean_parser.pings.Ping object at 0x105f495e0>, 'spoc': <glean_parser.pings.Ping object at 0x105f49580>, 'top-sites': <glean_parser.pings.Ping object at 0x105f49760>}}

So that explains the exception - baseline isn't in that set. I swear it was in there before when I was working on this... what have I done wrong?

Comment 4

[Chris H-C :chutten]

The "baseline" and the other builtin pings are defined in the Glean SDK's glean-core/pings.yaml which is not in mozilla-central. (And, actually, they're defined inside the code in internal_pings.rs and the pings.yaml is for making tooling happier... but that's probably not necessary detail for this particular wrinkle).

I'm gonna throw to Travis and Jan-Erik who were on the glean_parser part of this feature to weigh in on what to do about builtin pings like "baseline".

Comment 5

[Travis Long [:travis_]]

Sorry for the delay in responding to the ni?, I had to think about this and let it simmer over the weekend.

I have no clue why this would have been working at first after looking at the way it is implemented since the baseline/events/metrics ping generation happens when glean-core is built. If this were a test or something within glean, then I could see where it would work but outside of glean it just isn't going to work for scheduling custom pings along with internal pings because the schedules_pings list property implies that these are other pings that this ping also submits, but it would need to know those at time of code generation for the baseline/events/metrics pings.

So, my sincere apologies for not thinking through all of this when reviewing your PR, it didn't dawn on me that this information would need to be available at the point where the glean internal pings are generated at the time, and of course scheduling custom pings along with internal pings seems like a valid and reasonable use-case for this.

The only opportunity I see for this information sharing to happen with the way things are currently implemented would be for glean-core to parse registered pings for this information on initialization before scheduling any internal pings, and then update the schedules_pings of the internal pings dynamically. This runs into the problem of pings that are registered after initialization "missing the boat", but that's probably a solvable problem to put ping registration in consuming apps before glean.init.

I'm curious to see what Jan-Erik thinks, but that's my 2¢ on this, with the tl;dr being that this'll likely need a little more code to make it happen.

Comment 6

[Jan-Erik Rediger [:janerik]]

I talked with Travis earlier today and we have an idea how this should be done. I will put it on my schedule for Thursday and get back here then.

Comment 7

[Nadia Florez]

Linking to this experiment which would also benefit from MzA <> telemetry linkage to complete the experiment analysis: https://docs.google.com/document/d/1Qzck25jjLTunFaLI4d0re7fjlInTTfbSnTb7quC8mVE/edit#

Comment 8

[Mark Reid [:mreid]]

Comment on attachment 9395878 [details]
Data collection request

The reviewers have looked at this data collection request and approved it provided the following items are addressed before we activate data collection (code can land pref'd off):

• [legal and policy team] Update the privacy FAQ on the website (this was identified as an action from the earlier decision brief and still needs to be done)
• [data engineering] Honour deletion requests coming from the related Mozilla Account
• [data engineering] Add ability to export data by Account ID for Data Subject Access Requests, tie in to existing DSAR process

Comment 9

[Mike Conley (:mconley) (:⚙️)]

Attachment: Bug 1889123 - Add tests for the client association ping. r?markh!

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3149207d1caf
Add a ping / metrics to help measure accounts usage. data-review=mreid, r=markh,firefox-desktop-core-reviewers ,data-stewards,chutten

Comment 11

[pstanciu]

https://hg.mozilla.org/mozilla-central/rev/3149207d1caf

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a13076353ca0
Add tests for the client association ping. r=markh,chutten

Comment 13

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/a13076353ca0