1461690 - Create an uninstaller telemetry ping

Status: VERIFIED FIXED

(Firefox :: Installer, enhancement, P1)

Comment 1

[Rachel Tublitz [:rachel]]

Link to DS consult request

Comment 2

[Rachel Tublitz [:rachel]]

Romain, could you confirm that we do want to correlate this with client_id? I assume the answer is yes, but since that will make a difference for implementation here I want to double check before we move forward.

Comment 3

[Romain Testard [:RT]]

(In reply to Rachel Tublitz [:rachel] from comment #2)

Romain, could you confirm that we do want to correlate this with client_id? I assume the answer is yes, but since that will make a difference for implementation here I want to double check before we move forward.

Yes, this is they key to allow retention measurements.

Comment 4

[Su-Young Hong]

After discussing offline, here are some additional data that should be captured in the uninstall ping:

ClientId 
	* based on default or most recently used profile connected to installation

Firefox Installs Present
        * Number of other Firefox installations present on machine (excluding application being uninstalled) 

Environment Info
	* OS
	* OS version
	* OS arch 32/64 Bit

Application Info
	* version
	* update channel (release, beta, etc.)
	* locale
	* app arch 32/64 Bit
	* partner / distribution ID

Comment 5

[Rachel Tublitz [:rachel]]

(In reply to Su-Young Hong from comment #4)

After discussing offline, here are some additional data that should be captured in the uninstall ping:

Can we also add os_locale and country to Environment Info? For the former, it would be useful to capture details where we might have a mismatch between application locale and system locale. And country is useful for us to have as we try to better understand worldwide usage.

Comment 6

[Adam Gashlin (he/him) [:agashlin]]

The metadata.geo and normalized_country_code columns which you can see in firefox_installer.install will already be present in any pings, I believe these are based on geolocation of the submitting IP address.

It's possible to add OS locale. We don't currently collect it in the installers, so adding this would involve some error-prone NSIS System code and additional testing. Do we still want it as part of this project?

Comment 7

[Adam Gashlin (he/him) [:agashlin]]

ni? for comment 6 question re: OS locale.

Comment 8

[Rachel Tublitz [:rachel]]

(In reply to Adam Gashlin (he/him) [:agashlin] from comment #6)

The metadata.geo and normalized_country_code columns which you can see in firefox_installer.install will already be present in any pings, I believe these are based on geolocation of the submitting IP address.

Ah, ok great.

It's possible to add OS locale. We don't currently collect it in the installers, so adding this would involve some error-prone NSIS System code and additional testing. Do we still want it as part of this project?

No, I think we can skip for now to keep this work simple. We can always return down the road to delve into it if we decide it's critical. Thanks for double checking.

Comment 9

[Rachel Tublitz [:rachel]]

Link to PI request

Comment 10

[Adam Gashlin (he/him) [:agashlin]]

Attachment: Bug 1461690 Part 1: Write uninstall ping. r=chutten!,Gijs!

This covers the in-Firefox portions of the install ping,
except for the otherInstalls count, which will be added in
a later part. Later parts will cover the uninstaller.

Comment 11

[Adam Gashlin (he/him) [:agashlin]]

Attachment: Bug 1461690 Part 2: Count other installs. r?mhowell!

Comment 12

[Adam Gashlin (he/him) [:agashlin]]

A few updates about the design now being implemented:

Similarly to bug 1647422, we wanted to avoid the "other Firefox installs" count from acting as a high-resolution identifier if there is a large, unique number of installs. After discussing this with Su we agreed to implement this by capping the count at 11.

After a discussion with :chutten and Su, I decided to use the standard Telemetry system to write the ping from within Firefox, rather than save just the client ID and attempt to construct a bespoke ping in the uninstaller. This means that the common ping structure, including the environment block, can be used. The result will look like any other ping, with a clientId, otherInstalls as payload, and the rest of the data points mentioned in comment 4 provided in the environment block. The uninstaller then just uploads the ping file when uninstalling.

Comment 13

[Adam Gashlin (he/him) [:agashlin]]

Attachment: uninstall ping data review request

Comment 14

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla-services/mozilla-pipeline-schemas/pull/629

Comment 15

[Tim Smith 👨‍🔬 [:tdsmith]]

Comment on attachment 9180553 [details]
uninstall ping data review request

1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, in the Firefox source tree docs.

2. Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, the Firefox telemetry opt-out.

3. If the request is for permanent data collection, is there someone who will monitor the data over time?

Romain Testard.

4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

The new collection is Category 2, interaction data.

5. Is the data collection request for default-on or default-off?

Default-on.

6. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No.

7. Is the data collection covered by the existing Firefox privacy notice?

Yes.

8. Does there need to be a check-in in the future to determine whether to renew the data?

No, permanent collection.

9. Does the data collection use a third-party collection tool?

No.

Comment 16

[Adam Gashlin (he/him) [:agashlin]]

Attachment: Bug 1461690 Part 3: HttpPostFile plugin for uploading uninstall ping via POST. r?mhowell!

Depends on D92533

Comment 17

[Adam Gashlin (he/him) [:agashlin]]

Attachment: Bug 1461690 Part 4: Submit ping in uninstaller. r?mhowell!

Depends on D93717

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c842251ed635
Part 1: Write uninstall ping. r=chutten,Gijs
https://hg.mozilla.org/integration/autoland/rev/fe4ce626ccc0
Part 2: Count other installs. r=mhowell
https://hg.mozilla.org/integration/autoland/rev/be2a3a582745
Part 3: HttpPostFile plugin for uploading uninstall ping via POST. r=mhowell
https://hg.mozilla.org/integration/autoland/rev/79fe5a3f53f5
Part 4: Submit ping in uninstaller. r=mhowell

Comment 19

[Sandor Molnar]

https://hg.mozilla.org/mozilla-central/rev/c842251ed635
https://hg.mozilla.org/mozilla-central/rev/fe4ce626ccc0
https://hg.mozilla.org/mozilla-central/rev/be2a3a582745
https://hg.mozilla.org/mozilla-central/rev/79fe5a3f53f5

Comment 20

[Adam Gashlin (he/him) [:agashlin]]

Comment on attachment 9179761 [details]
Bug 1461690 Part 1: Write uninstall ping. r=chutten!,Gijs!

Beta/Release Uplift Approval Request

• User impact if declined: If declined, the user impact will be seen indirectly; our visibility into bugs/problems/external events that may cause users to uninstall Firefox will be increased significantly by the addition of this feature. Ie, we may use this data to determine that users in a specific locale are having a poor experience using Firefox and are uninstalling. The user impact for THOSE particular users would be non-existent. However, the user impact for future users in the same situation could be significant; this extra bit of telemetry will give us the ability to fix potential issues and problems inside of Firefox before they cause users so much friction that they choose to uninstall completely.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This is working on Nightly, but QE haven't finished a full suite of tests. There is some but not complete automated testing. Changes to the uninstaller require manual testing, I've done limited testing with these patches applied to beta.
• String changes made/needed: N/A

Comment 21

[Pascal Chevrel:pascalc]

Comment on attachment 9179761 [details]
Bug 1461690 Part 1: Write uninstall ping. r=chutten!,Gijs!

Approved for 83 beta 6, thanks.

Comment 22

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/a8bc474ef8ba
https://hg.mozilla.org/releases/mozilla-beta/rev/59d0d9d24f04
https://hg.mozilla.org/releases/mozilla-beta/rev/d736bc9eb146
https://hg.mozilla.org/releases/mozilla-beta/rev/0b537f611f46

Comment 23

[Timea Cernea [:tbabos]]

Marking this as verified-fixed based on the testing requested and completed for PI-807 request on Nightly 84.0a1 and Beta 83.0b6-0b8.