Closed Bug 1461690 Opened 3 years ago Closed 1 month ago

Create an uninstaller telemetry ping

Categories

(Firefox :: Installer, enhancement, P1)

enhancement

Tracking

()

VERIFIED FIXED
84 Branch
Tracking Status
firefox83 --- verified
firefox84 --- verified

People

(Reporter: RT, Assigned: agashlin)

References

(Blocks 1 open bug, )

Details

(Whiteboard: [iu_tracking])

User Story

As a product manager I want to understand how many people uninstall Firefox on Windows so I can understand churn and user life cycle on Windows better, and trend uninstall volumes.
As a product manager I want to understand statistical significance of the uninstaller survey

Acceptance criteria:
- A telemetry ping gets sent upon uninstall completion on Windows with a build ID of the uninstalled build

Next steps:
- Design doc including schema
- Get reviews with Georg and Mika

Attachments

(6 files)

No description provided.
User Story: (updated)
Priority: -- → P2
See Also: → 1631588
Whiteboard: [iu_tracking]
Priority: P3 → P2
Assignee: nobody → agashlin
Status: NEW → ASSIGNED
Priority: P2 → P1

Romain, could you confirm that we do want to correlate this with client_id? I assume the answer is yes, but since that will make a difference for implementation here I want to double check before we move forward.

Flags: needinfo?(rtestard)

(In reply to Rachel Tublitz [:rachel] from comment #2)

Romain, could you confirm that we do want to correlate this with client_id? I assume the answer is yes, but since that will make a difference for implementation here I want to double check before we move forward.

Yes, this is they key to allow retention measurements.

Flags: needinfo?(rtestard)

After discussing offline, here are some additional data that should be captured in the uninstall ping:

ClientId 
	* based on default or most recently used profile connected to installation

Firefox Installs Present
        * Number of other Firefox installations present on machine (excluding application being uninstalled) 

Environment Info
	* OS
	* OS version
	* OS arch 32/64 Bit

Application Info
	* version
	* update channel (release, beta, etc.)
	* locale
	* app arch 32/64 Bit
	* partner / distribution ID
Blocks: 1665381

(In reply to Su-Young Hong from comment #4)

After discussing offline, here are some additional data that should be captured in the uninstall ping:

Can we also add os_locale and country to Environment Info? For the former, it would be useful to capture details where we might have a mismatch between application locale and system locale. And country is useful for us to have as we try to better understand worldwide usage.

The metadata.geo and normalized_country_code columns which you can see in firefox_installer.install will already be present in any pings, I believe these are based on geolocation of the submitting IP address.

It's possible to add OS locale. We don't currently collect it in the installers, so adding this would involve some error-prone NSIS System code and additional testing. Do we still want it as part of this project?

ni? for comment 6 question re: OS locale.

Flags: needinfo?(rtublitz)

(In reply to Adam Gashlin (he/him) [:agashlin] from comment #6)

The metadata.geo and normalized_country_code columns which you can see in firefox_installer.install will already be present in any pings, I believe these are based on geolocation of the submitting IP address.

Ah, ok great.

It's possible to add OS locale. We don't currently collect it in the installers, so adding this would involve some error-prone NSIS System code and additional testing. Do we still want it as part of this project?

No, I think we can skip for now to keep this work simple. We can always return down the road to delve into it if we decide it's critical. Thanks for double checking.

Flags: needinfo?(rtublitz)

This covers the in-Firefox portions of the install ping,
except for the otherInstalls count, which will be added in
a later part. Later parts will cover the uninstaller.

A few updates about the design now being implemented:

Similarly to bug 1647422, we wanted to avoid the "other Firefox installs" count from acting as a high-resolution identifier if there is a large, unique number of installs. After discussing this with Su we agreed to implement this by capping the count at 11.

After a discussion with :chutten and Su, I decided to use the standard Telemetry system to write the ping from within Firefox, rather than save just the client ID and attempt to construct a bespoke ping in the uninstaller. This means that the common ping structure, including the environment block, can be used. The result will look like any other ping, with a clientId, otherInstalls as payload, and the rest of the data points mentioned in comment 4 provided in the environment block. The uninstaller then just uploads the ping file when uninstalling.

Attachment #9180553 - Flags: data-review?(tdsmith)

Comment on attachment 9180553 [details]
uninstall ping data review request

  1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, in the Firefox source tree docs.

  1. Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, the Firefox telemetry opt-out.

  1. If the request is for permanent data collection, is there someone who will monitor the data over time?

Romain Testard.

  1. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

The new collection is Category 2, interaction data.

  1. Is the data collection request for default-on or default-off?

Default-on.

  1. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No.

  1. Is the data collection covered by the existing Firefox privacy notice?

Yes.

  1. Does there need to be a check-in in the future to determine whether to renew the data?

No, permanent collection.

  1. Does the data collection use a third-party collection tool?

No.

Attachment #9180553 - Flags: data-review?(tdsmith) → data-review+
Pushed by agashlin@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/c842251ed635
Part 1: Write uninstall ping. r=chutten,Gijs
https://hg.mozilla.org/integration/autoland/rev/fe4ce626ccc0
Part 2: Count other installs. r=mhowell
https://hg.mozilla.org/integration/autoland/rev/be2a3a582745
Part 3: HttpPostFile plugin for uploading uninstall ping via POST. r=mhowell
https://hg.mozilla.org/integration/autoland/rev/79fe5a3f53f5
Part 4: Submit ping in uninstaller. r=mhowell

Comment on attachment 9179761 [details]
Bug 1461690 Part 1: Write uninstall ping. r=chutten!,Gijs!

Beta/Release Uplift Approval Request

  • User impact if declined: If declined, the user impact will be seen indirectly; our visibility into bugs/problems/external events that may cause users to uninstall Firefox will be increased significantly by the addition of this feature. Ie, we may use this data to determine that users in a specific locale are having a poor experience using Firefox and are uninstalling. The user impact for THOSE particular users would be non-existent. However, the user impact for future users in the same situation could be significant; this extra bit of telemetry will give us the ability to fix potential issues and problems inside of Firefox before they cause users so much friction that they choose to uninstall completely.
  • Is this code covered by automated tests?: Yes
  • Has the fix been verified in Nightly?: Yes
  • Needs manual test from QE?: No
  • If yes, steps to reproduce:
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): This is working on Nightly, but QE haven't finished a full suite of tests. There is some but not complete automated testing. Changes to the uninstaller require manual testing, I've done limited testing with these patches applied to beta.
  • String changes made/needed: N/A
Attachment #9179761 - Flags: approval-mozilla-beta?
Attachment #9179780 - Flags: approval-mozilla-beta?
Attachment #9181914 - Flags: approval-mozilla-beta?
Attachment #9181915 - Flags: approval-mozilla-beta?

Comment on attachment 9179761 [details]
Bug 1461690 Part 1: Write uninstall ping. r=chutten!,Gijs!

Approved for 83 beta 6, thanks.

Attachment #9179761 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Attachment #9179780 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Attachment #9181914 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Attachment #9181915 - Flags: approval-mozilla-beta? → approval-mozilla-beta+

Marking this as verified-fixed based on the testing requested and completed for PI-807 request on Nightly 84.0a1 and Beta 83.0b6-0b8.

Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.