1930868 - [Nimbus] Studies should be toggled back on after re-enabling Telemetry

Status: VERIFIED FIXED

(Firefox for Android :: Experimentation and Telemetry, defect)

Description

[Ciprian Muresan [:cmuresan], Formerly of Ecosystem QA - please needinfo @mcoman for things regarding the team]

Attachment: media_20241213_135232_6577138796750544085.mp4

[Affected versions]:

• Firefox Nightly 134.0a1 (Build #2016055623), hg-4e75481ca263+, GV: 134.0a1-20241111213954, AS: 134.20241109050310
• Firefox Beta 133.0b7 (Build #2016055535), hg-5aff34e4e12c+, GV: 133.0-20241111092015, AS: 133.0
• Firefox Release 132.0 (Build #2016051567), hg-0e15e2edd460+, GV: 132.0-20241021175835, AS: 132.0.1

[Affected Platforms]:

• Android 13 - Samsung Galaxy A32
• Android 14 - OnePlus 9 Pro

[Prerequisites]:

• Have the Firefox app installed.

[Steps to reproduce]:

1. Open Firefox and tap the menu button.
2. Tap the Settings option and scroll down to the Privacy and security section.
3. Tap the Data collection option.
4. Untoggle the Usage and technical data option.
5. Reopen the app and navigate to the same menu.
6. Toggle the Usage and technical data option and observe the Studies option.

[Expected result]:

• The Studies option is toggled back on.

[Actual result]:

• The Studies option remains toggled off.

[Notes]:

• Due to this behavior we risk never gaining those users back and diminishing our experimental reach.
• The Studies option should always revert to its initial state.
• Attached a screen recording of the issue.

Comment 1

[Roger Yang [:royang]]

There's a few combinations here
User have studies turned on: Yes to turn on studies when telemetry is turned back on.
User have studies turned off: No to turn on studies when telemetry is turned back on.

The problem here is by us turning off studies, we have to set the shared preference (so we show the correct value in settings) and also calls the Nimbus SDK (to turn off participation to experimentation). We have no value that remembers what the users original preference is.

Comment 2

[Roger Yang [:royang]]

The simple fix without causing confusing is to always turn studies back to on when user turn telemetry back on.
Pro: Handles the corner case of user turning telemetry back on (low percentage) without creating complicated implementation trying to store user's past preference on studies.
Con: For the user who turned studies off first before turning telemetry off, then proceeds to turn telemetry back on (very low percentage), they might not notice studies is turned back on as well.

I think for future maintenance, this is a good solution. Since turning telemetry back on is a low percentage use case. Also turning telemetry back on signals to us that the user does want to contribute to Mozilla's experimentation.

Comment 3

[Roger Yang [:royang]]

I'll also have to update the probe related to user's telemetry preference. Currently we report when user updates when user makes a choice for telemetry but we don't record how many turns it on vs off. Once updated, we can revisit later to see if there is a high amount of user turning telemetry back on after turning it off.

Comment 4

[Roger Yang [:royang]]

Attachment: Bug 1930868 - Turn on studies when user turns on telemetry collection.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/625878d03b57
Turn on studies when user turns on telemetry collection. r=android-reviewers,twhite

Comment 6

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/625878d03b57

Comment 7

[Roger Yang [:royang]]

Comment on attachment 9463801 [details]
Bug 1930868 - Turn on studies when user turns on telemetry collection.

Beta/Release Uplift Approval Request

• User impact if declined/Reason for urgency: Without this fix, users who turns back on technical and interaction data will not automatically turn on studies. We will unfortunately lose users to experiments.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This is only affecting the studies preference when turning on and off the technical and interaction data preference.
• String changes made/needed: N/A
• Is Android affected?: Yes

Comment 8

[Donal Meehan [:dmeehan]]

Comment on attachment 9463801 [details]
Bug 1930868 - Turn on studies when user turns on telemetry collection.

Approved for 136.0b4

Comment 9

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/155088799875

Comment 10

[Ciprian Muresan [:cmuresan], Formerly of Ecosystem QA - please needinfo @mcoman for things regarding the team]

I have verified that the issue is no longer reproducible on Firefox Nightly 137.0a1 (Build #2016073178), hg-321052fb2ab5+, GV: 137.0a1-20250211090130, AS: 137.20250208050325 and Firefox Beta 136.0b4 (Build #2016073002), hg-87338c725e7a+, GV: 136.0-20250210091604, AS: 136.0. The studies toggle is now always togged on when Data Collection is turned back on.

Comment 11

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9463801 [details]
Bug 1930868 - Turn on studies when user turns on telemetry collection.

Approved for 135.0.1

Comment 12

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-release/rev/78f459d30a52

Comment 13

[Ciprian Muresan [:cmuresan], Formerly of Ecosystem QA - please needinfo @mcoman for things regarding the team]

I have verified that the issue is no longer reproducible on Firefox Release 135.0.1 (Build #2016074234), hg-7c2ff1aa3394+, GV: 135.0.1-20250216192613, AS: 135.0.1.