1853113 - [Experiment] [Intermittent] The “metrics.search_widget_installed” probe is registered as “False” after the Search widget is installed

Status: VERIFIED FIXED

(Firefox for Android :: Experimentation and Telemetry, defect)

Description

[Valentin Bandac [:vbandac], Ecosystem QA]

[Affected versions]:

• Firefox Beta 118.0b9 (BUILD ID #2015974267), 23c3ed5b9b+, GV: 118.0 - 20230914180032, AS: 118.0

[Affected Platforms]:

• Samsung Galaxy Tab A8 - Android 13
• One+ 6 - Android 11

[Prerequisites]:

• Have the latest version of nimbus-cli installed.
• Have the nimbus.cli command for the android-onboarding-search-widget experiment (nimbus-cli --app fenix --channel beta enroll preview/android-onboarding-search-widget --branch treatment-b).
• Have the adb command for Glean (adb shell am start -n org.mozilla.firefox_beta/mozilla.telemetry.glean.debug.GleanDebugActivity --ez logPings true --es sendPing events --es sendPing metrics --es debugViewTag <chosen_tag>).

[Steps to reproduce]:

1. Use the nimbus-cli from prerequisites to enroll in the experiment and install the Search widget.
2. Close the app and perform a search using the Search Widget.
3. Use the adb command to send events into Glean.
4. Navigate to the Glean viewer and search for metrics.search_widget_installed.

[Expected result]:

• The “metrics.search_widget_installed” probe is registered as “true”.

[Actual result]:

• The “metrics.search_widget_installed” probe is registered as “false”.

[Notes]:

• The “metrics.search_widget_installed” is registered as “true” intermittently.
• We did not manage to reproduce the issue on a Samsung Galaxy A32 device using Android 13.

Comment 1

[Vlad Dreghici [:vdreghici]]

I have been looking into this problem and it seems like the way we currently register the widget being removed from the launcher is faulty. When adding the widget we are incrementing a count, but the onEnabled function is called only when a first instance of the widget has been added to the home screen and not called anymore to any successive widget adds, therefore, even if 20 search widgets have been added, we only have value 1 saved in our preferences. The problem appears when removing a widget, because we are using onDeleted to decrement that value mentioned before, but onDeleted is called whenever one widget is removed, thus leading to a scenario where if the user added 20 widgets and removed all of them, the value we would compare to zero to see if there are any widget on the home screen would be -19 giving us an erroneous result. Instead of using this increment/decrement solution, I tried following the documentation here and removed the onDeleted function in favor of onDisabled which is called when the last AppWidget instance for this provider is deleted and changing from an Int based pref to a Boolean.

Comment 2

[BMO Github Automation]

Attachment: [mozilla-mobile/firefox-android] Bug 1853113 - Monitor Search Widget presence based on activation (#3713)

Comment 3

[BMO Github Automation]

Authored by https://github.com/DreVla
https://github.com/mozilla-mobile/firefox-android/commit/f2134f7b79ab12e7c49686c3922fd95d5acf5355
[main] Bug 1853113 - Monitor Search Widget presence based on activation

Comment 4

[BMO Github Automation]

Attachment: [mozilla-mobile/firefox-android] Bug 1853113 - Monitor Search Widget presence based on activation (backport #3713) (#3748)

Comment 5

[Vlad Dreghici [:vdreghici]]

Comment on attachment 9354542 [details] [review]
[mozilla-mobile/firefox-android] Bug 1853113 - Monitor Search Widget presence based on activation (backport #3713) (#3748)

Beta/Release Uplift Approval Request

• User impact if declined: This patch aims to fix when we send the metric for the search widget. There is no impact to the user, but it will impact the results of the Juno Onboarding Add Search Widget experiment.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: 1. Enroll in the "add search widget" experiment.

2. On the Add Search Widget card, tap the positive button.
3. Check if the metric is registered correctly after adding the search widget

• List of other uplifts needed: none
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This changes only when we send the metrics.search_widget_installed metric.
• String changes made/needed: none
• Is Android affected?: Yes

Comment 6

[Daniel Holbert [:dholbert]]

There's a good chance this is behind the new startup crash in bug 1854739, where we've got an integer stored and we're reading it as a boolean and tripping a fatal runtime error of some sort.

So: probably we shouldn't uplift to release or beta, until/unless we have a better understanding of what's going on in bug 1854739.

(We also can't directly backout since now some users have a boolean value stored, so we'd potentially hit the inverse problem for those users if we revert the change directly.)

Arturo is investigating the best way forward here. In the meantime, I wanted to just be sure we don't go forward with the backport and inadvertently introduce the issue on beta/release.

Comment 7

[Pascal Chevrel (relman team) -> :pascalc]

Comment on attachment 9354542 [details] [review]
[mozilla-mobile/firefox-android] Bug 1853113 - Monitor Search Widget presence based on activation (backport #3713) (#3748)

Rejected for causing crashes.

Comment 8

[Valentin Bandac [:vbandac], Ecosystem QA]

I have verified that this issue is no longer reproducible. The “metrics.search_widget_installed” probe is registered as “true” after the Search widget is installed. I am marking this issue as Verified on Firefox 119.
Tested on:
Platforms:

• OnePlus 6 - Android 11
• Samsung A53 5G - Android 13

Versions:

• Firefox Nightly 120.0a1 (Build ID: 2015976355), f5b6df4cd3+, GV: 120.0a1-20230925160932, AS: 119.20230923050305
• Firefox Beta 119.0b1 (Build ID: 2015976283). D7b02b3259+, GV: 119.0-20230925132400, AS: 119.0