1636576 - Leanplum E_Search_Widget_Added only fires if the browser is running

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: Metrics, defect, P1)

Description

[Stefan Arentz | :st3fan | ⏰ EST | he/him]

The E_Search_Widget_Added event only fires if the browser is running in the background when the user adds the search widget. I assume the reason for this is that Leanplum is not actually active/running part of the Widget.

We may instead need to queue the event somewhere (prefs?) so that Fennec can send trigger it next time it starts up.

Comment 1

[Stefan Arentz | :st3fan | ⏰ EST | he/him]

Just to confirm this, I tried two scenarios:

The E_Search_Widget_Added event is not fired:

• Kill Firefox
• Add the Widget
• Open Firefox (cold start)
• Put Firefox in the background

The E_Search_Widget_Added is fired:

• Open Firefox (cold start)
• Put Firefox in the background
• Add the Widget
• Open Firefox (warm start - already running)
• Put Firefox in the background

Petru, what do you think?

Comment 2

[Stefan Arentz | :st3fan | ⏰ EST | he/him]

Made this a P1 to keep it on the board.

Question for cbonacuse - is this blocking a campaign? Do you think the campaign can be run by using just the E_Interact_With_Search_Widget event? Asking because it could take a while before a change for this lands in Fennec.

Comment 3

[Petru-Mugurel Lingurar [:petru]]

Can confirm your findings Stefan.
Indeed if the app is not running we will fail to track the E_Search_Widget_Added because we explicitly check down the road if mma is enabled and basically if Leanplum was started (by checking the context with which Leanplum was started).

There's no need though to manually cache these events, Leanplum does support this by default.
If the user adds the widget to the screen it means the app process will be started and as such we do have a valid context with which to check if mma is enabled and if so send this event to Leanplum.
Because Leanplum is though only started when the app is opened it will just cache the events and send them in bulk immediately after being started.

Comment 4

[Petru-Mugurel Lingurar [:petru]]

Attachment: Bug 1636576 - Have Leanplum queue events if not started

track() would previously check if "applicationContext != null" essentially
hecking if init() was called - if Leanplum was started.
It is possible for this static method to be called even before Leanplum start
(eg: by a BroadcastReceiver like in the case of the Search widget).

We'll now just check if "isMmaAllowed".
If it is allowed and enabled we will pass the event to Leanplum which will:

• send the event right away if it is started
• queue the event to be sent immediately after being started

Comment 5

[Petru-Mugurel Lingurar [:petru]]

The patch can be tested with these try builds - https://treeherder.mozilla.org/#/jobs?repo=try&revision=fd575eaf62eb30d718029a5d49698deee29f5160

Comment 6

[[e-mail]]

@stefan - apologies for the delay in my response. We will need E_Search_Widget_Added to reliably measure the conversion of a test we are running, however, we will be moving out the test knowing that this would not be in the upcoming release. For context, I'll be opening a Github request for Fenix as well to 1. get the search widget events added and 2. to follow your solution here for Fennec. Thanks!

Comment 7

[Petru-Mugurel Lingurar [:petru]]

When trying to test this in the above th builds we get in logs

[ERROR][Leanplum]: [com.leanplum.internal.RequestOld::parseResponseBody::457]: No app matching the provided app ID was found.

I'm thinking that maybe th got the leanplum api keys from my mozconfig.. which were for dev and not nightly..
So I'll try a new build with the nightly keys set in my local mozconfig.

Comment 8

[Petru-Mugurel Lingurar [:petru]]

After confirming with RyanVM that TH builds will not have the proper Leanplum keys I builded locally a Nightly with the proposed patch to be verified by QA - https://drive.google.com/file/d/1-ljnAj-9-sDdJkBL05J9iMPPMdhUF-0U/view?usp=sharing

Comment 9

[Sorina Florean [:sflorean]]

Verified as fixed on the build provided in comment 8, the event E_Search_Widget_Added is displayed after the widget has been added to the home screen.
Devices: Samsung Galaxy Note 10 (Android 10), Nokia 6 (Android 7.1.1), Huawei MediaPad M2 (Android 5.1.1).

Comment 10

[Petru-Mugurel Lingurar [:petru]]

Comment on attachment 9148359 [details]
Bug 1636576 - Have Leanplum queue events if not started

ESR Uplift Approval Request

• If this is not a sec:{high,crit} bug, please state case for ESR consideration: High priority data request, needed for upcoming experiments.
• User impact if declined:
• Fix Landed on Version:
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Small change, qe verified.
• String or UUID changes made by this patch:

Comment 11

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:st3fan, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 12

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9148359 [details]
Bug 1636576 - Have Leanplum queue events if not started

Approved for Fennec 68.9.

Comment 13

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr68/rev/aa761c20030b0992e5dfff769015e8dff7569550

Comment 14

[Diana Rus]

Hi, wanted to test this but i don't have access to leanplum, in order to verify. In case someone has access and want's to check it out.
@Stefan, in case you want to see and if you are able to access.
I'll leave below the device used and the telemetry client id for them. The scenarios have been run on Firefox Release 68.9.0.

Scenario A - The E_Search_Widget_Added event is not fired:
1.Kill Firefox
2.Add the Widget
3.Open Firefox (cold start)-(i've opened through widget)
4.Put Firefox in the background
Google Pixel 3XL (Android 9) - Client ID: 0dcd3aee-f856-4bb3-b630-6558ce5b5d90

Scenario B - The E_Search_Widget_Added is fired:

1. Open Firefox (cold start)
2. Put Firefox in the background
3. Add the Widget
4. Open Firefox (warm start - already running)
5. Put Firefox in the background
   Samsung Galaxy S9 (Android 8) - Client ID: ff008220-9987-4fee-868f-59ad553adea2

Comment 15

[[e-mail]]

@Diana & @Stefan - I was only able to update to 68.8.1 (which may be why this did not work for me) but I'm on a Google Pixel 3 (OS 10) (Leanplum device ID: 993fe6d2-1888-45ad-b5df-5d64bdcff007) and scenario A is still not working for me (I did cold start through the app and not the widget).

I don't think this is possible, but can we identify Diana's Leanplum device ID through the telemetry client ID's that were provided?

Comment 16

[Petru-Mugurel Lingurar [:petru]]

One important aspect here is that I think Leanplum processing does take a bit of time, more with the more users on the release channel.
For example when looking for the milestones of a device running a dev build they were available in Leanplum almost immediately but when checking for a nightly build the milestones were updated only after >15 minutes.
So it would be important to note if other milestones which would've happened after adding the widget appear in the CMS but the "E_Search_Widget_Added" not.

Comment 17

[Stefan Arentz | :st3fan | ⏰ EST | he/him]

This fix landed in 68.9 not 68.8.1, so let's verify this when that release is out on the 2nd.

Comment 18

[[e-mail]]

@Stefan - I tested today using the 68.9 build you sent me a link to, and my device ID (d38b259d-1632-426d-8b10-3db7b8e9f867) registered the E_Search_Widget_Added event when the app was closed (not backgrounded) and I added the widget from the homescreen then proceeded to open the app. This is working as expected, correct?

Comment 19

[Sorina Florean [:sflorean]]

Yes, @cbonacuse the event E_Search_Widget_Added should be displayed in the Leanplum dashboard based on the device ID, after you add the widget on the homescreen, this is how we verified on the build provided by Petru. Unfortunately, we don't have access to Release Leanplum and we can't check on the build 68.9.

Comment 20

[Stefan Arentz | :st3fan | ⏰ EST | he/him]

Closing this bug - feel free to re-open if this does not work as expected.