1860919 - Add Glean metrics for Wikipedia Firefox Suggestion impressions and clicks

Status: RESOLVED FIXED

(Fenix :: Search, enhancement)

Description

[Lina Butler (ex-Mozilla)]

Bug 1858542 and bug 1857092 added impression and click metrics for sponsored Firefox Suggestions from AMP.

We'd like to collect the same metrics, using the same new fx-suggest ping added in bug 1857092, for Wikipedia suggestions. This will give us a way to compare mobile numbers to Desktop's when we run our experiment.

Since we're targeting Firefox 120 for our experiment, and 120 is in Beta now, we'll want to uplift this change after it lands and has time to bake in 121.

Comment 1

[BMO Github Automation]

Attachment: [mozilla-mobile/firefox-android] Bug 1860919 - Send Glean pings for Wikipedia Firefox Suggestion impressions and clicks. (#4262)

Comment 2

[Lina Butler (ex-Mozilla)]

The STR for this bug are similar to bug 1858542, comment 6 with two differences: (1) these are triggered for Wikipedia suggestions from Firefox Suggest, and (2) the contents of the impression and click pings are slightly different. The same three scenarios apply for Wikipedia suggestions:

1. The user sees a Wikipedia Firefox Suggestion, and taps on it. Firefox should send an impression and a click ping for that suggestion.
2. The user sees a Wikipedia Firefox Suggestion, and navigates to a different page. Firefox should only send an impression ping for that suggestion.
3. The user sees a Wikipedia Firefox Suggestion, and dismisses the address bar without navigating to a different page. Firefox should not send any pings for that suggestion.

Scenario 1

1. Ensure "Enable Firefox Suggest" is switched on in Settings > Secret Settings, and "Suggestions from Firefox" are switched on in Settings > Search. (These should all be on by default in Nightly).
2. Type cali into the address bar. Verify that a suggestion called "Wikipedia - California" appears in the Firefox Suggest section, and is visible on the screen.
3. Tap on the Wikipedia suggestion.
4. Verify that two Glean pings are sent: one with an fx_suggest.ping_type of fxsuggest-impression, and one with an fx_suggest.ping_type of fxsuggest-click.

The impression ping should look like this:

{
   "metrics": {
      "uuid": {
         "fx_suggest.context_id": "c610f64c-7c16-438a-ab2e-55a113995070"
      },
      "boolean": {
         "fx_suggest.is_clicked": true
      },
      "string": {
         "fx_suggest.ping_type": "fxsuggest-impression",
         "fx_suggest.advertiser": "wikipedia"
      },
      "quantity": {
         "fx_suggest.position": 2
      }
   }
}

And the click ping should look like this:

{
   "metrics": {
      "boolean": {
         "fx_suggest.is_clicked": true
      },
      "uuid": {
         "fx_suggest.context_id": "c610f64c-7c16-438a-ab2e-55a113995070"
      },
      "quantity": {
         "fx_suggest.position": 3
      },
      "string": {
         "fx_suggest.advertiser": "wikipedia",
         "fx_suggest.ping_type": "fxsuggest-click"
      }
   }
}

Scenario 2

1. Ensure "Enable Firefox Suggest" is switched on in Settings > Secret Settings, and "Suggestions from Firefox" is switched on in Settings > Search.
2. Type london into the address bar. Verify that a suggestion called "Wikipedia - London" appears in the Firefox Suggest section, and is visible on the screen.
3. Tap on a different suggestion, or tap the "enter" button on the keyboard to trigger a search.
4. Verify that only one Glean ping is sent, with an fx_suggest.ping_type of fxsuggest-impression.

Here's an example of that impression ping:

{
   "metrics": {
      "boolean": {
         "fx_suggest.is_clicked": false
      },
      "uuid": {
         "fx_suggest.context_id": "c610f64c-7c16-438a-ab2e-55a113995070"
      },
      "string": {
         "fx_suggest.advertiser": "wikipedia",
         "fx_suggest.ping_type": "fxsuggest-impression"
      },
      "quantity": {
         "fx_suggest.position": 5
      }
   }
}

Scenario 3

1. Ensure "Enable Firefox Suggest" is switched on in Settings > Secret Settings, and "Suggestions from Firefox" is switched on in Settings > Search.
2. Type mex into the address bar. Verify that a suggestion called "Wikipedia - Mexico" appears in the Firefox Suggest section, and is visible on the screen.
3. Swipe back to dismiss the address bar, or dismiss it by clearing the address bar and tapping on the page behind it.
4. Verify that no fx-suggest pings are sent.

Comment 3

[BMO Github Automation]

Authored by https://github.com/linabutler
https://github.com/mozilla-mobile/firefox-android/commit/a98251ffdfdb3d945927706dc859feff9694a4f4
[main] Bug 1860919 - Send Glean pings for Wikipedia Firefox Suggestion impressions and clicks.

Comment 4

[Adina Petridean QA [:apetridean]]

I was able to test and verify the 3 scenarios.
For the 1st scenario, link1 and link2 are the Glean metrics.
For the 2nd scenario these are the Glean metrics.
And, for the 3rd scenarios, there were no fx_suggest metrics generated.

Tested on today's Nightly build 121.0a1 from 11/02 with Motorola G9 plus (Android 11).

Comment 5

[BMO Github Automation]

Attachment: [mozilla-mobile/firefox-android] Bug 1860919 - Send Glean pings for Wikipedia Firefox Suggestion impressions and clicks. (backport #4262) (#4338)

Comment 6

[Lina Butler (ex-Mozilla)]

Comment on attachment 9361682 [details] [review]
[mozilla-mobile/firefox-android] Bug 1860919 - Send Glean pings for Wikipedia Firefox Suggestion impressions and clicks. (backport #4262) (#4338)

Beta/Release Uplift Approval Request

• User impact if declined: No visible user impact. Product and business impact for Mozilla is that we won't be able to compare Wikipedia suggestion impressions and clicks for Fenix users enrolled in the Enhanced Cross-Platform Suggest experiment, to our baseline Desktop metrics.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce: STR in bug 1860919, comment 2.
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This is a small code change with extensive unit test coverage, leverages existing telemetry reporting code paths for sponsored suggestions (previously landed in bug 1857092 and bug 1858542), and has been verified in Nightly.
• String changes made/needed: None
• Is Android affected?: Yes

Comment 7

[Dianna Smith [:diannaS]]

Comment on attachment 9361682 [details] [review]
[mozilla-mobile/firefox-android] Bug 1860919 - Send Glean pings for Wikipedia Firefox Suggestion impressions and clicks. (backport #4262) (#4338)

Approved for Mobile 120.0b6

Comment 8

[BMO Github Automation]

Authored by https://github.com/linabutler
https://github.com/mozilla-mobile/firefox-android/commit/14e4566e6fa639ed765a857612feadcb9f2e82e0
[releases_v120] Bug 1860919 - Send Glean pings for Wikipedia Firefox Suggestion impressions and clicks.

Comment 9

[Lorand Janos]

Since there is no QA needed here, I'll remove the qe-verify+ flag.