1869750 - [Experiment] The "Addons Reminder" message cannot be displayed with the current trigger configuration

Status: NEW

(Firefox for Android :: Experimentation and Telemetry, defect, P2)

Description

[Patriciu Pop [:ppop], Ecosystem QA]

[Affected versions]:

• Firefox for Android Release Version 120.0.1.1, Build ID 2015990546

[Affected platforms]:

• Android 11 - OnePlus 6T
• Android 13 - Samsung Galaxy A32

[Prerequisites]:

• Have the latest Firefox Release 120 installed.
• Meet the targeting criteria for a casual user (app launched more than 7 times in the last 28 days, but less than 13)
• Have a profile enrolled in the treatment branch of the "Homepage banner - Add ons Reminder test (Android)" experiment.

[Steps to reproduce]:

1. Open the browser with the profile from prerequisites.
2. Navigate to a website and open a new tab.
3. Dismiss the "Set as default browser message" and open a new tab.
4. Observe the homepage.

[Expected result]:

• The "Addons Reminder" messaging banner is displayed on the homepage.

[Actual result]:

• No messages are displayed on the homepage.

[Notes]:

• The issue is most likely caused by the current trigger parameters requiring a user to be a USER_CASUAL, USER_REGULAR, and USER_CORE_ACTIVE at the same time.
• We've edited the recipe locally and we were unable to trigger the message using only the USER_CASUAL trigger. We've tried opening the application once per day for 7 days, navigating to a website once per day for 7 days, and opening a link from a 3rd part app once per day for 7 days.
• We've edited the recipe locally and verified that the message is correctly displayed using the ALWAYS trigger.

Comment 1

[Shane Caraveo (:mixedpuppy)]

I am triaging the priority on this bug based on the importance for our project. I'll followup with issues around the triggers.

Comment 2

[James Hugman [:jhugman] [@jhugman]]

The experiment (https://experimenter.services.mozilla.com/nimbus/homepage-banner-add-ons-reminder-test-android/summary ) treatment branch has contradictory triggers:

{
  "messages": {
    "Add-ons_reminder_121": {
      "action": "OPEN_SETTINGS_ADDON_MANAGER",
      "button-label": "addon_ga_message_button",
      "experiment": "{experiment}",
      "style": "MY_STYLE",
      "surface": "homescreen",
      "text": "addon_ga_message_body",
      "title": "addon_ga_message_title",
      "trigger": [
        "USER_RECENTLY_UPDATED",
        "USER_CASUAL",
        "USER_REGULAR",
        "USER_CORE_ACTIVE"
      ]
    }
  },
  "styles": {
    "MY_STYLE": {
      "max-display-count": 10,
      "priority": 40
    }
  },
  "triggers": {}
}

@ppop is correct:

The issue is most likely caused by the current trigger parameters requiring a user to be a USER_CASUAL, USER_REGULAR, and USER_CORE_ACTIVE at the same time.

The trigger is a list of named expressions that all have to be true in order for the message to be eligible to be shown.

The trigger should be amended to only have one of "USER_CASUAL", "USER_REGULAR", or "USER_CORE_ACTIVE".

We've edited the recipe locally and we were unable to trigger the message using only the USER_CASUAL trigger. We've tried opening the application once per day for 7 days, navigating to a website once per day for 7 days, and opening a link from a 3rd part app once per day for 7 days.

This is a concern.

Comment 3

[Chris Peterson [:cpeterson]]

The Android team has not been keeping our P1 bug list up to date, so we're resetting all our P1 bugs to P2 to avoid signalling that we're actively working on bugs that we're not. The BMO documentation https://wiki.mozilla.org/BMO/UserGuide/BugFields#priority says P1 means "fix in the current release cycle" and P2 means "fix in the next release cycle or the following (nightly + 1 or nightly + 2)".

If you are actively working on this bug and expect to ship it in Fx 122 or 123, then please restore the priority back to P1.