1834760 - [Experiment] The ““Promote VPN on Firefox” spotlight is not dismissed after pressing “X” (close) button on Firefox Release 109 and 110

Status: VERIFIED FIXED

(Firefox :: Messaging System, defect)

Description

[Carmen Tataru [:ctataru], Ecosystem QA]

Attachment: Screen Recording 2023-05-24 at 14.25.46.mov

[Affected versions]:

• Firefox Release 109.0, BUILD ID: 20230112150232
• Firefox Release 110.0, BUILD ID: 20230214051806

[Affected Platforms]:

• Windows 10x64
• Windows 11x64

[Prerequisites]:

• Have the following prefs in the “about:config” page:
  - nimbus.debug set to true
  - browser.newtabpage.activity-stream.asrouter.devtoolsEnabled set to true
• Have the “browser.newtabpage.activity-stream.asrouter.devtoolsEnabled” pref set to true in the “about:config” page.

[Steps to reproduce]:

1. Open the browser with the profile from the prerequisites.
2. Navigate to the “about:studies?optin_slug=vpn-spotlight-modal-phase-1&optin_branch=treatment-a&optin_collection=nimbus-preview” for force enrollment.
3. Navigate to the “about:newtab#asrouter” page.
4. Select the “eco” option from the “Show messages from” section.
5. Search for the “vpn-spotlight-modal-phase-1-fr:treatment-a “ message.
6. Click the “Show” button.
7. Click the “X” (close) button from the “Promote VPN on Firefox” spotlight.
8. Observe the results.

[Expected results]:

• The “Promote VPN on Firefox” spotlight is dismissed.

[Actual results]:

• The “Promote VPN on Firefox” spotlight is not dismissed after pressing the “X” button.

[Notes]:

• The “click_button” and “dismiss” telemetry pings related to the “X” button are displayed in the “Browser Console”.
• The issue is reproducible on all the branches of the experiment.
• This issue is reproducible on all “Promote VPN on Firefox” experiments.
• Starting with the Firefox Release 111.0 build this issue is not reproducible.
• Attaching a screen recording of the issue.

Comment 1

[Emily McMinn :emcminn]

Turns out the DISMISS action was only added to all multistage screens in 111: https://bugzilla.mozilla.org/show_bug.cgi?id=1805928
(Previously, the dismiss button used navigate: true and only worked for single-screen messages or on the last screen of a message.)

Since this is a single screen message, we could fall back on the old navigate: true action for the dismiss button, which seems to send the same telemetry pings (NB this only works because it is a single screen spotlight; with more than one screen navigate: true does not send the DISMISS telemetry)

Current pings:

TELEMETRY PING (about:welcome): {"experiments":{},"locale":"en-US","version":"115.0a1","release_channel":"default","event":"CLICK_BUTTON","event_context":"{\"source\":\"dismiss_button\",\"page\":\"spotlight\"}","message_id":"FX_100_UPGRADE_0_UPGRADE_PIN_FIREFOX","addon_version":"20230524094121","client_id":"8b852d5e-5074-4caa-a8d3-da2183073b0f","browser_session_id":"c76f1987-9e20-4f35-8fb3-99a85c47ab5a"}

TELEMETRY PING (activity-stream): {"experiments":{},"locale":"en-US","version":"115.0a1","release_channel":"default","message_id":"FX_100_UPGRADE","event":"DISMISS","addon_version":"20230524094121","client_id":"8b852d5e-5074-4caa-a8d3-da2183073b0f","browser_session_id":"c76f1987-9e20-4f35-8fb3-99a85c47ab5a"}

Comment 2

[Carmen Tataru [:ctataru], Ecosystem QA]

I have verified that this issue is no longer reproducible using Firefox Release 109.0 (BUILD ID: 20230112150232) and Firefox Release 110.0.1 (BUILD ID: 20230227191043) on Windows 10x64 and Windows 11x64. The ““Promote VPN on Firefox” spotlight is dismissed after pressing “X” (close) button.
Considering this I think that we can close this issue.