1830316 - [Experiment] The Credit Card- Sync and Credit Card- Backup messages can’t be triggered in other regions than US,CA,GB,FR,DE as the Credit Card Autofill feature is enabled only in those countries

Status: RESOLVED WONTFIX

(Firefox :: Messaging System, defect)

Description

[Simona Rosu [:srosu], Ecosystem QA]

Attachment: CreditCardMessages.gif

[Affected versions]:

• Firefox Beta 113.0b8 (Build ID: 20230425192707)

[Affected Platforms]:

• Windows 10 x64
• Windows 11 x64
• macOS 11.7.1
• Ubuntu 22.04 x64

[Prerequisites]:

• Have the Firefox Beta 113.0b8 en-US build installed and opened.
• Have the “browser.search.region” pref set to “RO”.
• Have the “nimbus.debug” pref set to true in the “about:config” page.

[Steps to reproduce]:

1. Force enroll in the Treatment E branch of the experiment using the following link: about:studies?optin_slug=prompt-fxa-creation-after-first-bookmark&optin_branch=treatment-e&optin_collection=nimbus-preview
2. Navigate to https://codepen.io/quinlo/pen/YONMEa website and complete the fields.
3. Refresh the page.
4. Observe the left part area of the Address Bar.

[Expected result]:

• The “Save Credit Card” doorhanger is displayed.

[Actual result]:

• The “Save Credit Card” doorhanger is NOT displayed.

[Notes]:

• This issue is reproducible only on the experiment's Treatment E and F branches.
• This issue is not reproducible when the “browser.search.region” pref is set to US/CA/GB/FR/DE.
• Attached is a screen recording of the issue.

Comment 1

[Shane Hughes [:aminomancer]]

Thanks for testing! The ability to save and sync credit cards is only enabled in select regions by default, so this is the expected behavior. The "Save Credit Card" doorhanger isn't part of the experiment, but is just built into Firefox. The experimental payload for the credit card branches is actually the doorhanger that appears on the app menu button after clicking Save in the "Save Credit Card" doorhanger.

Comment 2

[Simona Rosu [:srosu], Ecosystem QA]

That's right, Shane! I filed this issue because I've observed in the experimenter ticket that this experiment will be launched to all countries, so the users that will have as default other regions than US, CA, GB, FR, and DE will not see the Credit Card - Sync/ Backup messages as they will not have the chance to click "Save" button from the Credit Card Autofill doorhanger as the feature is not enabled on those regions.

Comment 3

[Shane Hughes [:aminomancer]]

Yeah that's correct. Users who get enrolled in the credit card branches outside of the regions that support credit card autofill will see no message, since they won't be able to save a credit card anyway. The reach recorded for the credit card branches will reflect the reach the message would ultimately have if landed. Users in the regions where credit card autofill is unsupported can still see messages from the other branches, so there wouldn't be a good reason to change the experiment's region targeting.

The experiment itself will be launched without any regional restrictions, since there are no legal issues with any of the messages. But only a subset of users within each branch will be eligible to see the message. The ability to save credit cards is more restricted than the ability to save bookmarks or logins, since it's a newer feature, so those branches will ultimately have lower reach.

There are other restrictions as well though, like users who are already signed into FxA will not see any of the messages, even though they can enroll into the experiment. That is because it's possible for the user to sign into FxA after enrolling in the experiment, and we wouldn't want to show these messages to a user that's already signed in, so we check for FxA before showing the message, not just before enrolling in the experiment. That has the effect that some users will be able to enroll in the experiment but never see any message.

Comment 4

[Simona Rosu [:srosu], Ecosystem QA]

Thanks @Shane for the reply! If you want, you can close this as Wontfix.

Comment 5

[Shane Hughes [:aminomancer]]

No problem, thanks for verifying :)