1658027 - Separate visual interactions on Credit Card Autofill for A/B testing

Status: VERIFIED FIXED

(Toolkit :: Form Autofill, task, P1)

Description

[Zibi Braniecki [:zbraniecki][:gandalf]]

We want to be able to run heuristics of credit card autofill, but control whether the UI interactions are displayed (dropdown and doorhanger).

Comment 1

[Zibi Braniecki [:zbraniecki][:gandalf]]

Attachment: Bug 1658027 - Separate visual interactions on Credit Card Autofill for A/B testing. r?abr

Comment 2

[Zibi Braniecki [:zbraniecki][:gandalf]]

I'm adding a single pref - extensions.formautofill.creditCards.hideui which has to be explicitly added and set, and which, when credit card autofill is enabled, will hide the UI interactions:

• doorhanger
• autofill popup
• Preferences UI

It still records the events around form detection and submission.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9825feb6a05e
Separate visual interactions on Credit Card Autofill for A/B testing. r=abr

Comment 4

[Atila Butkovits]

Backed out for failure at browser_creditCard_telemetry.js

Backout link: https://hg.mozilla.org/integration/autoland/rev/c33697aac4f1d9e4ca7ff4bfea5ea0bf80318519

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedTaskRun=MyVv7W_LR9m--x6XkHp2JQ.0&resultStatus=testfailed%2Cbusted%2Cexception&revision=9825feb6a05ef73b85130d9ddfa562eaebe02ba1&searchStr=bc

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=312467959&repo=autoland&lineNumber=6313

Comment 5

[Zibi Braniecki [:zbraniecki][:gandalf]]

Agh, a new test conflicted with the change from bug 1657909.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1cf57f1b0ed0
Separate visual interactions on Credit Card Autofill for A/B testing. r=abr

Comment 7

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/1cf57f1b0ed0

Comment 8

[Zibi Braniecki [:zbraniecki][:gandalf]]

Comment on attachment 9168925 [details]
Bug 1658027 - Separate visual interactions on Credit Card Autofill for A/B testing. r?abr

Beta/Release Uplift Approval Request

• User impact if declined: We need this pref for A/B testing
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: We need to verify that flipping extensions.formautofill.creditCards.hideui on and off makes the UI for autofill be visible, while cc_form detect/submit events go off for in both scenarios.
• List of other uplifts needed: None
• Risk to taking this patch: Medium
• Why is the change risky/not risky? (and alternatives if risky): This is intended to turn off the UI around credit card autofill, while preserving the heuristics around form identification. The code has not been originally written with such split in mind, therefore there's some risk associated with attempting to disconnect the UI from heuristics.

The mitigation is that the patch is very simple and attempts to be as non-invasive as possible.

• String changes made/needed:

Comment 9

[Cosmin Muntean [:cmuntean], Ecosystem QA]

I have verified this issue on the latest Nightly 81.0a1 build (Build ID: 20200809213940) on Windows 10 x64, Mac 10.14.6 and Ubuntu 20.04 x64. In order to verify this issue I have used the following scenarios:

• Verified that the extensions.formautofill.creditCards.hideui boolean pref is present in the laest Nightly 81.0a1 build.

• Verified that the extensions.formautofill.creditCards.hideui pref is set to "false" by default.

• Verified the following cases if the pref is set to true:

  • Verified that the pref remains set to true after restarting the browser.
  • Verified that the "Autofill credit cards" option from "about:preferences#privacy" is hidden.
  • Verified that the "autofill" is no longer displayed when completing a Credit Card form.
  • Verified that the "Save/Update your credit card" doorhanger is no longer displayed when submitting a Credit Card form.
  • Verified that the creditcard cc_form detected event is correctly registered if a credit card form is detected.
  • Verified that the creditcard cc_form submitted event s correctly registered if a credit card form is submitted.

• Verified that the Credit Card feature works and the UI is shown if the pref is changed back to "false".

Please let me know if there are any other scenarios that I could verify.
I will also verify this on Beta 80 as soon as the bug will be uplifted. However, I will leave the "qe-verify+" flag set and I will remove it after verifying the bug in Beta 80.

Comment 10

[Julien Cristau [:jcristau]]

Comment on attachment 9168925 [details]
Bug 1658027 - Separate visual interactions on Credit Card Autofill for A/B testing. r?abr

approved for 80.0b7

Comment 11

[Julien Cristau [:jcristau]]

https://hg.mozilla.org/releases/mozilla-beta/rev/11c649b83796

Comment 12

[Cosmin Muntean [:cmuntean], Ecosystem QA]

I have verified this bug on the latest Beta 80.0b7 (Build ID: 20200811133449) on Windows 10 x64, Mac 10.14.6 and Ubuntu 20.04 x64.

• In order to verify this issue, I have used the same scenarios from comment 9. During testing, I haven't encountered any new issues.