Bugzilla

Comment 2

•

10 years ago

Quick feedback after initial review of the AutoComplete feature -- It was added to our default -- on desktop (Mac):

I just tested the form and autocomplete, and it’s definitely not making the experience smoother! 

I took screenshots using Chrome on my Mac Air laptop.

**Screenshot 1: https://www.evernote.com/shard/s202/sh/80b33172-df06-470b-83c0-5759bd24378e/ddf87d0819a48ef3b0a2859a6574b808/deep/0/Support-Mozilla.png
>> When I dropped my cursor into the first name field, this box popped up. It's overwhelming and likely to reduce conversion (esp. since it pops up on a browser, instead of mobile-only).

**Screenshot 2: https://www.evernote.com/shard/s202/sh/50f0adc7-323f-498a-92f2-3cec18ff7623/d360b12bb86ec4dda5fa4094060a3e57/deep/0/Support-Mozilla.png
>> When I select auto-fill my name disappears and my street address if duplicated in both address fields (I have no apartment #)

**Screenshot 3: https://www.evernote.com/shard/s202/sh/3372c215-9509-4263-917d-e8a55a5f0a02/6b5ac96c75611ea7080b09daeb4de55c/deep/0/Support-Mozilla.png
>> Our payment processor doesn't accept AMEX. I was rerouted to fill out the original form with a different card. Huge abandonment risk if our payment processor doesn't communicate with rAc about acceptable forms of payment.

**Screenshot 4: https://www.evernote.com/shard/s202/sh/d9b5ac8a-c295-414d-8465-7d17108195cc/6be7170c3694b42a1b1a8edc0de5fbb4/deep/0/Support-Mozilla.png
>> I tried the form again today and my autofill info is now included (though I still don't have wallet turned on)

Another major issue is that the form requires so many fields, including a phone #. This could be perceived as invasive and unnecessary PII from a user perspective if they simply want to make a one-time donation quickly.

Comment 3

•

10 years ago

We have created a duplicate donation form and added the script for rAc -- we can use this as a testing instance (we won't be making this form public)

https://sendto.mozilla.org/page/contribute/autocomplete-demo

Comment 4

•

10 years ago

Here is the note from Tom based on the edits you discussed over the phone -- 

to show on desktop in Chrome, use:
https://sendto.mozilla.org/page/contribute/autocomplete-demo?wallet=true

Otherwise, just use this for only showing on mobile:
https://sendto.mozilla.org/page/contribute/autocomplete-demo

Also, it will only show for user whose language is set to U.S. English, so that should do a pretty good job of limiting to U.S. based people, though it could also including some of Canada. Let us know if that works....

Thanks, and let me know if you have any other questions or requests for the autocomplete.

Katie

Comment 5

•

10 years ago

Hi Andrea.

Is this bug a duplicate of https://bugzilla.mozilla.org/show_bug.cgi?id=1007176 ? That bug has a lot of integration tips which address the four issues listed above.

WRT "Google Wallet is not supported with this merchant", that just means Wallet isn't ready for this site yet --- it's a simple process of adding sendto.mozilla.org to a whitelist. I've pinged someone about doing that (it should happen automatically when you start developing, but my email might expedite the process).

Comment 6

•

10 years ago

Also, when users do choose to use Google Wallet, the fronting card that requestAutocomplete returns will always be MC. sendto.mozilla.org will never see the backing card (i.e. the user's real card), and it won't matter what type it was. In this way, requestAutocomplete + Wallet allows users to donate with American Express with no changes to sendto.mozilla.org.

Comment 7

•

10 years ago

Yes, I closed the other bug #1007176 to avoid confusion (which was mostly mine). 

Now that the rAc is configured for mobile-only users we'll do more testing there. Thanks so much for the clarifications. And thanks for expediting whitelisting if possible for GW.

Comment 8

•

10 years ago

The other bug doesn't seem closed (still says Status: NEW). Which bug should I add future comments to?

Comment 9

•

10 years ago

Please use this bug, i'm working with Bobby to close it (I think he has to flip the switch).

Tae E. Kim

Comment 10

•

10 years ago

@andrea, just confirming that the whitelist should have gone through. Let us know if you continue to run into issues on that front. What should we set as a reasonable ETA to start A/B testing?

Comment 11

•

10 years ago

Attached image A_insertcursor.png — Details

Comment 12

•

10 years ago

Attached image B_MC_generated.png — Details

Comment 13

•

10 years ago

Attached image C_error.png — Details

Comment 14

•

10 years ago

Attached image D_noFnameLname.png — Details

Comment 15

•

10 years ago

Attached image E_insertcursor_infiniteLoop.png — Details

Comment 16

•

10 years ago

Hi Tae and Evan -- 

I just tested the donation page we set up for testing using my HTC One with Android on the Chrome browser. I wasn't able to complete a donation and ran into some issues. I've attached screenshots of the sequence:

Screenshots and explanations:

A_insertcursor.png: Google Wallet launched when I inserted my cursor into the First Name field.
B_MC_generated.png: Saw this popup that tells me a "fake" mastercard was generated by wallet. This was really confusing since the card I chose to use in Wallet *was* a Mastercard. I click continue.
C_error.png: The form throws an error saying three fields were missing (first, last, form of payment)
D_noFnameLname.png: I insert my cursor to type my first name in response to the error
E_insertcursor_infiniteLoop.png: Google Wallet launches again and the whole sequence appears to restart. 

In sum:
-- Why did wallet generate a "fake" MC even though I was using a MC?
-- There seems to be a glitch with the name fields not "holding" the autocomplete data. I'm not sure why "form of payment" was also missing - this results in an infinite loop.

Not ready for A/B testing :)

Comment 17

•

10 years ago

- Wallet will always generate a virtual card. See [1]. If the user opts out of Wallet, the card will be a "real" card.
- For the name, you should be using autocomplete="name"
- The test page should not show a form and then pop up requestAutocomplete when the user clicks in the first field. The form should be totally hidden if requestAutocomplete is available (except for the parts that requestAutocomplete doesn't support, like donation amount). A nice call-to-action button should be what launches requestAutocomplete.


[1] https://support.google.com/wallet/answer/2740044?hl=en

Comment 18

•

10 years ago

hi Evan, the coder working on this page is going to drop in some comments. The non profits link on this page is broken at the moment: https://checkout.google.com/seller/npo/index.html but we're looking for a proper button to drop on that page.

Comment 19

•

10 years ago

Ok, so currently I'm seeing these issues/questions in response to Comment 17:

:: We can't set autocomplete="name" -- Blue State Digital requires first and last (which is better for us anyway: if we don't get separate entries, then we can't use first-name personalization in email).

:: They say that the Google Wallet should be displayed with a call to action button, and that all user/address/CC fields should be hidden -- do we have design or text for a button? Is there a vision for how the rest of the form that needs to be there will flow around this button? This button does not appear non-profit oriented: https://checkout.google.com/seller/checkout_buttons.html

Comment 20

•

10 years ago

re: name. The best recommendation we have right now is to automatically parse the name: first word is first name, rest is last name. We're working on adding something more accurate than this into Chrome in the future. Out of curiosity, what would Blue State Digital do if first name or last name were empty?

re: the button. requestAutocomplete does not imply Google Wallet will be used. The button should not have Wallet branding. I'll attach a rough, non-designer's mock of what the page could look like.

Comment 21

•

10 years ago

Attached image mozilla_donation.jpg — Details

mock of page optimized for requestAutocomplete. Pressing "Donate now" with mastercard or visa selected should invoke rAc.

Comment 22

•

10 years ago

Thanks Evan, the developer working on this is out but returning tomorrow. I'll be connecting with him asap.

Comment 23

•

10 years ago

The changes were made to this page by our developer:
https://sendto.mozilla.org/page/contribute/autocomplete-demo

And I just made a wonderfully smooth 2-click donation!

My only recommendation is that after selecting the card, and after the popup says "your card was generated" it would be good to have a progress spinner or some indication that the pmt is being processed. It took a few seconds and donors might think they need to click the "donate" button again.

Updated

•

10 years ago

Comment 24

•

10 years ago

Hi Andrea,

Those edits, including the PayPal re-showing of fields, have been made!

Matthew N. [:MattN]

Comment 25

•

10 years ago

Hello, I'm working on requestAutocomplete for Firefox and have a suggestion:

From what I can see, the page isn't listening for the autocompleteerror event to know if there were errors during requestAutocomplete. It should at least check to see if the feature is disabled and probably return to the non-requestAutcomplete flow in that case.

See http://www.whatwg.org/specs/web-apps/current-work/#autocompleteerrorevent

Thanks

Comment 26

•

10 years ago

Good point. I have adjusted the page so that now if you cancel the Google Wallet popup, it will show the normal name/address/credit card fields, and alert you to the fact that you can fill them out.

Matthew N. [:MattN]

Comment 27

•

10 years ago

Thanks, though I personally think the alert provides a bad UX for the cancel/disabled/invalid reasons:
* disabled - If the feature is disabled, the user wouldn't even know it was going to use requestAutocomplete so we should simply show the form like the case where requestAutocomplete doesn't exist.
* cancel - If the user clicks cancel, simply showing the form seems like enough to me.
* invalid - I think the message should change to a normal validation error which may already exist.

Also note that |event| is undefined in Gecko on the line with event.preventDefault(). The event argument should be included as an argument to the onsubmit handler.

Comment 28

•

10 years ago

Hi Matthew,

The disabled and invalid conditions should already be handled.

I have adjusted the canceled condition to not show a dialog.

And I have adjusted the event line to work with in Firefox, once Firefox supports requestAutocomplete/Google Wallet.

Comment 29

•

10 years ago

Hey Tom, Andrew, how's progress? Any other questions we can help with?

Comment 30

•

10 years ago

Hi Evan, I think this is good to go. Unless you see any lingering issues on the page? Tom addressed the UX feedback and the PayPal hitch. So yeah i think we're golden on doing testing. What does that look like from your point of view? What kind of data would be most helpful?

Comment 31

•

10 years ago

I think the web page looks pretty good! The only issue I would work on is that the first time I went through, I didn't see the terms of service/privacy policy checkbox at the bottom. The big red donate button grabbed my attention (good!) but then I went through checkout and got an error (bad), which forced me to do checkout again. I think it's best to not let the user go through checkout twice. One quick fix would be to pop up the alert box when I first press "Donate now", instead of waiting till after checkout. There might be even better treatments than an alert box (red outline on checkbox label?), but the main thing is to avoid double checkout.

For testing the efficacy of requestAutocomplete, I think it would be best to enable it on all platforms that support it (as detected by the presence of form.requestAutocomplete). Then we can compare data from

a) before and after you enabled it
b) platforms that do have it and platforms that don't have it

The "data" in question would be stuff like:
1) conversion rate (i.e. number of visitors who contribute)
1b) number of times there's some error charging the card that the user enters
2) time on page
3) behavior if/when user bounces without donating (e.g. did they click around then get stuck, did they enter some info, see an error they couldn't fix, then abandon, etc.)
4) donation amount

etc. Of course (1) is easily the most important one of these, but the other data also help us analyze user behavior and make improvements. Also, logging all this by user agent will let us get a better picture of its true impact (you'd expect greater effect on mobile than desktop, for example).

Hopefully you already track some or all of these metrics so that we have a baseline.

Comment 32

•

10 years ago

I have addressed the privacy policy checkbox issue -- it will popup a dialog prior to showing Google Wallet. Hopefully we should now be all set!

Comment 33

•

10 years ago

Thanks Tom!

Evan and all, I am going to create a new ticket for this testing. I'll cc everyone. I need to check with Adam Lofting who is our internal metrics guru on how we can get this setup properly.

Comment 34

•

10 years ago

Hi Andrea. Did you get a chance to file that ticket? I can't seem to find it.

Adam Lofting (:adamlofting)

Comment 35

•

10 years ago

Hi Andrea,

The most basic testing we would like is just to know conversion rate by platform/device, or value per visit by platform/device. If you have kept track of that in the past, it should be easy to see how turning on rAc affects it.

Would you be able to confirm whether your track this kind of analytics data, which would help us to move forward with pushing this live? 

Thanks.

p.s. I think it would be best to turn it on for all platforms that support requestAutocomplete, rather than restricting to mobile devices.

Comment 36

•

10 years ago

I suggest we put live a duplicate of the donation form with the new autocomplete feature enabled. This would be functional, but not linked to directly from any of our fundraising asks.

Then, I can setup Optimizely to split 50% of the traffic from the existing page to the new page, and we'll have a controlled test.

Comment 37

•

10 years ago

Our recommendation in the past has been to simply turn it on at 100% (aside from whatever new-feature turnup procedures you have in place). The main reason for this is that requestAutocomplete will work better for repeat users as it remembers their data. These repeat users may visit across different devices, so it would be hard to permanently correctly bucket them in the "on" or "off" category. If you turn on rAc at 100%, you can still get good data by, e.g., looking at how rAc platforms fared compared to non-rAc platforms, in the month before and the month after enabling.

That said, I'm not sure how many repeat visitors you have over the period of a month.

Comment 38

•

10 years ago

Hi Evan - our donors typically give 1x per year, a minority give 2x. So data on repeat giving is unlikely to be significant. Here is the form we'll be turning on for the masses soon:
https://sendto.mozilla.org/page/contribute/Give-Mobile

We've been digging into our giving data and confirmed that we have relatively low mobile traffic to our donation forms. Not surprisingly most people who land on our donation form are using Firefox desktop browser (over 90%).

That could mean there may be some time before we collect enough data to be useful. There are opportunities to increase traffic to this form in the coming months. We will launch a small email fundraising campaign related to Webmaker at the end of September. But by far the biggest traffic of our entire year happens in late November /  December during our year-end fundraising campaign. During that period we raised over $1 million last year. I realize that potentially means waiting until late November to start getting good data, but it's basically a guaranteed period of rich traffic to this form.

All that is to say, we can switch the new form on and start collecting data, but we may not get statistical significance until our year-end fundraising campaign launches in late November.

Comment 39

•

10 years ago

Thanks for the info. I wonder if Firefox will have requestAutocomplete ready by that time[1].

Given the low instance of repeat traffic, it would probably be OK to do a more traditional A/B test, i.e. enable rAc at some percentage split (50/50 maybe). That allows us to do a more direct comparison and we're not losing much in terms of the value of rAc to potential repeat donors.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=990367