Closed Bug 662177 Opened 14 years ago Closed 12 years ago

Show sign up captcha only in exceptional cases

Categories

(Cloud Services :: Operations: Miscellaneous, task)

Product:
Cloud Services
Component:
Operations: Miscellaneous
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED INVALID

People

(Reporter: philikon, Unassigned)

References

Details

(Whiteboard: [ux-eye][qa?][closeme-sync.next][sync:setup])

I'm proposing that under normal circumstances, we no longer show the sign up captcha. Reasoning: our sign up process right now has many sticks before the user can enjoy the carrot that is Sync. The captcha is arguably a pretty big piece of that stick. My anecdotal evidence suggests that users find captchas simply annoying. "Please prove that you're not a robot" seems like a poor excuse. Really, we can't tell abuse otherwise? Actually, we can. We already do with the key exchange server. So I propose let's do a similar thing with the auth server: don't make users go through captchas. But if they're connecting from an IP that is flagged because we've gotten an unusual number of requests from that IP, we can still show it. We can implement this today by returning a 404 from the server, the client will understand it. It might be a bit ugly until bug 658675 has made it into a release, but I don't think we should let that stop us.
Adding dmills to make sure there aren't implications for browserID stuff, but overall I'm of course in favor since this provides a better experience :)
Joke from the earlier meeting: we're a browser, so technically we can watch how many times we've seen this user successfully complete a captcha as they browse the Web :p
That's of course assuming that malware on the user's machine isn't trying to create an account on their behalf, and leveraging the fact that they've been behaving like a human in the past.
This will, of course, depend on some sort of abuse/correlation system getting hooked up so we have some way to detect this case. And, probably, quotas in place to make us even less of a tempting target.
Target Milestone: --- → Future
Not to rain on your party but in the ID user testing the captcha was the least of our problems. Have you talked with Diane about running some quick tests to identify whether there is something else that is stopping people from signing up? That aside, I'm not sure whether we need a captcha before making verified email assertions. Maybe we don't, I will try to find out (though it's hard to get this data--we're talking RP psychology here). I suggest you keep a boolean (has solved captcha) for each user so that we can ask for the captcha later if it turns out we need to.
(In reply to comment #5) > Not to rain on your party but in the ID user testing the captcha was the > least of our problems. That's good to hear. Certainly, our current sign up process has other problems. But any step that's eliminated, especially something as annoying as a captcha, is a win. > Have you talked with Diane about running some quick > tests to identify whether there is something else that is stopping people > from signing up? We have not, and it's a great idea to do that, but it's quite orthogonal to making the captcha optional. > That aside, I'm not sure whether we need a captcha before making verified > email assertions. Maybe we don't, I will try to find out (though it's hard > to get this data--we're talking RP psychology here). I don't understand what that means. Are you saying the captcha is also for the user's benefit? It seems to only be for our (the service's) benefit, and given how laborious captchas are, that's how users see it. > I suggest you keep a boolean (has solved captcha) for each user so that we > can ask for the captcha later if it turns out we need to. We could certainly do that. We could also decide dynamically at a later point that we'd like the user to enter a captcha, e.g. when they change a password, or do other things to their account.
(In reply to comment #6) > > Have you talked with Diane about running some quick > > tests to identify whether there is something else that is stopping people > > from signing up? > > We have not, and it's a great idea to do that, but it's quite orthogonal to > making the captcha optional. On our tests, zero users decided to stop the sign-up process because of the captcha. Our sample size was small, but I would be surprised if you are losing a sizable percent of users at this step. So, I'm not arguing about correctness, I'm just suggesting you at least determine whether your energy might be better spent here or somewhere else. FWIW, Alex & Diane set up and executed the ID user test in all of 2 weeks, and we could've potentially done it even quicker (I was on PTO for a while, etc). The process was positively awesome, and we learned a ton. > > That aside, I'm not sure whether we need a captcha before making verified > > email assertions. Maybe we don't, I will try to find out (though it's hard > > to get this data--we're talking RP psychology here). > > I don't understand what that means. Are you saying the captcha is also for > the user's benefit? It seems to only be for our (the service's) benefit, and > given how laborious captchas are, that's how users see it. With Verified Email there is another party involved, the "relying party". This would be the website you are signing into, and they rely on us (the "identity provider") to assert to them who we claim the user is. The stronger our account processes are, the more reliable our assertions will be, and the higher the value to the RP to use our system. For example, if we guarantee our users have been captcha'ed, it is possible the RP can skip a captcha step when signing into their site. > > I suggest you keep a boolean (has solved captcha) for each user so that we > > can ask for the captcha later if it turns out we need to. > > We could certainly do that. We could also decide dynamically at a later > point that we'd like the user to enter a captcha, e.g. when they change a > password, or do other things to their account. Right, or in the case I had in mind: when you attempt to sign into another site using us as the ID provider.
>On our tests, zero users decided to stop the sign-up process because of the >captcha. yeah, but they were specifically instructed to finish. A better way to gauge this would be having test pilot monitor drop off through the process for a large set of users.
>FWIW, Alex & Diane set up and executed the ID user test in all of 2 weeks, and >we could've potentially done it even quicker (I was on PTO for a while, etc). >The process was positively awesome, and we learned a ton. I'm all for having a bit more informed view before moving forward, especially given some of the technical hurdles this may entail. If we find out indeed it's a big drop off point we'll work through those hurdles. Alex, do I talk to Jinghua about having a test pilot monitor?
>Alex, do I talk to Jinghua about having a test pilot monitor? yep, or Jono
Thanks! I'm on it.
Is this relevant any more? We're very unlikely to mess with either the client or server signup process before we switch over to BID, which has a totally separate auth method.
Whiteboard: [ux-eye] → [ux-eye][qa?]
Blocks: 780477
See Bug 780477. I think Persona login is sufficiently [future] that this is worth solving, but close enough that we don't care too much if we don't have a perfect (de-) implementation. It sounds like this is a small amount of server-side work, and (nearly?) nothing in the client. telliott?
Target Milestone: Future → ---
There's the suggestion, as rnewman succinctly phrased it, that "it's mostly pointless, harms signups, is pretty ugly, etc." We've been talking about disabling it for quite a while now, and apparently the audio version of captcha is now horked. Seems reasonable to try disabling it for a bit, to ask a couple simple questions: 1) Do we see a rise in registrations? Should be pretty easy to see in the medium-term - is there an upwards increase in the registration slope over a few weeks/months? 2) Do we see a rise in fake accounts? Obviously, the main concern is someone trying to DDOS account creation. However, there are already DDOS vectors in the system that, while perhaps not as easy, show no signs of being tried. That being said, ops monitoring should keep an eye out for sudden spikes in registration and reenable captcha to combat it if necessary. Also, we'll want to keep an eye out for new accounts that don't conform to our standard munged username. Those are highly unlikely at this point. I don't endorse turning off captcha in account portal at this time. We use it there for password resets, and it's reasonable to have it in place for that. Enabling/disabling captchas is a simple change in production, just toggling the captcha.use config variable and restarting the gunicorn workers.
Moving to operations for action. They can return it to Registration for discussion followup if needed.
Component: Server: Registration → Operations
Blocks: 691851
Any traction on this, folks? According to Bug 780477, we're screwing visually impaired users by continuing to require captcha.
Whiteboard: [ux-eye][qa?] → [ux-eye][qa?][closeme-sync.next][sync:setup]
this is still hurting people as brought up in this sumo thread for example: https://support.mozilla.org/questions/975474 (when i tried the audio captcha for myself it was totally inaudible and i wasn't able to solve it either)
This sign in model will change completely with the use of Firefox Accounts in an upcoming release of Firefox (early 2014). It will use email verification rather than Captcha.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → INVALID
(In reply to James Bonacci [:jbonacci] from comment #19) > This sign in model will change completely with the use of Firefox Accounts > in an upcoming release of Firefox (early 2014). It will use email > verification rather than Captcha. So why not turn off the captcha now for existing Sync?
Awesome idea. No Stage environment to test this out. I would not feel comfortable just "shutting it off" w/o some testing. Thoughts?
We've known the client can handle it since its inception -- indeed, self-hosted users don't have captchas, and Firefox handles that fine. This should be as simple as 404ing the captcha URL on the server (which I think is a configuration parameter). We've also broken user signup before for extended periods, so apparently that's not a big deal. Seems straightforward enough to make the server configuration change, push it out, run through a couple of signups, and roll back if it regressed anything.
Agreed. Would love to hear from the following people on this: :bobm :rfkelly :telliott If it's a thumbs up from all - I will reopen this ticket and assign to OPs.
I'm fine just turning it off (it's a config parameter). If ops sees a substantial spike in new accounts (as opposed to a small spike), we may need to revisit.
:thumbsup: turn it off, monitor signups, re-enable if thing go pear-shaped.
You need to log in before you can comment on or make changes to this bug.