592465 - Show a passphrase strength meter for custom passphrases

Status: VERIFIED FIXED

(Firefox :: Sync, defect)

Description

[Philipp von Weitershausen [:philikon]]

We need to discuss what metrics it should use.

Comment 1

[Alex Faaborg [:faaborg] (Firefox UX)]

copying over from an email I sent:

>I'm very interested in learning how these metrics work, since I
> would really like us to mention how long it would take someone to
> break the automatically generated sync key with a brute force attack
> on the physical artifact. Do we think it is fair to estimate it to
> "years" (assuming of course that no one creates a quantum computer, or
> time travel).

Comment 2

[Philipp von Weitershausen [:philikon]]

Time to break grows exponentially with the number of key bits. The more diverse your choice in characters is the more bits per character you have and the longer your passphrase is you have the more bits add up.

We could analyze the passphrase and see if it only contains lower case letters (4.7 bits per character), letters + numbers (5.2 bits) uppercase + lowercase + numbers (6 bits) or even punctuation (...) and unicode characters (8 bits due to UTF-8) and then do the exponential thing. Just be warned that our 20 character a-z Sync Key has 94 bits which, assuming 1 billion operations per second, would take 628077138145 years to break. Do we really want to show these numbers in the UI? Are they even meaningful to users?

There's another aspect. Long dictionary words / phrases may appear to be safe from the brute force point view but susceptible to a dictionary attack, *especially* if background knowledge is involved.

Comment 3

[Ed Lee :Mardak]

(In reply to comment #2)
> unicode characters (8 bits due to UTF-8)
Shouldn't this depend on the character? Like 好 converts to 3 characters. So we'll want to make sure the strength calculator factors that in.

> our 20 character a-z Sync Key has 94 bits which, assuming 1 billion operations > per second, would take 628077138145 years to break.
That's a pretty big number ;) I'm assuming Alex doesn't want the traditional weak/ok/strong, but we don't want to give users the wrong impression of security either.

Also, these strength calculations are assuming somebody has gotten your encrypted private key or Mozilla has gone malicious, so I'm not sure how you would put context to the displayed number.

Comment 4

[Philipp von Weitershausen [:philikon]]

(In reply to comment #3)
> (In reply to comment #2)
> > unicode characters (8 bits due to UTF-8)
> Shouldn't this depend on the character? Like 好 converts to 3 characters. So
> we'll want to make sure the strength calculator factors that in.

Yes of course, forgot to mention that under UTF8 the length would change accordingly. We should use the UTF8 representation since that's what gets used as the passphrase.

> That's a pretty big number ;) I'm assuming Alex doesn't want the traditional
> weak/ok/strong, but we don't want to give users the wrong impression of
> security either.

That's my concern, too.

> Also, these strength calculations are assuming somebody has gotten your
> encrypted private key or Mozilla has gone malicious, so I'm not sure how you
> would put context to the displayed number.

Very good point. Though the respective wizard page is titled "Firefox Respects Your Privacy" so you could interpret this as "even if Mozilla wanted it would take them XYZ bajillion years".

Comment 5

[Alex Faaborg [:faaborg] (Firefox UX)]

>these strength calculations are assuming somebody has gotten your
>encrypted private key or Mozilla has gone malicious, so I'm not sure how you
>would put context to the displayed number.

This is a bit of a paradox with the entire feature, for technical users it does kind of look like we are protecting them from us.  On the printed sync key this is something that are trying to directly address (since we have more room for context).

For non technical users, I think they will view this as more of "if someone sat down and started to try to guess, how long would it take them" now of course in reality it's a computer guessing at 1 billion operations per second, but same general idea. Our objective is to warn them that 1234 isn't good enough.  (they aren't going to put together the hypothetical scenario that either we've leaked your data or are trying to crack it ourselves).

>Do we really want to show these
>numbers in the UI? Are they even meaningful to users?

I think they are meaningful in terms of orders of magnitude of time, so:

"Estimated time to force access: n second(s)"
"Estimated time to force access: n minute(s)"
"Estimated time to force access: n hour(s)"
"Estimated time to force access: n day(s)"
"Estimated time to force access: n months(s)"
"Estimated time to force access: n years(s)"

After you reach years, we don't have to be as we can use intervals of 1,000 / 10,000 / 100,000 / 1 million / and then "more than a billion"

We should have some of those strings from work on the download manager.

Comment 6

[Mike Connor [:mconnor]]

(In reply to comment #5)
> >these strength calculations are assuming somebody has gotten your
> >encrypted private key or Mozilla has gone malicious, so I'm not sure how you
> >would put context to the displayed number.
> 
> This is a bit of a paradox with the entire feature, for technical users it does
> kind of look like we are protecting them from us.

Well, we are in fact protecting them from us, as the service operator.  That's tricky to message, but the reality is that we built a system that doesn't require you to trust the server operator, on purpose.

Comment 7

[Axel Hecht [:Pike]]

(In reply to comment #2)
> Time to break grows exponentially with the number of key bits. The more diverse
> your choice in characters is the more bits per character you have and the
> longer your passphrase is you have the more bits add up.

Statistically, 
'aaaaaaaaaa' 
is just as likely as 
'AbCd*2fHu!'
as the bits you have to check are the same. Unless you design the system such that it only allows 'a' as characters in the password. But given an allowed range of input, both are equally statistically probable.

Comment 8

[Philipp von Weitershausen [:philikon]]

(In reply to comment #7)
> (In reply to comment #2)
> > Time to break grows exponentially with the number of key bits. The more diverse
> > your choice in characters is the more bits per character you have and the
> > longer your passphrase is you have the more bits add up.
> 
> Statistically, 
> 'aaaaaaaaaa' 
> is just as likely as 
> 'AbCd*2fHu!'
> as the bits you have to check are the same. Unless you design the system such
> that it only allows 'a' as characters in the password. But given an allowed
> range of input, both are equally statistically probable.

You're right, of course. This applies to the random passphrase we generate for the user. Once the let the user enter their own passphrase, however, things change. Heuristics say that humans pick certain letters and numbers more often than others, which is why the NIST algorithm assigns less entropy to characters at the end than those at the beginning of a human generated password.

Comment 9

[Philipp von Weitershausen [:philikon]]

By "NIST algorithm" I mean NIST Special Publication 800-63 Appendix A.1 btw. See http://csrc.nist.gov/publications/nistpubs/800-63/SP800-63V1_0_2.pdf

Comment 10

[Axel Hecht [:Pike]]

I'm not convinced that we can gauge password strength without giving attackers hints on how to reduce the space of passwords to brute-force attack. We're big, and we're open source, which makes us a good target for fine-tuning algorithms.

That the result is globally harder to hack ... I don't know.

Comment 11

[Philipp von Weitershausen [:philikon]]

(In reply to comment #10)
> I'm not convinced that we can gauge password strength without giving attackers
> hints on how to reduce the space of passwords to brute-force attack. We're big,
> and we're open source, which makes us a good target for fine-tuning algorithms.

By default we auto-generate an entirely random phrase. We and any attacker already know its strength: 20 characters a-z = 94 bits of entropy. That's why -- assuming I interpret faaborg's mockups (bug 589980) correctly -- we only show the strength meter when the user chooses to enter a *custom* passphrase, simply to ensure they don't enter something with a ridiculously low entropy.

Comment 12

[Axel Hecht [:Pike]]

I'd go for just length, and ignore any bits per byte speculation.

(And don't use seconds? like, that's just ridiculous)

(And the bar can go to a limit?)

I'd ask someone to revisit that mockup, tbh.

Comment 13

[Mike Connor [:mconnor]]

Axel, you're using words like "ridiculous" and "speculation" without providing any tangible evidence/argument for why a) we shouldn't follow the NIST algorithim or b) why seconds is an unrealistic time to brute force trivially short keys.  If you have links that back up your assertions, great, please share them.  Otherwise you're not really doing anything productive...

Comment 14

[Axel Hecht [:Pike]]

a) is answered. We're too big to assume that nobody would tweak NIST to benefit from our guidance. Not gonna read the 64 pages and write a thesis on this. Not that the comments in this bug indicated anything founded as strongly as a 64 pages paper, too.
b) 1 year old DELL PC vs a hand full of ASICs, or cleverly coded graphics cards. What do you use? Would an attacker throw *real* money at the problem? That's easily an order of magnitude. Also, those are averages that are non-understandable by average users.

Comment 15

[Philipp von Weitershausen [:philikon]]

(In reply to comment #14)
> a) is answered. We're too big to assume that nobody would tweak NIST to benefit
> from our guidance. Not gonna read the 64 pages and write a thesis on this. Not
> that the comments in this bug indicated anything founded as strongly as a 64
> pages paper, too.

The "algorithm" itself is described on the ~4 pages of Appendix A.1.

I'm not sure I understand your concern. Let's say we implement a NIST-like algorithm in which we assign a certain number of bits of entropy to each character and then give bonus bits for different composition rules (upper and lower case, numbers, punctuation, non-ASCII are what I have on my list). Users would then hopefully take the advice the meter gives them and make their passphrases stronger by applying as many composition rules as they can and know. How is this bad for us? How does this give an attacker any more of an angle than they would already have?

In the end user generated passphrases are what they are: not very random and susceptible to dictionary attacks. That's why we're trying to encourage to stick with the generated Sync Key or, in case of a user generated one, a passphrase that's less likely to be susceptible to dictionary attacks. I wonder how this is worse than not doing it.

> b) 1 year old DELL PC vs a hand full of ASICs, or cleverly coded graphics
> cards. What do you use? Would an attacker throw *real* money at the problem?
> That's easily an order of magnitude. Also, those are averages that are
> non-understandable by average users.

I kind of agree that "Time to brute force attack" isn't such a useful measure, mostly because it's simple to get to a time of hundreds of years with very simple passphrases that could be susceptible to much faster dictionary attacks. That said, I think it's most meaningful measure (meaningful to users) to give them a qualitative idea of how secure their data is. "Time to brute force attack" is something tangible, even if it's off by an order of magnitude (we can be conservative and assume a real powerful computer). The time scale goes exponentially with the number of bits of entropy anyway, so the only sensible way to look at this is on a logarithmic scale.

Comment 16

[Richard A Milewski[:richard]]

As it says in http://xkcd.com/792/ "Password entropy is rarely relevant".  The big risk is that a user will reuse a phrase in dozens (hundreds?) of places.
One of those *will* be compromised eventually.  

If you're trying to protect the users from picking weak pass phrases, then you really need to give them a  S I M P L E  way to create a strong one that's easy to remember.

I've been struggling with that problem re passwords. 
See http://people.mozilla.com/~rmilewski/topics/passwords/master/

In the case of Sync, we're cajoling users into either choosing strong pass phrases or accepting our auto-generated (guaranteed strong?) ones.    ...then we're giving them the option to email them to themselves in unencrypted emails.

This presents a seriously mixed message.

Comment 17

[Alex Faaborg [:faaborg] (Firefox UX)]

>In the case of Sync, we're cajoling users into either choosing strong pass
>phrases or accepting our auto-generated (guaranteed strong?) ones.

Since we default to the automatically generated one, the vast majority of users aren't going to go to the extra effort to modify it.  We are mostly providing this option to the small minority of users that might feel more comfortable by generating their own key.

>then we're giving them the option to email them to themselves
>in unencrypted emails

The only alternative is to have millions of people experience data loss as they lose the saved or printed sync key.

Comment 18

[Axel Hecht [:Pike]]

I have read in particular Appendix A.2 by now, and that's a sorry flock of assumptions and "we think", "we consider", etc. I was hoping for some better research, and found http://csrc.nist.gov/publications/PubsDrafts.html#SP-800-118, which doesn't make me puke ad hoc. I've read less of that even, and found it much more educating. Section 3.2 there is much better than the appendices from 2006.

Comment 19

[Philipp von Weitershausen [:philikon]]

Attachment: v0.1

Implement a strength meter according to the NIST algorithm. Not the final patch yet (lacking unit tests for instance), trying to do this incrementally.

Also looking for feedback from faaborg on the UX part as this differs from the mockup: The general feeling was that a dimension less meter would be better as it's hard to guess how long it would take to do a brute force attack. Instead we're just displaying a meter with our 94 bits of entropy from the 20 char random passphrase as a yard stick. Note that it's really hard to get to those 94 bits with the NIST algorithm. For that reason we added a link a la "What does this strength mean" that would take you to a website (perhaps SUMO) explaining this better.

Comment 20

[Philipp von Weitershausen [:philikon]]

Attachment: screenshot weak

Comment 21

[Philipp von Weitershausen [:philikon]]

Attachment: screenshot medium

Comment 22

[Philipp von Weitershausen [:philikon]]

Attachment: screenshot strong

Comment 23

[Mike Connor [:mconnor]]

(In reply to comment #17)
> >then we're giving them the option to email them to themselves
> >in unencrypted emails
> 
> The only alternative is to have millions of people experience data loss as they
> lose the saved or printed sync key.

Millions is rather strong.  If people are expecting Sync to be a backup service, we need to change that positioning.  It's not, and we are not planning on building out the service to have that type of redundancy, at least for free accounts.

The only dataloss happens if you lose your sync key _and_ you lose your local storage without backups/another machine.  But unless we're a backup service, I don't think that's a case we're trying to guard against.

Comment 24

[Ed Lee :Mardak]

Not sure what the final meter is supposed to look like, but it might be useful to have an end-bar or border around the meter to help indicate that the weak red bar is actually really low.

Comment 25

[Alex Faaborg [:faaborg] (Firefox UX)]

Comment on attachment 476344 [details]
screenshot weak

I would really prefer that we could give a time estimate, since I think that makes the strength considerably more real to users.  I understand that no algorithm is exact, but it seems like regardless of fidelity, there's a difference between 5 minutes and 62 billion years.

We shouldn't over engineer this and refuse to provide an estimate if it might be wrong.  If someone hacks the password in 40 billion years instead of 62 billion, we aren't going to be around to be embarrassed about that.

That said, if we absolutely feel that we lack the confidence to provide a reasonable estimate, this interface still sort of works.

Comment 26

[Alex Faaborg [:faaborg] (Firefox UX)]

>Not sure what the final meter is supposed to look like, but it might be useful
>to have an end-bar or border around the meter to help indicate that the weak
>red bar is actually really low.

We'll be going back to do a lot of little of polish and style improvements later.

Comment 27

[Philipp von Weitershausen [:philikon]]

(In reply to comment #25)
> I would really prefer that we could give a time estimate, since I think that
> makes the strength considerably more real to users.  I understand that no
> algorithm is exact, but it seems like regardless of fidelity, there's a
> difference between 5 minutes and 62 billion years.

Sure, but the actual number is pretty useless. If the goal is to give users a rough idea of how long it would take somebody, perhaps we should just say "a few seconds", "several minutes", ..., "many years", etc. At least that way we wouldn't pretend to have an exact answer.

Comment 28

[Richard A Milewski[:richard]]

The problem with showing any time estimate is that it strongly implies that a brute-force attack the the only compromise risk.   A high-entropy password that gets reused on a hundred sites has a strength measured in milliseconds if one of those sites is like RockYou.com.

A high-entropy password copied out of a blog post on how to make secure passwords is just as bad.

Anyone have stats?  I suspect brute force attacks are the exception.

Comment 29

[Philipp von Weitershausen [:philikon]]

(In reply to comment #28)
> The problem with showing any time estimate is that it strongly implies that a
> brute-force attack the the only compromise risk.   A high-entropy password that
> gets reused on a hundred sites has a strength measured in milliseconds if one
> of those sites is like RockYou.com.
> 
> A high-entropy password copied out of a blog post on how to make secure
> passwords is just as bad.

Of course. If wanted to protect users from reusing passwords for their passphrase or picking obvious choices ("password", "12345678", etc.), then perhaps we should prevent them from entering their own passphrase in the first place. "Here's your Sync Key. guaranteed 94 bits of entropy. Write it down. Keep it secret." Works for me!

Comment 30

[Mike Connor [:mconnor]]

Comment on attachment 476343 [details] [diff] [review]
v0.1

>diff --git a/services/sync/services-sync.js b/services/sync/services-sync.js

>+pref("services.sync.syncKeyHelpURL", "https://services.mozilla.com/help/synckey");

We'll want this to be a SUMO link.  We can do that as a followup bug though, as we'll need to have a document name.  Probably prefs-sync-key is fine (and we should file on renaming prefs-weave to prefs-sync while they're adding the redirect for prefs-sync-key.  We should also file a bug on updating that documentation, if we haven't already.

>+    meter.value = bits;
>+    if (bits > 63)
>+      meter.className = "strong";

64 bits is strong enough?  I thought we were using 94 as our baseline for "strong" here?

Comment 31

[Ragavan S [:rags]]

I'm in favor of doing something simple, well understood and informative to users. I actually rather like the strength meter implementation for Google Accounts. Users understand the utility via a simple visual feedback indicator and label. There's a helpful link to what makes a strong password and how to create one.

I do fear that just showing time to break a passphrase only addresses one attack vector (brute force) while others such as social engineering attacks might be more prevalent/riskier.

Comment 32

[Philipp von Weitershausen [:philikon]]

(In reply to comment #30)
> Comment on attachment 476343 [details] [diff] [review]
> v0.1
> 
> >diff --git a/services/sync/services-sync.js b/services/sync/services-sync.js
> 
> >+pref("services.sync.syncKeyHelpURL", "https://services.mozilla.com/help/synckey");
> 
> We'll want this to be a SUMO link.  We can do that as a followup bug though, as
> we'll need to have a document name.

Yeah. Alternatively we can have this URL point to s.m.c and let s.m.c redirect to wherever we want. That way we can change where it goes for everybody rather than just those who run the latest version.

> >+    meter.value = bits;
> >+    if (bits > 63)
> >+      meter.className = "strong";
> 
> 64 bits is strong enough?  I thought we were using 94 as our baseline for
> "strong" here?

Oh yeah, right. So right now 100% on the meter is 94 bits. Do we want to scale the meter from 0 to, say, 128 and consider everything >=94 "strong"?

Comment 33

[Mike Connor [:mconnor]]

Yeah, that WFM.

Comment 34

[Philipp von Weitershausen [:philikon]]

Attachment: v1

Changes from v0.1 patch:
* Meter displays range 0..128 bits, 94 is considered "strong" (=green)
* Do not show meter if there's an error with the custom passphrase (e.g. too short)
* Show meter in the Sync Key dialog.

Comment 35

[Philipp von Weitershausen [:philikon]]

Attachment: screenshot: generated key

Comment 36

[Philipp von Weitershausen [:philikon]]

Attachment: screenshot: custom key too short

Comment 37

[Philipp von Weitershausen [:philikon]]

Attachment: screenshot: custom key, weak

Comment 38

[Alex Faaborg [:faaborg] (Firefox UX)]

Comment on attachment 477521 [details]
screenshot: generated key

let's use capital letters for the sync key, easier to read and more consistent with all of the other cases where users encounter activation codes.  Also it doesn't initially appear is if it should be containing words.

Comment 39

[Philipp von Weitershausen [:philikon]]

(In reply to comment #38)
> let's use capital letters for the sync key, easier to read and more consistent
> with all of the other cases where users encounter activation codes.  Also it
> doesn't initially appear is if it should be containing words.

We chose lower case for a reason: it's easier to enter, *especially* on mobile phones. Also, it has been this way ever since bug 591118 landed, why the sudden change of opinion now? I'm not dismissing this, but if we want to change our mind like this, let's file a new bug and discuss it there instead of doing it on this potential beta7 blocker ;) (that doesn't have anything to do with the generated Sync Key anyway).

Comment 40

[Mike Beltzner [:beltzner, not reading bugmail]]

If this has strings, it blocks beta7+ until at least those strings are landed.

Comment 41

[Mike Beltzner [:beltzner, not reading bugmail]]

Oops, blocked Firefox over in bug 598641; sorry for churn.

Comment 42

[Philipp von Weitershausen [:philikon]]

Attachment: v1.1 as landed

Now with unit tests for Weave.Utils.passphraseStrength() which I forgot to add to v1.

Comment 43

[Philipp von Weitershausen [:philikon]]

http://hg.mozilla.org/services/fx-sync/rev/ca0b9f550f2e

Comment 44

[Alex Faaborg [:faaborg] (Firefox UX)]

>We chose lower case for a reason: it's easier to enter, *especially* on mobile
>phones. Also, it has been this way ever since bug 591118 landed, why the sudden
>change of opinion now?

Sorry, I didn't catch that earlier, filed bug 598840 to follow up.

Comment 45

[Tony Chung [:tchung]]

Verified fix on Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0b7pre) Gecko/20100924 Firefox/4.0b7pre

Comment 46

[Aaron Train [:aaronmt]]

Verified fixed on Mozilla/5.0 (X11; Linux x86_64; rv:2.0b7pre) Gecko/20100924 Firefox/4.0b7pre