Closed Bug 840882 (CVE-2013-1699) Opened 11 years ago Closed 11 years ago

Homograph attack prevention is incomplete

Categories

(Core Graveyard :: Networking: Domain Lists, defect)

Product:
Core Graveyard
Component:
Networking: Domain Lists
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox20+ wontfix, firefox21+ wontfix, firefox22 fixed, firefox-esr17 unaffected, b2g18 unaffected)

Status:
RESOLVED FIXED
Milestone:
mozilla22
Tracking Flags:
Tracking Status
firefox20 + wontfix
firefox21 + wontfix
firefox22 --- fixed
firefox-esr17 --- unaffected
b2g18 --- unaffected

People

(Reporter: rnewman, Assigned: gerv)

References

Details

(Keywords: csectype-spoof, regression, sec-moderate, Whiteboard: [adv-main22+]) 

http://www.shmoo.com/idn/

Observe that our presentation of both 'evil' and 'good' URIs is identical.

Then compare to Safari (which shows "www.xn--theshmogroup-bgk.com" instead of "www.theshmoogroup.com").

This is probably filed under the wrong component, but worth filing than not at all!
Looks like we might need to add о to network.IDN.blacklist_chars

gerv/dveditz: thoughts?
or, really, maybe it was still premature to set network.IDN.whitelist.com to true :/
(In reply to Reed Loden [:reed] from comment #2)
> or, really, maybe it was still premature to set network.IDN.whitelist.com to
> true :/

(bug 770877)
(In reply to Reed Loden [:reed] from comment #1)
> Looks like we might need to add о to network.IDN.blacklist_chars
> 
> gerv/dveditz: thoughts?

I agree, but...

(In reply to Reed Loden [:reed] from comment #2)
> or, really, maybe it was still premature to set network.IDN.whitelist.com to
> true :/

I also agree with this. How are we going to find all the other ones that have been overlooked?
Component: Security → Networking: Domain Lists
Keywords: regression
Product: Firefox → Core
Summary: Homograph protection → Homograph attack prevention is incomplete
By the way: full credit to 3ric johanson (ericj@shmoo.com) for pointing this out (as you might have gathered from the URL!), and Holt Sorenson for passing it along to me. Both CCed.
If ya'll are trying to block chars based on a blacklist, I wish you the best of luck with that *very complex* task.   Is there a table of homograph-attack based chars in unicode somewhere?
There are some details from the original bug here:  https://bugzilla.mozilla.org/show_bug.cgi?id=279099
The situation is as follows:

Since the registration of www.xn--theshmogroup-bgk.com, Verisign has adopted more strict spoofing-related registration policies. You can find them here:
http://www.verisigninc.com/en_US/products-and-services/domain-name-services/value-added-products/idn-domain-names/idn-code-points/registration-rules/index.xhtml

In particular, www.xn--theshmogroup-bgk.com would no longer be permitted to be registered because it contains characters from both the Latin and Cyrillic scripts, which are not both together in any of the character tables linked from that page. (And if there is no applicable table, then no script mixing is permitted at all.)

We decided that we were not going to ask Verisign to go and revoke the registrations of legacy domains which combined characters in this way.

Due to the upcoming massive expansion of the root namespace, a whitelist-based approach is no longer scalable. So, we are moving to a new mechanism for determining whether or not to display a given IDN. Details of how it works, and the rationale behind it, can be found here:
https://wiki.mozilla.org/IDN_Display_Algorithm

That's bug 722299, and the next step is a security review, which will happen soon.

If that algorithm was implemented and was the only algorithm in place, the www.xn--theshmogroup-bgk.com URL would again display as Punycode. (Currently, the plan is to leave the whitelist in place for backward-compatibility reasons, so www.xn--theshmogroup-bgk.com would still display as now. However, as noted above, further registrations of this type of domain name are not possible.)

We are certainly not going to add cyrillic-o to the IDN character blacklist; that suggestion misunderstands the purpose of the blacklist. That blacklist is designed to prevent the use of characters which spoof separators and other protocol characters (".", "/" etc.) and prevents their use anywhere in the domain name, not just in labels which are registered directly with registries. (Once we start restricting permitted characters to those allowed by IDNA2008, this list will get a lot shorter. That's bug 479520.)

Gerv
Is that OK for everyone?

Gerv
I am still looking into this.  The fact is that all of the other browsers correctly detect something is wrong with the test URLs. It sounds like Mozilla got strong armed by verisign at the cost of customer security.
Er, what do you think Verisign "strong-armed" us with? What hold do they have over us, exactly?

If you object to our course of action, you will need to explain what you would do. Maintain the whitelist in the face of thousands of new TLDs? Require the revocation of an unknown number of domains registered (in most cases!) in good faith, and currently in use by their owners?

If there's still a security hole here, then go and pull the same trick by registering a new spoofing domain. If you can do that, then I'll agree there's a problem (but I'll blame Verisign for it, not us - as we should have done first time round).

Gerv
I don't have any insight into what relationship you have with Verisign. You said "Verisign have emailed me to ask that .com, .net and .name be added to the whitelist." in bug #770877. 

Here's my understanding of the history of IDN: 

 - ~2000: RFCs get drafted with zero foresight into security. 
 - ~2002 to present: verisign starts pushing for adoption
 - 2002: Security implications gets published about homograph attacks.  http://www.cs.technion.ac.il/~gabr/papers/homograph.html
 - 2002 to 2005: Everyone ignores these warnings, and IDN gets adopted.
 - 2005: I register a bogus paypal.com url which is silently accepted by most major browsers.  RFC authors have no comment. thx, hoffman.
 - 2005: Response: Nobody accepts responsibility, browsers are forced to disable or filter IDNs.  Lots of finger pointing ensues. 
 - 2005 to present: new standards slowly get drafted to better filter IDNs at the time of registration. 
 - Present: Thousands (hundreds?) of bogus registrations have been done in the past 10 years. 
 - Present: New standards (IDNA2008, " ICANN’s Guidelines for the Implementation of Internationalized Domain Names Section 5", and others) are mostly adopted by vendors. 
 - Present: Verisign decides to not apply the new rules to existing registrations, and not even review them. 

So at this point you have a 800 gorilla telling you to adopt something which will have a negative effect on a majority of your customers.   

What would I do?  I would not adopt standards like this, or adopt them with tooling in place which can help protect my customers.  As 99% of mozilla customers will only encounter Cyrillic chars/domains as part of a attack, it seems shortsighted to expose them to such risk without a warning dialog box of some kind.  

If my browser loads and renders a url which is in violation of the standards, (even it it shouldn't exist based on the registration rules) I would expect it to WARN ME, not just SILENTLY OWN ME. 

I will do some testing of how (in)effective their registration filtering is in the next week or so.  I suspect you can still register domain names via 3rd parties which don't meet their standards.   But I still expect my browser to protect me, regardless of the results of those tests. 

Mozilla is the only browser that fails this **** bogus IDN. This is a real shame, as it's my daily ride...
(In reply to 3ric Johanson from comment #12)
> I don't have any insight into what relationship you have with Verisign. You
> said "Verisign have emailed me to ask that .com, .net and .name be added to
> the whitelist." in bug #770877. 

That's fair comment. My discussions with Verisign were mostly by email.

> Here's my understanding of the history of IDN: 
> 
>  - ~2000: RFCs get drafted with zero foresight into security. 
>  - ~2002 to present: verisign starts pushing for adoption
>  - 2002: Security implications gets published about homograph attacks. 
> http://www.cs.technion.ac.il/~gabr/papers/homograph.html
>  - 2002 to 2005: Everyone ignores these warnings, and IDN gets adopted.
>  - 2005: I register a bogus paypal.com url which is silently accepted by
> most major browsers.  RFC authors have no comment. thx, hoffman.
>  - 2005: Response: Nobody accepts responsibility, browsers are forced to
> disable or filter IDNs.  Lots of finger pointing ensues. 
>  - 2005 to present: new standards slowly get drafted to better filter IDNs
> at the time of registration. 
>  - Present: Thousands (hundreds?) of bogus registrations have been done in
> the past 10 years. 

Do you have evidence for that assertion?

>  - Present: New standards (IDNA2008, " ICANN’s Guidelines for the
> Implementation of Internationalized Domain Names Section 5", and others) are
> mostly adopted by vendors.

IDNA2008 doesn't in itself directly tackle the homograph problem (and is not designed to). See http://www.unicode.org/faq/idn.html#26 It does reduce the list of acceptable characters by removing e.g. much punctuation, which helps, but is by no means sufficient.

>  - Present: Verisign decides to not apply the new rules to existing
> registrations, and not even review them. 
> 
> So at this point you have a 800 gorilla telling you to adopt something which
> will have a negative effect on a majority of your customers.   

Any suggestion that Verisign leaned on Mozilla in some way is incorrect. We applied the same criteria to them which we applied to other applicants - stricter, in fact, because in view of the historical problem (although what you did would have worked just as fine in .org or .somethingelse) we had a public discussion to see if anyone had particular concerns.

> What would I do?  I would not adopt standards like this, or adopt them with
> tooling in place which can help protect my customers.  As 99% of mozilla
> customers will only encounter Cyrillic chars/domains as part of a attack, it
> seems shortsighted to expose them to such risk without a warning dialog box
> of some kind.  

There are many reasons why that's a bad idea; if you were paying any attention to the years of discussion which followed what happened in 2005, you would know some of them. A quick run-down of ones which come immediately to mind:

- The goal here is to make all scripts and languages on the web (and thereby the people who use them) first-class citizens; having warning dialogs for some scripts does not meet that goal

- And no, you can't base it on the browser UI language to decide whether to alert; many, many people use the en-US build even if they don't speak English or don't speak it well

- Warning dialogs are terrible UI and get dismissed and ignored

> If my browser loads and renders a url which is in violation of the
> standards, (even it it shouldn't exist based on the registration rules) I
> would expect it to WARN ME, not just SILENTLY OWN ME. 

Fundamentally, it's not possible for a browser to determine with certainty if a given domain name is confusable with another one. You can try certain proxies for determining if it is, but all of them are fallible in nasty ways. There needs to be human decision involved - which is why registries need to have sensible anti-spoofing homograph policies. Mozilla has been requiring this for IDN enablement since 2005, and in doing so has probably done more to prevent users seeing IDN spoofing problems than anyone else. However, the period of time for which that is feasible is coming to an end, and so the registries (all 2000+ of them, as there soon will be) need to take responsibility for not letting customer A shaft customer B.

> I will do some testing of how (in)effective their registration filtering is
> in the next week or so.  I suspect you can still register domain names via
> 3rd parties which don't meet their standards.

That should not be so; if you discover that it is, then I will take it up with Verisign immediately.

> Mozilla is the only browser that fails this **** bogus IDN. This is a real
> shame, as it's my daily ride...

Once the new script mixing code is in place, it'll fail again.

Gerv
Where is this public discussion?
  Mozilla, incoming PR shitstorm.
Thanks for the tip.

Gerv
Please supply functional instructions for now to disable idn.  The public must be warned about this by-desgin vulnerability, and I would like to give them an option besides not use Firefox.  You have 2 weeks before full disclosure. If you need more time let me know and I will consider it.
s/now/how
historical aside, 3ric, have you had a chance to look at the current proposal?  

Also, we will be having a security review of this on Feb 21st.  Could you possibly join us for that?
I would like that. Falling offline for a few days on boat.
i have added you to the sec review bug.  the details are all in there.
(In reply to 3ric Johanson from comment #18)
> Please supply functional instructions for now to disable idn.

1. Open about:config
2. type "idn." into the filter field
3. double-click on "network.IDN.whitelist.com" to toggle it false

To disable IDN more globally you could instead toggle the pref "network.IDN_show_punycode" from false to true.

There is another pref that may _look_ like what you want, but if you set "network.enableIDN" to false then non-ascii URLs will simply fail to load, disenfranchising many legitimate sites around the world. If you're going to suggest Americans don't need to see IDN at least tell them the show_punycode pref rather than how to erect their own Great Firewall of Xenophobia.
(In reply to Daniel Veditz [:dveditz] from comment #23)
> There is another pref that may _look_ like what you want, but if you set
> "network.enableIDN" to false then non-ascii URLs will simply fail to load

We should probably remove that pref. It's confusingly named, and when you switch it, it breaks the web. Bug 842282 filed.

Gerv
(In reply to Gervase Markham [:gerv] from comment #13)
> (In reply to 3ric Johanson from comment #12)
> > Mozilla is the only browser that fails this **** bogus IDN. This is a real
> > shame, as it's my daily ride...
> 
> Once the new script mixing code is in place, it'll fail again.

Not so, as you yourself pointed out above in comment 8. Or do we plan to revert the last few additions to the whitelist when checking in bug 722299?
(In reply to Simon Montagu from comment #25)
> Not so, as you yourself pointed out above in comment 8. Or do we plan to
> revert the last few additions to the whitelist when checking in bug 722299?

That's a course of action I'm seriously considering (or perhaps some other criteria for trimming the whitelist once the code goes in); I want to hear (shareable) answers to my questions sent to my Verisign contact first.

Gerv
Depends on: 843739
Filed bug 843739 on backing out 722299.
A couple of days ago, my contact at Verisign told me the following, by phone:

* As of November 2012, there are approximately 2100 domain names in .com and .net which are IDNA2008-invalid. (He doesn't expect there to be many at all in .name.) This is not the same as the number which might be invalid under the current registration rules, because the current rules specify more than just IDNA2008-compatibility. In particular, there are table-based character restrictions.

* In late September 2012, as part of the process of applying for inclusion in Mozilla, Verisign contacted all its registrars who had registered a domain name which is IDNA2008-invalid. It asked (not required) them to contact the registrant to ask (not require) them to give up the domain and replace it with a valid one. This process did not include domains which are IDNA2008-valid but which do not match a character table. One example of such a domain is the TSG test domain.

It seems to me that this procedure is insufficiently rigorous for us to be sure that any existing dangerous domains have been dealt with. Many dangerous domains will not have been caught by this process and, even if they were, the owner of one could simply refuse to give it up. Therefore, I think that we should take the following actions:

0) We should get the new script-mixing-checking code (bug 722299) reviewed and checked in as quickly as possible. If we don't have this, removing a TLD from the whitelist will cause display breakage for a lot of valid domains.

1) We should investigate the possibility of forward-porting it to Aurora, at least. Given that it is additive to the existing whitelist, there is no possibility of breaking the display of domains, except in the case where we remove a TLD from the whitelist.

2) Once the new code is checked in, we should immediately remove from the whitelist any TLDs who have been added since the inclusion criteria changed (1 Feb 2012), and who have not been operating anti-spoofing measures for the entirety of the time they have been issuing IDNs, and who have not pursued a policy of compulsory de-registration of names which are now disallowed.

This list happens to be:
.com
.net
.name

This is bug 843739. This will deal with the legacy problem, and mean that TSG's test domain and any other risky legacy domains will no longer work. We reserve the right to remove other TLDs from the whitelist where we find their procedures are insufficiently rigorous.

Dan would also like to remove the whitelist entirely. I am not opposed to that, but I think we need to do more research on the impact of doing so.

(In reply to Daniel Veditz [:dveditz] from comment #27)
> Filed bug 843739 on backing out 722299.

You mean bug 770877.

Gerv
We should have an assignee here, is Gerv a good person for that?
Assignee: nobody → gerv
Yes, that's fine.

Gerv
Fixed by bug 843739.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla22
Bug 722299 won't go any further than Aurora 21 (see https://bugzilla.mozilla.org/show_bug.cgi?id=843739#c7), and appears to be a requirement for bug 843739's resolution (see https://bugzilla.mozilla.org/show_bug.cgi?id=843739#c1).

Given that, wontfixing for FF20. Please set the status flag back to affected if there's disagreement around the necessity for bug 722299 here.
status-firefox20: affected → wontfix
Given https://bugzilla.mozilla.org/show_bug.cgi?id=722299#c62 is going ride the trains and not get uplifted, Bug 843739  and this bug will remain wontfixed for the Fx21 cycle.

Please let me know if there is any concerns around this and set back the flags as needed.
Whiteboard: [adv-main22+]
Alias: CVE-2013-1699
Group: core-security
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.