Closed Bug 296952 Opened 16 years ago Closed 7 years ago

Improve Phishing Detector to recognize host names of the same domain

Categories

(Thunderbird :: Security, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED
Thunderbird 29.0

People

(Reporter: mscott, Assigned: squib)

References

(Blocks 2 open bugs)

Details

Attachments

(1 file)

I just saw a false positive on the phising detector because Fidelity sent out
anchor text that looked like:

https://scs.fidelity.com/customeronly/emailoption.shtml

but the actual URL had a slightly different domain:

http://efs.fidelity.com/cgi-bin/ibgRedirect.pl?id=595417636&template=MQSTMT&ev=URLCLICK&url=https://scs.fidelity.com/customeronly/

note the scs.fidelity.com vs. the efs.fidelity.com

I wonder if there's a way we can get necko to tell us if the two urls refer to
the same domain (fidelity.com) even though they are slightly different.
Target Milestone: --- → Thunderbird1.1
you can't jsut check the last two parts of the domain, that will fail for .co.uk
(or geocities.com, but i don't think that is the target of the phishing detector)

If you found a good way, please tell me, because cookies need it too (bug 252342)
Because https://scs.fidelity.com/customeronly/ exists as substring of both the
URL and the anchor text (substantial common text), it should mark it as suspect
only (look different than a full positive).  It should not mark it as a full
positive.  Positives should only be marked if the URL and the anchor text are
completely different (fails substantial substring match test).  With a
substantial substring match it should be marked suspect (and not negative)
simply because you don't know what the page might do if this link were clicked
(in this case, it's a referral).  Sure, it appears legitimate by visual
inspection, but it could be a DNS hijack situation as well.  However, it's
better to know that it's suspect instead of assuming that it's negative and not
show anything.  But, it shouldn't show a positive.  Of course, a phisher could
use this to their benefit by attaching most (or all) of the anchor text as part
of the URL (at the end) to get their link to be marked suspect rather than positive.

A substantial substring match means most (85% or more) of the text in the anchor
matches any part of the URL.  To test, I would start by lopping off character by
character at the end of the anchor text and attempting to match the new
substring against the URL until you reach the treshhold of the match test (<85%)
or until you find a match.  In fact, the substring percentage threshold could be
set by the user in preferences for more control (but I wouldn't allow any less
than 50% be set by the user).

I don't know if this idea will work for the cookie problem.

If it all possible, the provider of the email should be contacted regarding
their link construction to not use links constructed like this.
mvl, what if we did something like breaking each host name into it's component
parts (using '.' as the delimeter).

Then start backwards comparing each part. If all of the parts match except for
the very first one, then consider them to be from the same site. This means:

scs.fideltiy.com
and
efs.fidelity.com 

.com would match, .fidelity would match and we don't compare the scs and efs.
Declare it a match.

scs.fidelity.co.uk
and
efs.fideltiy.co.uk
.uk would match, .co would match, .fidelity would match, don't compare the top
most domain, declare a match.

Of course with such a rule, you'd have to make sure you compare at least two
parts of the domain to avoid the following:

myevilsite.com
fidelity.com

.com would match, fidelity and myevilsite would not match. Declare phish.

Oh but that doesn't work at all for:
myevilsite.co.uk
fidelity.co.uk

Oops. oh well, back to the drawing board.

How about checking whether entries in the DNS NS (name server) or SOA
(authority) records are the same for both URLs' hostname component?
... and couple that with checking that the root name servers indicate that the
ip address of the "advertised" name server for www.xyzcorp.com is in fact what
the "NS" record returns.
Anyway, just my €0.02.
I couldn't come up with a static check to tell if two url hosts are contained
within the same domain so pushing off until we can think of a way to do it.
Target Milestone: Thunderbird1.1 → Thunderbird2.0
(In reply to comment #3)
> 
> Oh but that doesn't work at all for:
> myevilsite.co.uk
> fidelity.co.uk
> 
Well, according to the "official rules of old" those *should* be two _hosts_ in
the same (second-level) _domain_.
The original RFC said that names were of the form "host.domain", where anything
up to the first period was the hostname, and anything else was the domain name
(which in turn could obviously have other periods in it).

Since all UK domains really are third-level, there should _not_ be an answer to
a DNS A query asking for just "myevilsite.co.uk": the fact that in reality a DNS
A query of these two names does returns an IP address is a breakage of the
original semantics, where it was assumed that every record in the same domain
would be under the control of the same organization.

Anyway, since DNS really is an insecure protocol, subject to various kinds of
attacks (e.g. poisoning) I'd advise against being lax in hostname matching.

The guys at Fidelity really should get their act together and simply avoid
sending out messages with advertised links different from the real ones.

Just my € 0,02.
The fix for bug 252342 will help here too.
Target Milestone: Thunderbird2.0 → Thunderbird 3
QA Contact: front-end
This bug hasn't seen activity in nearly three years...  Since it doesn't have anything that can be implemented, should it be closed?

(For the original two links, I'm personally happy with Thunderbird's marking that as potential phishing, since Fidelity isn't being up front about the fact that they're tracking the click, even though it's not an outright scam.  But I may be in the minority on that.)  :-)
Whiteboard: closeme 2008-08-21
It's still worthwhile, and bug 385299 is relevant here.
Assignee: mscott → nobody
Whiteboard: closeme 2008-08-21
Blocks: mail-scam
Isn't this more specifically covered by bug 291865, though obviously more discussion being done here than there already?
Duplicate of this bug: 291865
Duplicate of this bug: 674203
OS: Windows XP → All
Hardware: x86 → All
Summary: Improve Phishing Detector → Improve Phishing Detector to recognize host names of the same domain
Target Milestone: Thunderbird 3 → ---
This is actually really easy (use nsIEffectiveTLDService), and will probably solve 90% of people's issues with the phishing detector, so let's fix it!
Assignee: nobody → squibblyflabbetydoo
Status: NEW → ASSIGNED
if I got this wrong, just move it where it should go
Component: Mail Window Front End → Networking
Product: Thunderbird → MailNews Core
The code's in mail/, so it's a Thunderbird bug (but Seamonkey devs are encouraged to port it if they like).
Component: Networking → Security
Product: MailNews Core → Thunderbird
Ok, this should help remove one more source of false positives from the phishing detector. Tests pass locally.
Attachment #8362787 - Flags: review?(mkmelin+mozilla)
Comment on attachment 8362787 [details] [diff] [review]
Use nsIEffectiveTLDService to compare base domains

Review of attachment 8362787 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good, thx! r=mkmelin

::: mail/base/content/phishingDetector.js
@@ +29,1 @@
>     * update the local tables if necessary

Capitalize while you're touching it.

@@ +222,5 @@
>      // so strip the spaces out (see bug 326082 for details).
>      aLinkNodeText = aLinkNodeText.replace(/ /g, "");
>  
> +    // Only worry about http: and https: urls.
> +    if (aLinkNodeText && aLinkNodeText.search(/(^https?:)/) != -1)

you could make this /(^https?:)/.test(aLinkNodeText) instead

@@ +227,2 @@
>      {
> +      let eTLD = Components.classes["@mozilla.org/network/effective-tld-service;1"]

we have Services.eTLD

::: mail/test/mozmill/message-header/test-phishing-bar.js
@@ +153,5 @@
> +}
> +
> +
> +
> +

lot of extra empty lines
Attachment #8362787 - Flags: review?(mkmelin+mozilla) → review+
Landed: https://hg.mozilla.org/comm-central/rev/9e9dbdb601b7

I forgot to push the review comments, so I will do that in a followup once I see green tests locally. Sorry about that!
Flags: in-testsuite+
Target Milestone: --- → Thunderbird 29.0
Landed fixes from review: https://hg.mozilla.org/comm-central/rev/dc59ad3ee2f7
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Blocks: TB2SM
You need to log in before you can comment on or make changes to this bug.