Closed Bug 427938 Opened 12 years ago Closed 12 years ago

Phishing protection does not work in Firefox 3 for users in some regions / google gethash service unavailable in some regions

Categories

(Toolkit :: Safe Browsing, defect, critical)

defect
Not set
critical

Tracking

()

VERIFIED FIXED

People

(Reporter: andrewm715+bugzilla, Assigned: dcamp)

References

Details

(Keywords: regression)

Attachments

(2 files)

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9pre) Gecko/2008040404 Firefox/3.0pre
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9pre) Gecko/2008040404 Firefox/3.0pre

I realize this is very similar to bugs that have already been filed claiming that phishing protection in Firefox 3 is not working and which have been marked FIXED or WORKSFORME; please try to reproduce using the sites below as it seems something is still going wrong.

Reproducible: Always

Steps to Reproduce:
1. Open up a clean profile in both Firefox 2 and Firefox 3.
2. Wait 5-10 minutes to make sure the first set of data has downloaded for each (usually it's much quicker of course).
3. Visit these phishing sites in both browsers:

http://131.115.broadband.iol.cz/online.bancadiroma.it/index.php
http://76.162.161.220/private/avirtual1-bancatlan.com/PYMES/hlogin.htm
http://82.115.26.230/www.irs.gov/irfofgetstatus.htm
http://www.ailia.ca/Lloyds/securityupdate3/www.lloydstsb.com/online.lloydstsb.co.uk/customer.ibc/customer.ibc.htm
http://www.usuarios.lycos.es/vlf/bay114021001010=1577522364mail=bay041024/
http://125.46.60.213/login.htm
http://haelimysk.co.kr/img/wowwowwow/www.RBC.com/www.RBC.com/cgi-bin/rbaccess/rbunxcgi/RBC.htm
http://jangho.hs.kr/bbs_2006/skin/gangjwa/Lloyds/Lloyds/customer.htm
http://pponline.ptpp.co.id/upload/tmp/index.html

Note that all 9 URLs can be found in Google's blacklist (the one used for Firefox 2): http://sb.google.com/safebrowsing/update?version=goog-black-url:1:-1
Actual Results:  
In Firefox 2, the phishing warning bubble is shown for every site.

In Firefox 3, the user is not warned or blocked from visiting any of these active phishing sites.

Expected Results:  
Firefox 3 should not regress phishing protection for users; phishing sites that the user is warned about in Firefox 2 should be warned about in Firefox 3.

I do not believe this is related to the urlclassifier3.sqlite file downloading slowly (e.g. bug 402469) because I have had this instance of Firefox 3 (my main profile) running for several days now. The size of the urlclassifier3.sqlite file is currently 13.9 MB and the date modified timestamp is about 16 minutes ago.
Blocks: 387196, 399233
Flags: blocking-firefox3?
Keywords: regression
Version: unspecified → Trunk
Can you run with NSPR_LOG_MODULES=UrlClassifierDbService:5,UrlClassifierHashCompleter:5 and NSPR_LOG_FILE=urlclassifier.log and attach urlclassifier.log please?
I can confirm that I am not warned or blocked in Linux either.

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9pre) Gecko/2008040804 Minefield/3.0pre
...I tried setting those NSPR variables in Linux but I can find no such log file. I'm probably doing it wrong though.
(In reply to comment #1)
> Can you run with
> NSPR_LOG_MODULES=UrlClassifierDbService:5,UrlClassifierHashCompleter:5 and
> NSPR_LOG_FILE=urlclassifier.log and attach urlclassifier.log please?
> 

Hey dcamp, i'm able to create a urlclassifier.log but even when i load all of this URL's from comment #0 its empty ? 

I can also confirm this bug report, i don't get any warning at this sites
Status: UNCONFIRMED → NEW
Ever confirmed: true
also confirmed on Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9pre) Gecko/2008040814 Firefox/3.0pre ID:2008040814 (Debug Build) , no warning on this sites.
OS: Windows Vista → All
Hardware: PC → All
Attached file urlclassifier log mac
url classifier log from Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9pre) Gecko/2008040814 Firefox/3.0pre ID:2008040814, its a debug build, so maybe a little more output then usual
That log indicates that it's finding matches and just failing to actually block after finding it.

I'm not able to reproduce on any machines I have access to (everything seems to be blocked correctly), so tomorrow I'll put together a build for you to test with some more debugging info.

      
(In reply to comment #1)
> Can you run with
> NSPR_LOG_MODULES=UrlClassifierDbService:5,UrlClassifierHashCompleter:5 and
> NSPR_LOG_FILE=urlclassifier.log and attach urlclassifier.log please?

I don't know how to set those variables; I just test nightlies :) I'll be happy to try if someone tells me how.
Also, note that the third (http://82.115.26.230/www.irs.gov/irfofgetstatus.htm
) and fifth phishing sites (http://www.usuarios.lycos.es/vlf/bay114021001010=1577522364mail=bay041024/) have been taken down; all of the other ones seem to be still active.
I've got the same issue.

Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9pre) Gecko/2008040907 Minefield/3.0pre
Attached file new log
new log as discussed with campd from Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9pre) Gecko Firefox/3.0pre ID:2008040920

also i can only reproduce this problem now on mac, on windows xp i get now the default and expected phishing warning page
I can reproduce this on Linux.
I talked to the google guys, and it looks like some of the gethash servers (which we use to request additional information to confirm a phishing hit) didn't have  the proper data.  So the response to the request is treated the same as "this entry was removed from the phishing database, don't block".

Google is working on a fix now, after which we will need to verify that this fixes what people are seeing here.
Flags: blocking-firefox3? → blocking-firefox3+
Summary: Phishing protection does not seem to be working properly in Firefox 3; this is a serious regression in effectiveness from Firefox 2 → Phishing protection does not work in Firefox 3 for users in some regions / google gethash service unavailable in some regions
Out of the 9 active phishing site URLs I posted earlier, 5 are still active:

http://131.115.broadband.iol.cz/online.bancadiroma.it/index.php
http://76.162.161.220/private/avirtual1-bancatlan.com/PYMES/hlogin.htm
http://haelimysk.co.kr/img/wowwowwow/www.RBC.com/www.RBC.com/cgi-bin/rbaccess/rbunxcgi/RBC.htm
http://pponline.ptpp.co.id/upload/tmp/index.html
http://jangho.hs.kr/bbs_2006/skin/gangjwa/Lloyds/Lloyds/customer.htm

On my main profile (size of urlclassifer3.sqlite: 14.2MB; date modified timestamp: 7:48PM (just over an hour ago)) I am now blocked from visiting 4 out of the 5 sites.

On my newer profile (size of urlclassifer3.sqlite: 8.91MB; date modified timestamp: 1:04AM this morning), however, I am only blocked from visiting one of the sites (http://pponline.ptpp.co.id/upload/tmp/index.html). I can still access these with no warning:

http://131.115.broadband.iol.cz/online.bancadiroma.it/index.php
http://76.162.161.220/private/avirtual1-bancatlan.com/PYMES/hlogin.htm
http://haelimysk.co.kr/img/wowwowwow/www.RBC.com/www.RBC.com/cgi-bin/rbaccess/rbunxcgi/RBC.htm

And I can access http://jangho.hs.kr/bbs_2006/skin/gangjwa/Lloyds/Lloyds/customer.htm with no warning on both profiles.
Yeah, there does seem to be something going wrong with the jangho.hs.kr blacklisting - I'm working with google to figure out what's going on there.
Assignee: nobody → dcamp
OK, after close examination, I'm going to close this bug and reopen 402469.

The failed gethash requests have been fixed on google's side, which is (now) the topic of this bug.

The URLs here are failing in chronological order:  The one that is blocking on all of your profiles is the newest, the ones blocking on only one profile are older, and the one that isn't blocking anywhere is (by far) the oldest.

So I reopened 402469 to track the fact that google is feeding us old data too slowly.

Thanks a lot for the bug report.  Keep an eye on 402469, and please file any new bugs that don't seem related to that one.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
verified fixed using  Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9pre) Gecko/2008040907 Minefield/3.0pre.

The getash Problem is now fixed and Bug 402469 tracks the other Problem (see comment#16)

-> Verified fixed
Status: RESOLVED → VERIFIED
Thanks for your and Google's efforts in fixing this. I'd just like to clarify a few things:

When bug 402469 is fixed, should every URL in Google's Firefox 2 phishing blacklist (http://sb.google.com/safebrowsing/update?version=goog-black-url:1:-1) be blocked by Firefox 3? I.e., if there is an active phishing URL in that blacklist for which no warning message is shown in Firefox 3, is something going wrong? Or if it is an older URL, could it simply have been expired already?

Also when 402469 is fixed, will the performance of phishing protection in Firefox 3 be equal to that of Firefox 2, or will it always be slower and less-complete? That was partly why I filed this bug, because it seemed that phishing URLs were blocked almost instantly in Firefox 2 but Firefox 3 had to be open for a long, long time before equivalent protection is in place.

And lastly, for the next version of Firefox, is it technically feasible to measure how current and complete the local phishing and malware databases are versus Google's master list, so that some sort of progress meter or message in the UI could let the user know if the data isn't all there yet (what Jesse suggested in bug 423622 comment 4)? This might help prevent confusion and false alarms (like in bug 423622) in the future if a user perceives that phishing or malware protection aren't working.
Product: Firefox → Toolkit
You need to log in before you can comment on or make changes to this bug.