435365 - It can take a long time for Safe Browsing to detect a known malware/phishing URL

Status: RESOLVED DUPLICATE of bug 402469

(Toolkit :: Safe Browsing, defect)

Description

[Stephen Donner [:stephend] Not actively reading bugmail]

Build ID: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9) Gecko/2008051202 Firefox/3.0

Summary: It can take a long time for Safe Browsing to detect a known malware/phishing URL

Steps to Reproduce:

1. On 10.5, with rc1, create a new profile, and pick some URLs from the Google Safe Browsing blocklist
2. Load any of the given sites, finding one that loads

Expected Results:

When a given blocklisted-site loads, it should throw up the warning immediately

Actual Results:

[1] It took literally a few hours for every site I tried to be recognized as a phishing site, at first.
[2] Also, we checked via livehttpheaders, and it was sending POST requests with the hash.
[3] Adding 'regression' keyword because Firefox 2 with a new profile found this site and alerted us that it's a phishing site

Comment 1

[Jo Hermans]

The current database is 55MB large - it takes quite some time before it's all downloaded.

Comment 2

[Samuel Sidler (old account; do not CC)]

(In reply to comment #1)
> The current database is 55MB large - it takes quite some time before it's all
> downloaded.

That might be, but unlike Firefox 2, users aren't fully protected until they have the entire db. That's a huge regression, imo.

Comment 3

[Bartłomiej Brzozowiec (BartZilla)]

(In reply to comment #0)
> (...)
> [2] Also, we checked via livehttpheaders, and it was sending POST requests with
> the hash.
> (...)

Unfortunately, this is not a bug, but a "feature". FF3 uses new protocol, different than the one used in FF2. See specification of the new protocol here: http://code.google.com/p/google-safe-browsing/wiki/Protocolv2Spec (note that it is hosted on google.com, not mozilla.org; in practice this means that they can add some funny texts like "This specification is not yet for general use. Do not use this protocol without explicit written permission from Google." or "Note: This is not a license to use the defined protocol. This is merely a description of the protocol." Curious mind wants to know: if I implement it (as I've parially done with the old protocol, see http://bb.homelinux.org/firefox/sb/ ), is Google going to sue me?).

Comment 4

[Johnathan Nightingale [:johnath]]

(In reply to comment #2)
> (In reply to comment #1)
> > The current database is 55MB large - it takes quite some time before it's all
> > downloaded.
> 
> That might be, but unlike Firefox 2, users aren't fully protected until they
> have the entire db. That's a huge regression, imo.

Incidentally, the protocol delivers the most recent entries first, so that the updates that take longest to get are the ones that matter least.  Having said that though, there has been substantial discussion around the right speed at which to deliver updates.  The DB is also a lot bigger than FF2, a lot *more* protection is available, so calling it a straight regression is a little bit misleading, imo (Firefox 2 users were *never* protected from most of these sites, period).  I'm not trying to sound like an apologist, though, we do need to make sure we get that to users as quickly as possible.

That's what bug 402469 is about.