444942 - SafeBrowsing prevents spin-down

Status: RESOLVED INVALID

(Toolkit :: Safe Browsing, defect)

Description

[Stefan Monnier]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9) Gecko/2008062910 FireFox/2.0.0.12 (Debian-3.0~rc2-2)
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9) Gecko/2008062910 FireFox/2.0.0.12 (Debian-3.0~rc2-2)

Firefox seems to keep updating its safebrowsing database even when there is no user activity, so the harddisk has to spin-up every 30minutes (on the user's machine as well as any proxy along the way).


Reproducible: Always

Steps to Reproduce:
1.
2.
3.

Comment 1

[Bartłomiej Brzozowiec (BartZilla)]

(In reply to comment #0)
> Firefox seems to keep updating its safebrowsing database even when there is no
> user activity, so the harddisk has to spin-up every 30minutes (on the user's
> machine as well as any proxy along the way).
> 

I guess this is intended and will not be fixed. (See also my little project related with so-called "safebrowsing" in "remote checking" mode in FF2: http://bb.homelinux.org/firefox/sb/ ).
In FF3 it is potentially even worse, because Google decides about the period between connections each time, i.e. the number of seconds before the client should contact the server again (see the protocol implemented(*) in FF3: http://code.google.com/p/google-safe-browsing/wiki/Protocolv2Spec, section 3.5.2).
BTW - I think Debian shouldn't distribute Firefox, because it implements protocol that isn't free (as in "freedom") -- namely, Google restricts usage of the protocol: "This specification is not yet for general use. Do not use this protocol without explicit written permission from Google." "Copyright 2007 Google Inc. All Rights Reserved." "Note: This is not a license to use the defined protocol. This is merely a description of the protocol."
The question is - am I allowed to implement this protocol on server side? I guess not (but I'll do this anyway ;)).

(*) I've checked the code, but also Mozilla says so: http://www.mozilla.com/en-US/firefox/phishing-protection/ - section "How does Phishing and Malware Protection work in Firefox?" points to this spec.

Comment 2

[Stefan Monnier]

This bug report is not about the SafeBrowsing protocol per se, just about the fact that it is active even when there is no other activity.  Better would be (when Firefox is otherwise idle) to delay the next sb download to when FF is not idle any more.

I.e. as long as there's no net activity, add the SB download to a list of pending downloads, and when some real page http://some/new/page needs to be downloaded, then first process the pending downloads before fetching http://some/new/page.

This should be just as safe, with the only drawback that after being idle, the first page';s download will take slightly longer, but it should be lost in the noise (this delay can happen anyway with the current code: the difference would be that it would happen more deterministically, i.e. more often).

Comment 3

[Bartłomiej Brzozowiec (BartZilla)]

(In reply to comment #2)
> This bug report is not about the SafeBrowsing protocol per se, just about the
> fact that it is active even when there is no other activity.  Better would be
> (when Firefox is otherwise idle) to delay the next sb download to when FF is
> not idle any more.
> 
> I.e. as long as there's no net activity, add the SB download to a list of
> pending downloads, and when some real page http://some/new/page needs to be
> downloaded, then first process the pending downloads before fetching
> http://some/new/page.
> (...)

Well, the problem is that "pending downloads" could be quite huge (especially in case of FF3, where they added "malware protection", also based solely on blacklists downloaded from Google; urlclassifier3.sqlite could be very large...).

Anyway, I guess you still don't get it -- this whole "safebrowsing" thing is not (only) about "protecting users", but about collecting by Google a huge amount of data about users. (Moreover, Google also unilaterally decides what pages are "good" and what pages are "bad" and can effectively block access to them.)

Comment 4

[Stefan Monnier]

I can't see why the "pending downloads" would be any larger than the single URL that's being postponed.

I perfectly get what safebrowsing does, thank you very much.  Whether Google is evil or not, and how Google wants safebrowsing to work is irrelevant to this bug-report: we're discussing Firefox and how the Firefox code works.  AFAIK this code is not under the control of Google.

Comment 5

[Bartłomiej Brzozowiec (BartZilla)]

(In reply to comment #4)
> (...) AFAIK
> this code is not under the control of Google.
> 

Well, actually it is under exclusive control of Google. See eg. history of bug 368255. Note also that Google Safe Browsing originally was an anti-phishing extension released by Google on labs.google.com in December 2005(http://wiki.mozilla.org/index.php?title=Phishing_Protection&oldid=49046#Overview).
See also headers of relevant files in source tree (eg. http://bb.homelinux.org/firefox/sources/2.0.0.15/mozilla/browser/components/safebrowsing/content/application.js.html#14
http://bb.homelinux.org/firefox/sources/2.0.0.15/mozilla/toolkit/components/url-classifier/src/nsUrlClassifierDBService.cpp.html#17 ).

Anyway, despite the fact that code is open we have no control over data that is fed to Firefox by Google. So, actually, Google controls everything regarding so-called "safebrowsing".

Comment 6

[Johnathan Nightingale [:johnath]]

BartZilla, as you well know, bugzilla is not for advocacy ( https://bugzilla.mozilla.org/page.cgi?id=etiquette.html ). Your comments here are not constructive in solving a well-scoped problem relating to the interaction of power management and the safebrowsing database.  The Mozilla project, not Google, controls the safebrowsing code; we control all the code in Firefox (or else we wouldn't be able to license it) and if we honestly thought that any part of it was a detriment to our users or to the open web, we would disable or remove it.  Absolutely we partner with Google to do it, because they offer our users a valuable service free of charge, but for the purposes of this bug, that is utterly and profoundly beside the point.

"Constructive" here is a patch, or analysis leading to a patch, or a reason why this shouldn't be fixed, but it is categorically not constructive for you to try to turn it into a passive aggressive discussion of partnering policy.

Don't confuse this for engaging - I'm not interested in continuing this discussion in this bug. You have been a helpful contributor in other parts of our code base and I really am glad to see your passion for issues of privacy; I understand them.  I think you recognize too, though, that your comments in this bug have not served to help Stefan.

Stefan - the behaviour you describe is intentional, since for most users, updating in the background is, if anything, a positive thing, since it is less likely to impact their browsing.  That is not to say, though, that nothing can be done to help your situation.  I wonder if you would be willing to confirm that the problem still exists on Firefox 3?  Because the architecture of the service changed significantly between versions 2 and 3, any fix we made for 3 would be unlikely to make it back into version 2, which is nearing the end of its supported life.  I suspect you will see the behaviour in version 3 as well, since we do still pull periodic updates about every 30 minutes, but it would at least help us with any further diagnostic information we might need.

Comment 7

[Stefan Monnier]

Thank you Jonathan, I feel less lonly now.

Yes, I still see the same bug under Firefox 3 (the URL used is different, but it seems to also be refreshed every 30min, even without any other network activity).  I understand that fetching in the background might often be a good idea, but I'm pretty sure there's a way to balance this, e.g. only do it if there's been activity in the last 5 minutes.

Note also that AFAICT, each download related to safebrowsing results in pretty small documents, that are downloaded very quickly, so doing it synchronously when the user resumes his browsing shouldn't impact the user's experience much (most likely Firefox will already respond a bit more slowly when the user resume2 browsing because some things may have been partly swapped out in the mean time, so the additional slowdown is unlikely to be very noticeable).  As a matter of fact, those downloads pretty much have to be small, otherwise they'd be too costly even with the current background downloads.

Comment 8

[Jeremy Baron]

-> 3.0 Branch per comment 7.

(In reply to comment #0)
> User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9)
> Gecko/2008062910 FireFox/2.0.0.12 (Debian-3.0~rc2-2)
> Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9)
> Gecko/2008062910 FireFox/2.0.0.12 (Debian-3.0~rc2-2)

On closer inspection it seems your User Agent string not only has the miscapitalized "FireFox" (Firefox is the correct variant) that initially caught my eye but also contradicts itself: it states that the Gecko version is 1.9 and the "FireFox" version is 2.0.0.12 .  Firefox 2.0.0.x has and will always use Gecko 1.8.1.x *not* Gecko 1.9 (or any point release thereof).  Also, as I understand it Debian does not ship with any official Mozilla branding for any MoCo products, Firefox included.  If it were a Mozilla build it wouldn't have the Debian appendage and a Debian build shouldn't be using Mozilla trademarks (unless my previous sentence is off).

Anyway, I was wondering how you ended up with that (did you take a stock Debian Iceweasel 3 install and use an extension or about:config to tweak the user agent?) and whether there are actually builds in the wild that send that string straight out of the box.

Comment 9

[Stefan Monnier]

Regarding the User Agent, yes I seem to remember that I manually changed it at some point because of some brain dead site which didn't like Iceweasel (and told me something like "this site only works with IE and Firefox" as if they knew better).

Comment 10

[Bartłomiej Brzozowiec (BartZilla)]

Bug 488868 is a duplicate of this bug and should be marked as such (since this bug has been reported earlier).
Also, similar bug is bug 428565.

Comment 12

[Justin Dolske [:Dolske]]

When this bug was filed we used a sqlite-based DB, which can be prone to this kind of issue. Totally different code now, so almost certainly no longer an issue.