429263 - Add an about: page to see current state for safebrowsing

Status: RESOLVED WONTFIX

(Toolkit :: Safe Browsing, defect)

Description

[Marria Nazif]

In bug 427938, comment #18 makes an interesting point that we have very little visibility into client state in the new protocol.  In Firefox 2, at least we had the current version number in a pref, but now it's harder to see how up to date the client is.

I think it would be great if we add a simple about: page to dump client chunk state (maybe about:safebrowing ?)  We can also add a page on the server that dumps current server state.  That will make it more obvious whether or not a client is up to date.  It seems essential to have something like this to gather useful bug reports in the future.

Comment 1

[Johnathan Nightingale [:johnath]]

Implementing this seems (in principle!) straightforward, but it is coming very late in the cycle for Firefox 3.  In particular, we are past string freeze, meaning that the text content of this page cannot be translated at this point.

One option is to package this as an add-on for Firefox 3, with an eye to quick uplift in Firefox.Next.  That way people with problems can be directed to the add-on as part of their debugging process.

Another option is to try to land it either in v3 or in a point release, if it can be done in a way that doesn't include strings, but that doesn't seem very user-friendly, nor does shipping an English-only version, given that the majority of our users are on non-en-US builds.

I guess a big part of the question is what specific information we would want to show here, and then we can figure out how best to show it.

Comment 2

[Marria Nazif]

I think it should be possible to do this in a way that does not require string translations.  In particular, I had something very simple in mind, that would just basically dump the current chunk number ranges for each list.

I don't think it's really necessary to make it too user friendly, as it isn't a page that is likely to be visited often.  For a similar example, see about:metrics  That page just dumps the xml that gets sent to the server.

Comment 3

[Dave Camp (:dcamp)]

Attachment: simple extension

Attached .xpi adds an about:safebrowsing page that gives some data about the state of the database.

Comment 4

[Mike Beltzner [:beltzner, not reading bugmail]]

What do the numbers mean?

Comment 5

[Dave Camp (:dcamp)]

That's a good question!  The quick answer is "just look at the percentage of add chunks, and it shows you roughly how much of the list you have", the rest is mostly there to help debugging specific problems.

Basically, the malware/phishing data is split up into a series of persistent chunks on the server.  These chunks are sent to the client to be applied to the database.  Each add chunk corresponds to some amount (it varies per chunk) of blacklisted urls.  Each sub chunk corresponds to some amount of no-longer-relevant blacklisted urls.

So the number of sub chunks doesn't really matter, it's just there to help debug.  The number of add chunks DOES matter, but it's not strictly linear - chunks are of different sizes, so having 50% of the chunks doesn't necessarily imply that you have exactly 50% of the data (though it should be close).

Also, older data (served last) tends to have more irrelevant data, so actually if you have 50% of the add chunks, you probably have >50% of the relevant data.  But we have no way to determine the exact usage from the client, so the percentage of add chunks is a pretty good approximation of how much of the list you have.

Comment 6

[AndrewM]

Nominating for blocking-firefox3.1; this seems like something that could be very useful for debugging phishing/malware protection issues and from the standpoint of my limited technical knowledge of the subject, the add-on seems to work well in showing how up-to-date the urclassifier3.sqlite database is.

Comment 7

[AndrewM]

I just tried installing the extension above on Firefox 3.0.1 and when I typed in about:safebrowsing and hit Enter, it says "Loading..." and the throbber goes for about 5 seconds and then just stops.

In the Error Console when I install the extension it says:

No chrome package registered for chrome://aboutsafebrowsing/locale/aboutsafebrowsing.properties

and after typing in about:safebrowsing and then hitting Enter it says:

Error: subs is undefined
Source File: file:///C:/Users/Andrew/AppData/Roaming/Mozilla/Firefox/Profiles/spuijs7a.default/extensions/[e-mail]/components/AboutSafebrowsing.js
Line: 150

(Line 150 is the line that reads: table.serverSubs = subs.split(":")[1]; )

Just to clarify (given that the comment above this one was me nomming this bug for blocking Firefox 3.1) the extension worked fine for me when I installed it a while ago, but it's not working for me now on this profile (or indeed a new profile). Any ideas?

Comment 8

[Dave Camp (:dcamp)]

Oh, yeah.  I ran in to this a little while back, and haven't posted my updated/fixed version.

Google added a table that doesn't have any subs to the list.  It's a fairly simple fix, I'll post the updated version when I can find where I put it...

Comment 9

[juan becerra [:juanb]]

campd, I can haz extenshun update?

Comment 10

[Dave Camp (:dcamp)]

Attachment: updated extension

Here it is.

Comment 11

[Dave Camp (:dcamp)]

A few things to add to this would be:

* When was the last fetch request
* What was the result of that fetch request
* Are we in backoff

Comment 12

[Johnathan Nightingale [:johnath]]

Now at https://addons.mozilla.org/en-US/firefox/addon/9112

Comment 13

[Mike Beltzner [:beltzner, not reading bugmail]]

I think this is better as an add-on than a built-in about: page, especially since the benefit is entirely for debugging purposes.

Comment 14

[Przemyslaw Bialik]

If about:safebrowsing page (or more friendly ver of it for the average user) is not wanted then maybe at least show % of local database in Options -> Security ?

Database is not bundled with Firefox and it takes some time to load it (internet download speed varies) and it has to be updated constantly so I think there must be some info available regarding safebrowsing state in the core.
My idea is simple but it would cover most of the needs.