439666 - Sandboxie reports excessive file size for urlclassifier3.sqlite

Status: RESOLVED WORKSFORME

(Toolkit :: Safe Browsing, defect)

Description

[Patrick]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9) Gecko/2008052906 Firefox/3.0
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9) Gecko/2008052906 Firefox/3.0

I am not confident that firefox 3 is sufficiently secure when running sandboxed in sandboxie.

I regularly ran previous Firefox in sandboxie version 3.26 with no problems.

On first attempt to run this new version 3 of firefox, sandboxie presented th error message:
SBIE2102 File is too large to copy into sandbox: urlclassifier3.sqlite

The sandboxie data base shows that this warning relates to files larger than 4Gbytes.

Although I am able to work around this problem by changing the default settings in sandboxie, I am concerned that users of older computers may have problems with this large file size causing slowdown, or even an inability to run firefox sandboxed.


Reproducible: Couldn't Reproduce

Steps to Reproduce:
1. fresh install sandboxie 3.26 with default settings
2. fresh install Firefox 2.xx with import of bookmarks html file 173kb (or install in reverse order) - works fine to here.
3. fresh install Firefox 3 - full version as above.
Actual Results:  
On first re-use of sandboxie after new install of current firefox,
sandboxie showed the above error message.




Expected Results:  
Activate firefox in sandbox with no error messages as normal.



Apart from the large number of bookmarks, nothing else is imported or added.
I do not use themes, feeds etc.

My computer is a D805 @2.73gig on AWD8 mobo with 1 gig of ram.

The PC is extremely stable.  XP is fairly stable, about 2 minor problems per month.

I have never experienced any problems with previous firefox versions running in contemporary sandboxie versions.

I use firefox on a daily basis, of which on average 1 or 2 hours is in the sandbox for spam/suspect site testing.

Comment 1

[Daniel Veditz [:dveditz]]

There is, unfortunately, a lot of malware out there. This huge file is indeed the expected size.

The Firefox 2 database never got as large because it only tracked active phishing sites; Firefox 3 also tracks sites with detected malware, which can include innocent sites that were hacked so it turns out there were a lot more sources of badness.

If this is something you're doing for spam/suspect site testing you may want to turn off the malware detection feature anyway. Turning it off won't shrink the existing stored data (since it's so huge we keep it in case you turn it back on -- there will be some stale data, but depending on how long it was off a lot of it will still be relevant and save a lot of network traffic before the user is up to date).

I'm going to confirm this bug as a legitimate file size concern (though one we will probably decide in the end to live with). It's not an exploitable security issue that needs to be hidden, though.

dave: should the "Phishing protection" component be renamed to a more generic "safe browsing", or am I not finding the right component?

Comment 2

[Daniel Veditz [:dveditz]]

My copy of this file is a bit over 50Mb, not anywhere close to 4Gb (missed that bit). Are you sure your copy is this large?

Comment 3

[Patrick]

Attachment: DOS ASCII DIR list of Firefox installed files

The only over-large file I am seeing in this list is:
9,715,200 xul.dll

I do not see urlclassifier3.sqlite

I am not very proficient at this level, is *.sqlite within the dll ?

Regarding running Firefox without inbuilt protection:
by running with known-bad-site feature active, I can discriminate
new and unreported unsafe sites more effectively.

Comment 4

[Matthias Versen [:Matti]]

The urlclassifier3.sqlite should be in your profile directory as all user related data.
- http://kb.mozillazine.org/Profile_folder

Comment 5

[Patrick]

Attachment: documents and settings directory DOS ASCII list

Apologies for missing this essential info - user settings ASCII dir list.
................................
Related problems, should these be posted elsewhere?

1 ... AVG8 toolbar de-activated on new FF 3 install/upgrade - firefox install reports incompatibility.
AVG8 has a look-ahead feature which tests each web-page shown in google search results for active malware.

2 ... Known malware site not blocked or warned by new features.
For maximum security, web users are encouraged to type URL directly into the browser bar.
The common typo c m instead of c o m produces, for almost all sites on the web, a re-direct to agoga dot com - a McAfee Site Advisor RED site.
example: natwest dot cm >>> agoga dot com
Exception 1:
firefox dot cm >>>
http://searchportal.information.com/?o_id=65407&domainname=firefox.cm
shows Firefox's connection failed notice
note: information dot com is a McAfee SA YELLOW site.

Exception 2:
google dot cm >>>
padminie.com/?3838
blank html

..................
Anyone with admin privileges, please feel free to move any of the above to
new bug report/security notices or whatever action may be needed.

Comment 6

[Patrick]

Apologies for posting ACTIVE malware link.
Anyone with admin privileges, please edit above to hxxt:// ...

Thank you.

Comment 7

[Johnathan Nightingale [:johnath]]

Based on that listing there do not appear to be any files approach the 4GB limit you talked about earlier.  As Dan says, a 50MB malware db is eventually expected, but it looks in this case like Sandboxie is misreporting?

Comment 8

[Patrick]

In the listing, list2.txt >
17/06/2008  12:18         4,977,664 urlclassifier2.sqlite

as I write this, explorer shows
D:\Documents and Settings\Administrator\Application Data\Mozilla\Firefox\Profiles\9ddypvhe.default\urlclassifier2.sqlite

properties: created 12 Jan 2008, 23:20:30 4,977,664 bytes (on disk 4,980,736 bytes)

Sandboxie is not misreporting this file's size.

Comment 9

[Patrick]

I just spotted that you are refering to urlclassifier3.sqlite

but I have: urlclassifier2.sqlite

How come I seem to have an older version v2 of this file?

Just to confirm, I am using version: 
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9) Gecko/2008052906 Firefox/3.0

Install file: Firefox Setup 3.0 RC 3.exe
file version 4.42.0.0
timestamp wed 11 June 2008 10:16:54

Comment 10

[Johnathan Nightingale [:johnath]]

(In reply to comment #8)
> In the listing, list2.txt >
> 17/06/2008  12:18         4,977,664 urlclassifier2.sqlite
> 
> as I write this, explorer shows
> D:\Documents and Settings\Administrator\Application
> Data\Mozilla\Firefox\Profiles\9ddypvhe.default\urlclassifier2.sqlite
> 
> properties: created 12 Jan 2008, 23:20:30 4,977,664 bytes (on disk 4,980,736
> bytes)
> 
> Sandboxie is not misreporting this file's size.

4,977,664 bytes is 4,860 kilobytes, which is 4.7 megabytes.  If sandboxie is reporting it as being multiple *gigabytes*, it is reporting it as being one thousand times larger than it is.

Comment 11

[Daniel Veditz [:dveditz]]

A 4 megabyte urlclassifier2.sqlite for Firefox 2 sounds about right.

The Firefox 3 file was moved to the directory where we keep the cache files, basically stick "Local Settings" between "Administrator\" and "\Application Data" and the rest of the profile path should be the same. If it's larger than 55Mb or so move it out of the way (in case we want to inspect it) and let Firefox rebuild the file.

Comment 12

[trogdor]

I would like to make a few additional points:
FYI: I am not a developer; however some research shows:

1) You can shrink the SQLITE files using SQLITE3 (external install).  My urlclassifier3.sqlite went from 60 megs down to 20 megs.  Using the syntax:
sqlite3 urlclassifier3.sqlite VACUUM
For this to work Firefox cannot be running; because the db file is locked.
2) Using the "auto_vacuum pragma" might be work; but:

"As of version 3.4.0 (the first version that supports incremental_vacuum) this feature is still experimental. Possible future changes include enhancing incremental vacuum to do defragmentation and node repacking just as the full-blown VACUUM command does. And incremental vacuum may be promoted from a pragma to a separate SQL command, or perhaps some variation on the VACUUM command. Programmers are cautioned to not become enamored with the current syntax or functionality as it is likely to change."

http://www.sqlite.org/pragma.html#pragma_incremental_vacuum

3) This isn't only a windows specific issue.  It happens on Linux as well.  A temporary workaround can be used by creating a batch file or shell script that:

a) Detects if the Firefox process is running.
b) Check for the number of profiles
c) If not running then execute: sqlite3 urlclassifier3.sqlite VACUUM
d) Run this in the Windows scheduler or as a cron job or manually from the command line.

PS: Running sqlite3 xxx VACUUM on all files takes something like 10-16 seconds on a PIII 1 GigaHertz.

Comment 13

[David Dahl :ddahl]

We recently added VACUUM support to places.sqlite, which we can eventually use as a blueprint for VACUUMing more of our databases. Also there is a more comprehensive bug for a global vacuum componnent: https://bugzilla.mozilla.org/show_bug.cgi?id=541373

Comment 14

[Philip Chee]

Fixed in Bug 673470 ( Replace the sqlite safeb store with a flat file)