932036 - Forget About this Site doesn't clear moz_hosts in places.sqlite

Status: RESOLVED INACTIVE

(Firefox :: Bookmarks & History, defect)

Description

[Peter Ford]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0 (Beta/Release)
Build ID: 20130910160258

Steps to reproduce:

Open Firefox 24.0
Open a new tab
Proceed to www.guardian.co.uk
Site is deprecated so forwarded to http://www.theguardian.com/
Close Firefox
Firefox > History > Show All History
Type www.guardian.co.uk in Search History
Right Click on entry, choose forget about this site
Close Library

EITHER

Open New Tab
Start typing www.guardian.co.uk 

OR

Open place.sqlite
run "SELECT * FROM moz_hosts where host like "%guardian%""


Actual results:

Open New Tab
Start typing www.guardian.co.uk 
Full address will auto-suggest.

Open place.sqlite
run "SELECT * FROM moz_hosts where host like "%guardian%""
www.guardian.co.uk returned


Expected results:

No Auto-suggest
No entry in moz_hosts

Comment 1

[Peter Ford]

Also see:

https://support.mozilla.org/en-US/questions/937290

Shouldn't have to resort to altering places.sqlite

Comment 2

[Jeremie Miserez]

I can confirm this. In fact, I think it is a pretty big privacy/security issue: Even clearing the complete history does not remove the autocomplete suggestions from the url bar.


In my case, the moz_hosts table contained a log of all (!) hosts I ever visited, even after I cleared the history.

Other users (see e.g. http://askubuntu.com/a/277241/145754) also seem to experience the same issue with URLs not being removed from the moz_places table, although for me these entries were cleared.

I would like to stress that it is impossible to remove these URLs through the use of Shift-Delete, Delete, or any other method in the GUI (even though Firefox documentation suggests that it is possible). The entries do not appear in the dropdown menu of the URL bar, instead they just autocomplete the domain name inline. The only workaround that works is editing the moz_places and moz_hosts tables in the places.sqlite file manually or removing the place.sqlite file altogether.

Comment 3

[Marco Bonardo [:mak]]

Removing history does proper cleanup unless:
1. the database is broken (we have maintenance to fix most of these problems that runs once a week, but it was broken, will be fixed in  Firefox 39)
2. you check too early. expiration is asynchronous to avoid interrupting your work, and might take some time before everything is gone. If you do a clear history and check immediately it's likely entries stay there for some minutes.

moz_hosts contains host that have an entry in history or bookmarks, it doesn't track anything that is not blokced by some other entry.

We care a lot about privacy and we have tests ensuring we don't leave stuff behind, don't worry that we don't track anything by will.
The important thing is to ___NEVER___ edit places.sqlite manually, that will break data coherence and likely break expiration forever.

I mostly care about entries that are in moz_hosts but have no reference in moz_places so if you find any of those please let me know.

Comment 4

[Jeremie Miserez]

1. I don't understand what you are trying to say here. Is it broken or not? IMHO it's broken. Also, I had already noticed the URL suggestion being out of place a week ago, but did not further investigate. This suggests that the maintenance is not working, but I can't be sure.
2. I only noticed the problem because the particular URL was one that I haven't accessed in over two weeks, and I remember clearing the history this week at least twice. So checking too early is most definitely not the issue.

moz_hosts: My moz_hosts does in fact contain a lot of entries (seemingly every host I ever visited!) that are not linked to moz_places, moz_bookmarks, or moz_favicons. For moz_places and moz_favicons I checked the url field and for moz_bookmarks I looked at the id and fk fields. None of the foreign keys match, and I can't find any references to the URLs in any other tables.

moz_places: This seems to only contain the most recent history, as it should.

Regarding never editing the file: I never edited the file, as I did not know about places.sqlite until today. I have deleted it now and am using a new database, but losing all of the history and potentially the most recent bookmarks is NOT a solution that should be recommended to users.

Note that this issue only seems to affect regular (non-private) browsing. All of the entries are entries from regular non-private browsing.

Please let me know if you need any further information or if you have an idea about what is causing this bug. I am slightly hesitant to share my places.sqlite file as it contains quite a lot of information and might contain sen

Comment 5

[Jeremie Miserez]

...sitive information (it seems to contain every host I ever visited).

One more thing: After deleting places.sqlite and using a new file, deleting the history works without a hitch. But still, I have no idea how long this will last: besides the autocompleted URLs there is no hint to the user that the history might not have been cleared or that the database is corrupted, as everything else is working fine.

Comment 6

[Marco Bonardo [:mak]]

(In reply to Jeremie Miserez from comment #4)
> 1. I don't understand what you are trying to say here. Is it broken or not?

it's not broken as far as I can tell and according to our automated tests. But bugs happen!

> moz_hosts: My moz_hosts does in fact contain a lot of entries (seemingly
> every host I ever visited!) that are not linked to moz_places,
> moz_bookmarks, or moz_favicons.

This can only happen if the database is corrupt, any entry in moz_hosts should have at least one url in moz_places that points to that host. We have a trigger ensuring removals.

> moz_places: This seems to only contain the most recent history, as it should.

it contains any url, regardless it coming from history, bookmarks, tags or keywords.

> Please let me know if you need any further information or if you have an
> idea about what is causing this bug. I am slightly hesitant to share my
> places.sqlite file as it contains quite a lot of information and might
> contain sensitive information (it seems to contain every host I ever visited).

I'm not really interested in personal information, as the owner of this module I receive a lot of databases from users, I usually just delete them after debug, and I'm not really a curious person... Btw, I can understand.
Maybe we could try some remote debugging, for example you might try with my add-on (https://addons.mozilla.org/en-US/firefox/addon/places-maintenance/) to force the cleanup on your "broken" database and see if that helps.

> One more thing: After deleting places.sqlite and using a new file, deleting
> the history works without a hitch.

OK, this seems to confirm my doubts about database corruption.

> But still, I have no idea how long this
> will last: besides the autocompleted URLs there is no hint to the user that
> the history might not have been cleared or that the database is corrupted,
> as everything else is working fine.

Yes, that's why we have weekly maintenance, but as I said it has been broken up to Firefox 39 :/

Comment 7

[BMO Automation]

Per policy at https://wiki.mozilla.org/Bug_Triage/Projects/Bug_Handling/Bug_Husbandry#Inactive_Bugs. If this bug is not an enhancement request or a bug not present in a supported release of Firefox, then it may be reopened.