384420 - History and Bookmarks should be in two separate files

Status: RESOLVED WONTFIX

(Firefox :: Bookmarks & History, defect)

Description

[Mike Kaply [:mkaply]]

With places, history and bookmarks are combined into one file (places.sqlite). This means that is is no longer possible for an external application to clear the history (in the past it could delete history.dat).

History and bookmarks should be in two separate sqlite files so it is possible to clear the history externally without deleting bookmarks.

Alternatively, something hacky could be done like keep a small history.dat file around and if you see it got deleted, delete the history and recreate the history.dat file.

Comment 1

[Marco Bonardo [:mak]]

i think that an external application should connect to sqlite file, and clean the correct table, without deleting the entire database...

There is probably also an overhead in mantaining 2 opened databases at the same time, instead of one... 

Comment 2

[Mike Kaply [:mkaply]]

That external application would then have to include sqlite. that's not practical.

In the past we have had a very simple way to delete the history. We are taking that away...

Comment 5

[Michael Lueck]

Subscribe

I just bumped into this. I wanted to delete the history.dat as I suspected it is corrupt... found out that it has turned into places.sqlite which has also the gobbled bookmarks.html file!

hhhmmm, Greetings to Michael Kaply... its been a long time since we crossed paths! ;-)

Comment 6

[Michael Lueck]

I have confirmed I have a case of corruption in the places.sqlite file. I have noticed since upgrading my profile from FF2 to FF3 that links do not stay the visited color for the default 90 days, rather they change back to the non visited color after a day or so!

My recent posting to NNTP mozilla.support.firefox

=========================
Michael Lueck wrote:
> Perhaps I will test renaming my places.sqlite file and see if that
> doesn't fix it. Then I would have done your suggested method and that did
> not fix it, but my idea would have worked... We shall see.

Indeed, my idea seems correct. Steps I did:

1) I renamed off the places.sqlite file, start FF, and clicked several links on the test page. (URL above in this thread).
2) I exited FF, renamed that file as places.sqlite.TEST
3) Renamed back my production places.sqlite file

All of this was on the 30th.

Now I tested that fresh places.sqlite file I had created.

1) Renamed the test file, those links still show the visited color.
2) Renamed the production places.sqlite and the same links do not show up as the visited color.

Clearly, there is some sort of corruption in the history table. Since history and bookmarks are in the same file now, I can not solve this corruption by merely deleting the history file.
=========================


So, any tricks to get all of my bookmarks transfered to a clean/new places.sqlite file?

Comment 7

[Marco Bonardo [:mak]]

how is comment 6 related to the request of splitting places.sqlite into 2 separate files?

If your places.sqlite is corrupt you shoul be posting in a corruption bug or file a new bug, or ask for support on mozillazine's forum, this is not the right place, sorry.

Comment 8

[Michael Lueck]

Comment 6 is directly related as:

I had no corruption while on FF2. I migrated my profile to FF3, and all sorts of weird things started happening.

One was that clicked (visited) links changed back to the non visited color after a day or so rather than the default of 90 days, which I have it set to.

In researching, clearing the history did not correct the corruption.

However, creating a new places.sqlite does correct the problem.

Thus, one more vote to bring back the former separation of bookmarks from history as if history got corrupted one could simply delete the file.

aka "This is why the two files should NOT be combined into one file!"

Better?

Comment 9

[peter]

It is pitty the bookmarks in FF3 are no more usable, because of (a) privacy issue related to keeping history in places.sqlite and (b) the fact that it can be unstable no matter what firefox people do, as the browser may die at any moment while browsing non-correct pages.

As a result, I spent two months to program my own external bookmarks EXE tool now. I imported my old FF2 bookmarks there, and now I do not use FF3 bookmarks anymore. I simply wipe down securely the FF3 profile data when done with FF3. It is a bit inconvenient and I have to drag and drop between the tools when I have to save/open bookmarks, but I was left with no other choice. I choose not to do a FF3 extension and you never know with out FF team will come up. A separate bookmarks tool that can read/write the old Netscape1 format can be used with all browsers.

I have to thank FF3 places.sqlite decision for this painful motivation.

Comment 10

[Michael Lueck]

I found a workaround to at least keep the bookmarks while creating a fresh places.sqlite file:

"Unable to change bookmarks in FF3"
http://support.mozilla.com/tiki-view_forum_thread.php?comments_parentId=63721&forumId=1#threadId63788

I exported my bookmarks, exited FF, renamed my production places.sqlite, copied in the bookmarks.html I had exported, started FF, and it created a fresh places.sqlite from the bookmarks.html file.

Works for me!!!

And the size shrunk GREATLY! 6463488 was the old one, and with all of the bookmarks imported, the new one is only 372736.

Comment 11

[Hadmut Danisch]

PLEASE, put the history and the bookmarks, settings etc. in separate files. This is a severe violation of privacy since the complete history always goes into any backup or synchronized copy of the bookmarks and settings. 

It is definitely not a good idea to have this all in one file, which, btw. makes it difficult to synchronize bookmarks between different machines.

Comment 12

[Marco Bonardo [:mak]]

To move bookmarks without history you should use the JSON backup/restore function from Library (or import/export).

splitting places.sqlite will be considered for next major version, pros and cons are clear enough so please stop repeating them here.

Comment 13

[drichard]

We (City of Largo) are on board with the severity of this issue.  We have 750 users on one server and each night we delete the places.sqlite file.  We have no desire to keep and then have to backup this much data for sites visited that can be easily found again with a google search.  I had to really rig FF to resolve bookmarks + sites visited being in the same database.

I globally pushed this setting:

user_pref("browser.bookmarks.autoExportHTML", true);

This forces FF to dump a flat file at session close, and then when places.sqlite is missing the next morning a new one is generated using the bookmarks.html file.  This is a pretty clunky approach, and sometimes users crash FF and lose bookmarks they have added that day.

For enterprise use, we are requesting an optional setting to push this information into two databases.   

Thanks for consideration.

Comment 14

[Gervase Markham [:gerv]]

Bug 451915 - move Firefox/Places bugs to Firefox/Bookmarks and History. Remove all bugspam from this move by filtering for the string "places-to-b-and-h".

In Thunderbird 3.0b, you do that as follows:
Tools | Message Filters
Make sure the correct account is selected. Click "New"
Conditions: Body   contains   places-to-b-and-h
Change the action to "Delete Message".
Select "Manually Run" from the dropdown at the top.
Click OK.

Select the filter in the list, make sure "Inbox" is selected at the bottom, and click "Run Now". This should delete all the bugspam. You can then delete the filter.

Gerv

Comment 15

[Michael Lueck]

Well I am please to report that with the release of FF5, the problem with the "history file seeming corrupted" has been resolved. Websites are once again remembering that I have clicked on various links "days ago". So what ever contributed to that solution, thank you.

Comment 16

[Mike Kaply [:mkaply]]

Do we genuinely believe that we will ever split bookmarks and history?

Or has this ship sailed?

Comment 17

[Marco Bonardo [:mak]]

There is no plan to do this at the moment, and it would be sort of painful cause bookmarks should then grow a way to link to pages that is portable (to avoid mismatches if one db gets lost/corrupt and the other doesn't), the only viable way would be through url, but url indices are a pain, cause they take up the most space in the db. It's adding complication.
Based on the experience from bug 889561, I'd say this is unlikely to happen.

If in the future we'd decide to completely retain bookmarks in memory, then that's another story and will likely have its own bug, but it won't happen for the reasons expressed here.