295681 - Make infinite history work, and keep it working

Status: VERIFIED WONTFIX

(Firefox :: Bookmarks & History, enhancement)

Description

[August Mayer]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050517 Firefox/1.0.4 (Debian package 1.0.4-2)
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050517 Firefox/1.0.4 (Debian package 1.0.4-2)

Infinite history would be a very useful feature. For some users, it even could
replace the manually administrated bookmarks with an automatic, searchable
information gathering functionality. There are privacy concerns.


Reproducible: Always

Steps to Reproduce:
1. Set history expire to 99999 days.
2. Browse for some months with this setting


Actual Results:  
Some parts of Firefox slow down. Memory usage and startup time increase.


Expected Results:  
Firefox should keep reacting snappily. Firefox shouldn't use more memory for
this than when freshly installed.

Assuming that users browse the same set of pages (slashdot.org for example) most
of the time (except when they explicitly search for new information), database
growth would also be moderate. And, so what, 100 MB of collected URLS after two
years? No problem, the current java runtime alone eats this without providing
useful data.

Maybe it would be interesting to put the history data into a 'real' database,
which would keep indices and thus speed up searching in the history.

Also, the history sidebar would need some adaptation, so that the "older"
history would be structured (e.g. in monthly folders).

Simply searching the on-disk data doesn't suffice, because then Firefox would
start to disk-thrash when searching for URL bar history completion items. A
possibility would be an in-memory index cache with a maximum upper limit. Or a
'real' database, which likely already implements this stuff.

Comment 1

[August Mayer]

See also:
 Bug#241438 "please make history.dat easier to parse (i.e., not Mork)"
 Bug#193236 "History Window Excruciatingly Slow"
 Bug#223476 "large history makes Firefox painfully slow in several ways"
 Bug#294332 "big history.dat creates extremly long startup times"
 Bug#279342 "big history.dat file kills menubar pulldown performance"
 Bug#17091  "Migrate history DB"  (this bug is probably obsolete)
 ...and probably others.
 

Comment 2

[Jo Hermans]

Sigh .. that's being worked on. See
<http://wiki.mozilla.org/Mozilla2:Unified_Storage>

Comment 3

[tom]

History should show each page visited in the order visited historically.
Presently it only seems to show an alphabetic list which is little help unless
you constantly look up the  the url's every time you click. Every other browser
blows us away here.

Comment 4

[Marco Bonardo [:mak]]

This is somehow managed in Trunk, history is mantained longer, and is searchable through places organizer... having an infinite history will never be possible because of performances of an infinite database (those infos should be saved somewhere) and there is already a bug to "see how much history can we show"...

So i suggest mark this invalid

Comment 5

[u279076]

I agree with Marco.  Resolving INVALID.

Comment 6

[u279076]

resolution changed to WONTFIX following tracy's advice:

"It is a valid bug, but we're not going to waste resources and take the performance hit to fix it."

Comment 7

[Dimitrios Apostolou]

Found this old bug while searching how I can retain unlimited history. Can an option "Never expire history" be added in preferences? Or a hidden about:config switch? Obviously a user selecting it knows there are some performance drawbacks involved.

I guess this bug can be reopened now that there is a fully indexed db backend.

Comment 8

[u279076]

No, this bug won't be reopened. It's not in scope of core feature development, even if confined to a "hidden" pref in about:config. It's not something the masses are demanding, and it would likely cause a performance hit (at a time when we are working to improve performance).

I suggest this is something that *should* be solved by an add-on.

Comment 9

[u279076]

PS. If the case could ever be made for making this in scope of core feature development, it would have to be filed in a new bug.

It's often better to make a case for a feature in a new bug than trying to modernize a bug which has been closed for nearly 5 years.

Comment 10

[Masatoshi Kimura [:emk]]

(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #8)
> I suggest this is something that *should* be solved by an add-on.
How can add-ons ensure that the history is preserved when Firefox is launched in Safe Mode? I don't think it's possible unless Firefox itself provides an option.

Comment 11

[u279076]

(In reply to Masatoshi Kimura [:emk] from comment #10)
> I don't think it's possible unless Firefox itself provides an option.

I really don't know as I am unfamiliar with this particular part of the code. 

Regardless, this bug has been closed for 5 years. 

If you really feel like this is important, file a new bug and write a patch.

Comment 12

[Marco Bonardo [:mak]]

An advanced user is able enough to figure out how to do this in about:config, surely a pref will never be provided in the ui.

Comment 13

[Dimitrios Apostolou]

Hi Marco, I've done a fair amount of searching and the only relevant option I've found in about:config is "history_expire_days_min". However setting it to a number like 9999 is not the same with disabling expiration. Plus I've read bug #674210 and the option in about:config is not mentioned anywhere, so I assumed that it's a deprecated option, unused like many others. Can you verify that raising this option will indeed postpone expiration indefinitely?

The only way I've found so far to disable expiration is via the "Expire history by days" add-on, but I'd prefer not to handle history issues via an add-on.

Comment 14

[Marco Bonardo [:mak]]

(In reply to Dimitrios Apostolou from comment #13)
> The only way I've found so far to disable expiration is via the "Expire
> history by days" add-on, but I'd prefer not to handle history issues via an
> add-on.

Why? It exists for that exact purpose. Btw the only thing the add-on does is setting places.history.expiration.max_pages to a really large value. Clearly this will soon or later create you performance problems, and at that point you may forget you changed this option and just think Firefox is behaving badly. That by itself is the most valid reason to not expose an option.

Comment 15

[Dimitrios Apostolou]

(In reply to Marco Bonardo [:mak] from comment #14)
> Why? It exists for that exact purpose. 

I see my browsing history as an important log of my activities that serves me many times to remind stuff I wouldn't be able to find otherwise. Thus I can't risk losing history, for example by using safe mode or because the add-on was disabled in a new firefox version (I use beta or even nightlies pretty often).

> Btw the only thing the add-on does is
> setting places.history.expiration.max_pages to a really large value. 

Thanks for the tip, I didn't know about that option. But I can't find it, maybe you mean one of the following options?

places.history.expiration.transient_current_max_pages
places.history.expiration.transient_optimal_database_size

> Clearly
> this will soon or later create you performance problems, and at that point
> you may forget you changed this option and just think Firefox is behaving
> badly. That by itself is the most valid reason to not expose an option.

I understand the risks and why it shouldn't go in preferences, but as an advanced user I'd like to tweak that option in about:config. But still the danger of losing history doesn't go away when a new version of firefox changes the logic and max_pages gets deprecated in favor of some other option, like it happened with expire_days_min.

What I'm saying is that a clear, documented option in about:config would be an OK solution. Something like history.never_expire==true or even expire_days==-1. Since it is not a priority for you would a patch be acceptable?

Comment 16

[Dimitrios Apostolou]

Marco: Never mind about the places.history.expiration.max_pages option, I just figured out that it doesn't exist by default, but setting it actually affects expiration. Thanks for helping.