Closed Bug 193236 Opened 22 years ago Closed 16 years ago

History Window Excruciatingly Slow

Categories

(Core Graveyard :: History: Global, defect)

Product:
Core Graveyard
Component:
History: Global
Type:
defect
Priority:
Not set
Severity:
major

Tracking

(Not tracked)

Status:
RESOLVED INVALID

People

(Reporter: bill+mozilla-bugzilla, Unassigned)

References

Details

(Keywords: perf, Whiteboard: [asaP1])

1.3b, OS X 10.2.3, G4/400, 1GB RAM Opening the history window can take 30 seconds sometimes, and never less than 15. Beachball spins frequently. While it's open the whole browser drags ass. Closing History restores snappiness. I saw bug 91044, but that seems to be mostly scrolling and this problem doesn't show under linux (my other platform).
Keywords: perf
This should block 1.4a, as we need to enhance performance. Reporter, do you see this with a lot of entries in your history.dat? When you see this problem, roughly how big is your history.dat file?
Flags: blocking1.4a?
I'm still seeing this on the 1.3 candidate build. Right now my history.dat is 3.2M. I have it set for 60 days.
Flags: blocking1.4a? → blocking1.4a-
I've noticed this as well with Netscape 7.01 on Win32. My history.dat was probably in excess of 5 MB when I first noticed the problem. (It's currently about 11 MB.) I haven't yet confirmed that the problem still exists in 7.1 or in other products (Mozilla, Firebird), but I expect it will.
Changing to All as I have this problem with 1.6 (and previous milestones too) on Win2K.
OS: MacOS X → All
Hardware: Macintosh → All
I can confirm this on Firefox: UA: Mozilla/5.0 (Windows; compatible; U; Windows NT 5.1; en-US; rv:1.8a) Gecko/20040417 Firefox/0.8.0+ Although I'm currently using a trunk build, the history performance in Firefox has been really terrible for a long time now (well before the 1.7 branch). My history.dat file is approx. 8.36MB. I wonder what the problem is here. Is it just slow reading the file, or is it a sort issue? Here's hoping this gets looked at soon. I may take a peak and see if I can figure it out, but I think someone with more experience needs to take a look.
The slowness of the History window in Mozilla (and in Firefox by extension) is a vexing and long-standing inefficiency, and it is one of the few areas in which Gecko browsers suffer greatly in comparison to IE. I have a large history.dat file (over 18 MB) in my Moz 1.7.3/WinXP profile, so I expect a short wait when opening my history. But 30+ seconds is way too long, particularly because I am unable to access other parts of the browser in the meantime. Also, I notice a sluggishness when deleting history entries. In particular, deleting ten or more entries at a time results in an exponential increase in execution time. While waiting, I notice that my CPU utilization goes up to 100% and stays there continuously. In contrast, my IE history folder is also rather sizeable (nearly 12 MB including subfolders), but IE's history pane comes up instantaneously and remains responsive while I search, browse, and delete entries. IE uses a two-level tree structure to store its history. In the History folder is a master index.dat file that presumably contains basic information about all URLs browsed. Below this top level are subfolders for each week of entries. Each subfolder contains its own index.dat file, which presumably contains more detailed information about that week's entries. (I use "presumably" because I don't know anything about the internal data structure of the binary index.dat files, but this can probably be learned with some doing.) In my case, the master index.dat file is about 2.5 MB and the combined size of the secondary index.dat files is about 9.5 MB. I'm not necessarily advocating that Mozilla emulate IE's method of storing the browser history, but IE's advantage in quickness cannot be ignored. The Gecko browsers badly need a more efficient data structure to store history entries, because the existing one simply does not scale well.
Since my previous post I have read through the comments of bug 241438 (regarding Moz's history implementation), bug 245745 (re history in Firefox), and bug 175600 (re the disk cache). I am encouraged to see that my concerns are already being addressed with the intended conversion of history entries and other profile data from the kludgy Mork file format to a speedier, SQLite-powered database. This step represents a major change to the inner workings of Mozilla, but it is definitely worth the effort and time required to make it work. The open-source SQLite engine appears robust enough to address the scalability issues that I mentioned, yet compact enough to keep the Moz code footprint at a manageable size. Relational databases in general are a mature technology that is highly scalable and highly optimizable in practice. The poorly designed Mork format, however, offers no such advantages and is destined to wither once it is pruned from the Mozilla code base. IMO the SQLite code should be fast-tracked, as it will likely result in a major performance increase across the board. I hope that it will result in more efficient memory usage as well, as Moz's enormous memory footprint is another longtime bugaboo of mine.
its actually worse for me. my history.dat has grown to 16 megs, mozilla now takes a 3-4 sec break before it renders the first page. if i delelte the files its fast again...
Blocks: 223476
Flags: blocking1.8b?
Flags: blocking-aviary1.1?
Flags: blocking1.8b? → blocking1.8b-
Whiteboard: [asaP1]
Flags: blocking1.8b4?
Flags: blocking1.8b4?
Flags: blocking1.8b4-
Flags: blocking-aviary1.1?
Flags: blocking-aviary1.1-
Depends on: 245745
*** Bug 294332 has been marked as a duplicate of this bug. ***
Places replaces the current history system in current Firefox nightlies. It has different performance characteristics from the old system. The new places window being slow with large histories is bug 330906.
OK, I'm adjusting the depends-on field to reflect Brett's comment. depends on: 245745 -> 330906 at first I thought of dup'ing this to that, but that's not right either. We dumped Mork for SQLLite because Mork was too slow to be usable for bugs like this, but now SQLLite is too slow to be usable. So, this bug probably needs to live in case we dump SQLLite for a BDB or something to get acceptable performance, then we can change the depends-on again.
Depends on: 330906
No longer depends on: 245745
Assignee: bross2 → nobody
QA Contact: kasumi → history.global
I have a 10MB history (about a year). I was trying to clean it up manually; I like to keep only wikipedia and bug reporting sites history. It ran for more than an hour and didn't even complete. I was asked numerous times to stop or continue a slow script! I cannot believe this was implemented in javascript! It's definitely time for a real format with b-tree indexing. The comment above about using embeded DB opensource is a great idea. Hypersonic or Derby would work fine if you want to do that in java. The most important flaw to me is the history view itself. It lacks flexibility. Whoever decided to group everything older than 6 days together hasn't used the history on a long term, obviously. Ultimately, I expect a simple ORDER BY and GROUP BY syntax, with a WHERE clause on each field. Clearly a SQL wrapper could be made to vastly improve the history browsing. Meanwhile, all file i/o should be done in temporary RAM buffers (even 100MB ram isn't hard to find these days) and commit the final history file in 1 shot, then hot-swap the filename atomically. Finally, I am wondering how the painting of visited link is slowed down by searching these huge history DB.
Also, we should introduce the notion of blacklisting/whitelisting URLs from the URL history DB. This way, users cans not only disable the history or only disable/enable it for some sites, for any privacy reasons without the current ALL-or-nothing approach relative to the history file itself.
History in the nightly builds of 3.1b3pre is painfully slow. When I click on History it takes 10+ seconds for the Menu to display, and then when I click on Show All History, it takes another 10+ seconds for the window to display. I only have about 60 days worth of history but even scrolling the history window is laggy. I call this painfully slow because displaying the Bookmarks is instant, and to the best of my knowledge, History and Bookmarks share the places.sqlite file, and mine is only 62MB in size. It should hardly be an issue loading something like this on my Q9450 w/ 4GB RAM and fast SATA2 drive.
if you install the SQLite Manager extension and compact your places.sqlite file, it seems to help quite a bit.
#15, while I appreciate your suggestion, I'm just not interested in running an extension to fix a problem that shouldn't exist in the first place. If there is no lag/slowness when opening the bookmarks, there should be no lag/slowness when viewing your history. It's all in the same file. :)
Add oil!
Flags: blocking1.9.0.9?
Flags: blocking1.9.0.9?
As originally filed, this bug is INVALID. The back and front end for History have been completely re-written. Any issue you are currently seeing is *not* this bug, and you should file a new one.
Status: NEW → RESOLVED
Closed: 16 years ago
Resolution: --- → INVALID
Er, to clarify, I'm not saying we have a problem, but this bug is slowly morphing into different things that were not what it was really filed for. This ends up making things significantly harder for developers to track what the real problem is. Me marking this as INVALID is not saying we don't think this is a problem. In fact, there are several bugs open on things that can make history slow right now: bug 487777 bug 466421 bug 443328 bug 432706 bug 417262
(In reply to comment #19) > Er, to clarify, I'm not saying we have a problem He's really not saying that we _don't_ have a problem. :)
(In reply to comment #20) > He's really not saying that we _don't_ have a problem. :) Hey look - typing fail! Yes, I mean to say that we are not saying that we do not have a problem.
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.