417262 - History sidebar slow for "By Most Visited" and "By Last Visited"

Status: RESOLVED WORKSFORME

(Firefox :: Bookmarks & History, defect, P2)

Description

[Ondrej Brablc]

Bug 385245 complained about slow history sidebar. The latest patch in this bug solves performance problem of "By Site", "By Date" and "By Date and Site". However, it does not improve "By Most Visited" and "By Last Visited". As the number of days in history has increased from 9 to 180, the amount of data increased too.

While we could probably get some speed improvements in the javascript code to slightly improve the performance, with many sites it would be still slow. A solution would be to introduce some kind of grouping.

By Last Visited
---------------
We should group max visits in the same way as "By Date" groups all visits. However, instead of sorting by title inside of the date range, we would sort by last visit descending.

By Most Visited
---------------
We should group by the number of visits. I suggest something like this:

Top Visited Pages 
  - p(v) > 99th percentile
Most Visited Pages 
  - 99th percentile >= p(v) > 95th percentile
Commonly Visited Pages
  - 95th percentile >= p(v) > 50th percentile
Rarely Visited Pages
  - p(v) <= 50th percentile
Never Visited Pages
  - this could be imported but never visited bookmarks

Where "v" is number of visits. "p(v)" is the percentile - what is the percentage of places having "v" number of visits and less. Most likely users would only click on "Top Visited Pages" what would load quickly as there is only 1% of places.

This grouping will only be shown if you have more than 100 places.

Both can be implemented very easily on top of bug 385245. No changes on UI are required (apart of the new strings).

Comment 1

[Mike Beltzner [:beltzner, not reading bugmail]]

My instinct is to not block on this, since I believe that computers will only be getting faster and faster, but it's definitely wanted.

Dietrich: if you think I've made a bad call here, let me know.

Comment 2

[Ondrej Brablc]

(In reply to comment #1)
> My instinct is to not block on this, since I believe that computers will only
> be getting faster and faster, but it's definitely wanted.
> 
> Dietrich: if you think I've made a bad call here, let me know.
> 

With my biggest profile (80MB of places and their visits) and my release optimized Firefox it takes about 8 seconds to display history sorted "By Most Visited" or "By Last Visited". Waiting 8 seconds to see UI reaction on such a simple action (while having 100% CPU load at the same time) is not a tragedy, but definitely would not impress anyone. The alternative suggested in this bug is to have times less than 1 second without a visible delay and added functionality.

Anyway, I agree that it is not blocking as of now, it may get more important with the time, as people would get their database filled. So I would expect, that people start complaining couple of months after the release.

Comment 3

[Ria Klaassen (not reading all bugmail)]

A possible workaround would be to display only the first hundred, with a button on the bottom -> Next 100 items etc. Or/and an about:config option how many items the user wants to display.

Comment 4

[Ondrej Brablc]

(In reply to comment #3)
> A possible workaround would be to display only the first hundred, with a button
> on the bottom -> Next 100 items etc. Or/and an about:config option how many
> items the user wants to display.
> 

Yes, but this introduces new UI elements on place where we can utilize existing. Moreover, it really makes sense to separate visits somehow, flat list is not much helpful, especially when it does not display the information about the number of visits.

Comment 5

[Ria Klaassen (not reading all bugmail)]

Or a division:

By Last Visited:
Today
Yesterday
2 days ago
3 days ago
etc. (or just the date)

Then it would be much easier to find a visited site if you have no memory of name or domain.

Comment 6

[Marco Bonardo [:mak] (Away Apr 25 - May 5)]

imho split last visited into folders does not appear like a good choice, if you want to do that we can throw away "by last visited" and use directly "by date"... i'd prefer having the view limited. don't have data on general usage but personally i use the last visited sidebar when the results i'm looking for does not fit into the history menu, so i'd prefer a limited list rather than a grouped one. 

For most visited we could exclude items with visit_count = 0 (when visit_count will be reliable, see Bug 416313). That should reduce the number of results.

Still, both methods will not return ALL history entries, then we should somehow aware the user of that, maybe sending him to the history in the library (that we could then group "by date")

Comment 7

[Marco Bonardo [:mak] (Away Apr 25 - May 5)]

sorry i wanted to say "limit on visit_count" but i said a stupid thing, probably having some grouping on number of visits would be fine

Comment 8

[Ria Klaassen (not reading all bugmail)]

Maybe a tree that loads and expands on hover? 
My places.sqlite is only 19,8 MB but it takes 8 seconds to open (By Last Visited). It's not the fastest laptop in the world but I hope to use it for some more years :).

Comment 9

[Ryan A. C.]

Much like I suggest in bug 431273, why not use whatever fancy code the download manager is using to automagically expand the list whenever you get near the bottom?

Comment 10

[olistrut]

I was hoping this would be faster with FF 3.1, but I just installed beta 2 and displaying the history is still quite slow. With a 25MB places file, it takes 5-10 seconds for first time load and 3-5 seconds thereafter. This is on a dual core laptop with a lot of memory - not the fast computer, but still mainstream performance.

In addition to the delay in displaying the history, the browser UI blocks while loading the history, which makes this really annoying.

Comment 11

[Brian Carpenter [:geeknik]]

My places.sqlite is 70MB. I have a 3ghz Core2Quad CPU and 4GB of RAM and when I go to open the History Sidebar by pressing Ctrl-H, it takes 8-10 seconds and (Not Responding) shows up in the Firefox 3.5b4pre titlebar because Vista thinks the browser is hung. When I change the view to By Last Visited, it takes 7-8 seconds to update the view and once again, I see the (Not Responding) in the titlebar. Same with changing the view to By Most Visited. Even after everything is loaded, Firefox 3.5b4pre acts a little laggy for 30-45 seconds afterwards. Very annoying. 

Build ID 20090413044519.

Comment 14

[Gervase Markham [:gerv]]

Bug 451915 - move Firefox/Places bugs to Firefox/Bookmarks and History. Remove all bugspam from this move by filtering for the string "places-to-b-and-h".

In Thunderbird 3.0b, you do that as follows:
Tools | Message Filters
Make sure the correct account is selected. Click "New"
Conditions: Body   contains   places-to-b-and-h
Change the action to "Delete Message".
Select "Manually Run" from the dropdown at the top.
Click OK.

Select the filter in the list, make sure "Inbox" is selected at the bottom, and click "Run Now". This should delete all the bugspam. You can then delete the filter.

Gerv

Comment 15

[Marco Bonardo [:mak] (Away Apr 25 - May 5)]

afaict the situation is much better, new bugs should be fixed with profiles got through the profiler or actual improvements, though we don't need anymore this tracking bug.