407782 - Trunk hangs every 20 seconds due to expiration, with a real big history

Status: RESOLVED WORKSFORME

(Firefox :: Bookmarks & History, defect)

Description

[coth]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9b2pre) Gecko/2007121009 Minefield/3.0b2pre
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9b2pre) Gecko/2007121009 Minefield/3.0b2pre

Happens every time since trunk 20071205 after you start browsing. Keep, if all pages are closed.

Reproducible: Always

Steps to Reproduce:
1. Load FF.
2. Start browsing.

Comment 1

[coth]

Attachment: CPU usage

Comment 2

[coth]

Attachment: More of CPU and Memory usage.

core 0 - PE
core 1 - Firefox

Comment 3

[Ria Klaassen (not reading all bugmail)]

Does this also happen in Firefox's safe-mode? 

Comment 4

[coth]

Happen in safe made as well.

Comment 6

[coth]

Build in Bug 408003 is 2.0.0.11. Here it's trunk. Bug landed on 20071205 and I have feeling it relates to places.

Comment 7

[coth]

It's indeed places. Process Monitor shows extensive writing into places.sqlite every 20 seconds.

Comment 8

[coth]

Attachment: Pricess Monitor Log

Comment 9

[Ria Klaassen (not reading all bugmail)]

Does it also happen if you delete your live bookmarks?

Comment 10

[coth]

well, deleting live bookmarks takes not a one millisecond, so it's hard to tell. 

Comment 11

[coth]

but yes. deleting live bookmarks takes about same time as the hang every 20 seconds.

Comment 12

[Ria Klaassen (not reading all bugmail)]

I mean, does it also happen when you have no livemarks amongst your bookmarks?

Comment 13

[coth]

oh well) yes, happens after deleting all live bookmarks as well (with following restarting with new session)

Comment 14

[Marco Bonardo [:mak]]

i fear this is still an expire items problem... 

how big is your places.sqlite?

Comment 15

[Marco Bonardo [:mak]]

the biggest change in those build should be Bug 402880 "figure out pref UI for history expiration / limiting visits"

Comment 16

[Jo Hermans]

(In reply to comment #14)
> i fear this is still an expire items problem... 

That might be it, SUBSEQUENT_EXPIRATION_TIMEOUT is indeed 20 seconds. There might be a huge backlog of history entries that are now being removed. That should stop when it's done.

Note : apparently, MAX_SEQUENTIAL_RUNS isn't used anymore.

Comment 17

[Dietrich Ayala (:dietrich)]

(In reply to comment #16)
> (In reply to comment #14)
> > i fear this is still an expire items problem... 
> 
> That might be it, SUBSEQUENT_EXPIRATION_TIMEOUT is indeed 20 seconds. There
> might be a huge backlog of history entries that are now being removed. That
> should stop when it's done.
> 
> Note : apparently, MAX_SEQUENTIAL_RUNS isn't used anymore.
> 

Yeah, MAX_SEQUENTIAL_RUNS was an attempt to use a steady-state approach to history expiration per browser run. However, because there are various deterministic limits in the system (# expired per expiration run, etc), it failed horribly and caused huge backlogs (like the one it sounds like you're currently experiencing).

We don't delete much per expiration run, but given a large enough query, just finding what needs to be expired might be what's causing the freeze.

Comment 18

[Rocky]

To answer some questions that have been asked...

Yes, it does the same thing in "Safe Mode"

Also, I've tried clearing all bookmarks... and all history... EVERYTHING and it still does it.

Comment 19

[Jo Hermans]

If it's caused by a very large number of data in places database (even after clearing the history), then maybe bug 407286 might help. That is, if those changes really speed up the queries. But you say that you cleared the bookmarks too ? Then nothing should be left behind I think.

Note that the default pagesize for places.sqlite was changed from 1K to 4K a while ago (October I think). But the new size would only be used by new profiles. Could this affect the performance ?

Comment 20

[Marco Bonardo [:mak]]

pagesize could affect performance on Windows, but not too much (not SO much), and yes if the DB is clean and empty queries should not slowdown at all... 

Comment 21

[Rocky]

Yes, I cleared the bookmarks and everything.

Comment 22

[coth]

I wonder how did my history changed from 9999 to 90 days... I have now just 380 thousand entries in history out of 600 thousands I had..... Lost a lot of history! No wonder I have started noticing address bar does not find many sites now...

Changed back to normal range in 9999 and number of sites from 40000 (which is actually 40000, not 400000) to 1mln and the problem is gone.

Anyway, what is the purpose of so high demand to delete user's history if not performing Vacuum time to time?

Comment 23

[Marco Bonardo [:mak]]

9999... here is your problem of hang every 20 seconds...

the reason of expiring history at those rates is for performance problem, with the values you have setup, i think that soon you will start to have performance problems, default is  mantain history for no more than 180 days and no more than 40000 entries, this values will always guarantee good performance, higher values could be slow (could, really there are still no real numbers on the highest limit reachable, also because it depends on the hardware)...

Comment 24

[coth]

Yeah, I've got it already. But the problem is not gone. SQLite has lack of compression and if not doing vacuum time to time it will only keep growing, with no matter how old your history. So even people who uses 180 days will have problems in a non distant future. I had performance problems all the time with Firefox, simply because my history is over 2 years old already. But prior to the version 3 Firefox was not able to hold even middle long history. SQLite doing it a bit better, but still far away from to be perfect. At least I can use address bar now, not immediately after cold start, but at least after some time.

I have done vacuum on places.sqlite and now it is just a bit over 200mb.

Comment 25

[Rocky]

I've only been using Firefox for a little over a week... and I've got the problem.

Comment 26

[Jo Hermans]

(In reply to comment #25)
> I've only been using Firefox for a little over a week... and I've got the
> problem.
> 

Rocky, you filed bug 408003 in Firefox 2.0.0.11, which doesn't use the "places" database, but regular textfiles. Sorry, but this discussion here about expiration doesn't to Firefox 2.0, but to Firefox 3.0. And it's probably related to a very huge history file (more than 2 years old). Sorry.

Comment 27

[Jo Hermans]

coth: vacuuming is at the moment disabled, because it was very slow (much slower than your problem anyway). Clearing you history should help, but that's probably not what you would like to do. Bug 407286 is just checked in, and will be in tomorrows build. It might help a lot. A smaller database (the largest one which was used for testing) now takes 60 msec instead of 669 msec in your build (and several seconds earlier this month), so you might expect an improvement.

Comment 28

[Marco Bonardo [:mak]]

vacuum is not really useful with default values because of freelists that speed up inserts, it is useful if you change history limits, put them high, then after filling the db you change your mind and put them lower, so you'll end up with big unused space... 
hwv i think vacuum will be enabled in future with some kind of progress bar and user interaction

Comment 29

[Jo Hermans]

coth: since you are using a 200 MB places.sqlite file, can you tell us what amount of memory you are using when you start up, and then wait until the 20 seconds timer is starting to show (after you went to 1 website I think). And how much RAM do you have ?

I'm asking this in view of bug 398333. Your places.sqlite file should be much larger than the database cache, which is one of the reasons that the timer is slow.

Comment 30

[coth]

At cold start trunk loads and hungs until it will take about 40mb of RAM. Then after I trying to visit a first site it hangs for a few minutes until memory usage slowly (500kb per second, but after some moment ~130-140mb don't remember now it become faster) reaching 180mb. Today's peak - 291mb. But before dec 14 it was reaching 180mb very quickly just in few seconds.

Comment 31

[Dietrich Ayala (:dietrich)]

reporter, can you try a nightly build? bug 480211 moved expiration to occur every 2 mins *and* on a background thread so as to not block UI.

Comment 32

[Dietrich Ayala (:dietrich)]

marking WFM. i have 1000 days of history, don't see this anymore.

Comment 33

[Gervase Markham [:gerv]]

Bug 451915 - move Firefox/Places bugs to Firefox/Bookmarks and History. Remove all bugspam from this move by filtering for the string "places-to-b-and-h".

In Thunderbird 3.0b, you do that as follows:
Tools | Message Filters
Make sure the correct account is selected. Click "New"
Conditions: Body   contains   places-to-b-and-h
Change the action to "Delete Message".
Select "Manually Run" from the dropdown at the top.
Click OK.

Select the filter in the list, make sure "Inbox" is selected at the bottom, and click "Run Now". This should delete all the bugspam. You can then delete the filter.

Gerv