Closed Bug 397993 Opened 16 years ago Closed 15 years ago

Trunk hang after every cold start (history)

Categories

(Firefox :: Bookmarks & History, defect)

Product:
Firefox
Component:
Bookmarks & History
Platform:
x86
Windows XP
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

()

Status:
RESOLVED WORKSFORME

People

(Reporter: ria.klaassen, Assigned: dietrich)

References

Details

(Keywords: hang, perf, regression)

Attachments

(1 file)

Screenshot of the problem
168.98 KB, image/png
Details 
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a9pre) Gecko/2007092900 Minefield/3.0a9pre

Steps to reproduce:

- Shut down your computer and start it again
- Start latest Firefox trunk build (also reproducible in safe-mode)
- As soon as you see the bookmarks on your bookmarks toolbar, click on one of them (I middle clicked on GMail)

Result: hourglass and message in the title bar that Minefield doesn't react. It can take up to 30 seconds before the link starts loading.
The next time I start the same or another build this doesn't happen; I need to shut down the computer first.

Processor: AMD Athlon XP 2000+
518 MB RAM memory

I haven't tried to reproduce this yet on Vista.

Regression range: http://bonsai.mozilla.org/cvsquery.cgi?module=PhoenixTinderbox&date=explicit&mindate=1187732880&maxdate=1187733719
so I think it is probably caused by bug 386711.
Blocks: 380307
Keywords: hang, perf
Flags: blocking-firefox3?
Blocking, but it looks like this will be (was?) fixed by bug 386711?
Flags: blocking-firefox3? → blocking-firefox3+
Target Milestone: --- → Firefox 3 M9
Assignee: nobody → dietrich
(In reply to comment #2)
> Blocking, but it looks like this will be (was?) fixed by bug 386711?
> 
Re-checked: the bug was caused (not fixed) by bug 386711.

Since a few days (this range: http://bonsai.mozilla.org/cvsquery.cgi?module=PhoenixTinderbox&date=explicit&mindate=1192227180&maxdate=1192235339 ) this bug looks uglier. Minefield still takes 48 seconds to load (that is still the same) from the moment that I confirm the dialog "Continue in safe-mode" until the page appears, but for 40 seconds I only see a titlebar on my desktop and the chrome is missing. This looks a bit frightening. When the startpage finally loads the links are clickable. 
So the titlebar appears earlier and the chrome and website even later, but when they finally appear Firefox doesn't hang anymore.
I cannot reproduce this, on Mac or WinXP. Maybe something in your places.sqlite file? Can you provide it, either on the bug, or to me personally?
(In reply to comment #4)
> Minefield still takes 48 seconds to load (that is
> still the same) from the moment that I confirm the dialog "Continue in
> safe-mode" until the page appears, but for 40 seconds I only see a titlebar on
> my desktop and the chrome is missing. This looks a bit frightening. When the
> startpage finally loads the links are clickable. 
> So the titlebar appears earlier and the chrome and website even later, but when
> they finally appear Firefox doesn't hang anymore.
> 

this sounds different than the STR in your first comment.

does this hang occur at every startup post-reboot, regardless of the bookmark click in your original STR?
Yes, since I first reported it. Before a few days ago I saw links on my homepage but when I hovered over them the cursor didn't react. When I clicked I saw an hourglass.
Now it just takes longer before the page is displayed but once it is displayed I can use the site normally.
You can't reproduce the hang? I'll see if I have the opportunity and time to test this with my files on another computer. I once made a places.sqlite of 9,4 MB for testing but for some reason it doesn't show the bug as clear as with my normal places.sqlite file of 12,3 MB. 
I'll see what I can do.
I'm using a places.sqlite of 30mb, with 3+ months of history, and 1000+ bookmarks, and don't see any hang. It may be something particular to that places.sqlite file (especially considering your larger places.sqlite file isn't as hangy).
Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9a9pre) Gecko/2007101504 Minefield/3.0a9pre

On a faster system (exactly the same profile and circumstances) I can't reproduce the problem. So possibly the type of processor can't get along with the changes in Firefox?

Before bug 386711: cold start 9 seconds.
After bug 386711: cold start 9 seconds.

(In reply to comment #9)
> Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9a9pre) Gecko/2007101504
> Minefield/3.0a9pre
> 
> On a faster system (exactly the same profile and circumstances) I can't
> reproduce the problem. So possibly the type of processor can't get along with
> the changes in Firefox?
> 
> Before bug 386711: cold start 9 seconds.
> After bug 386711: cold start 9 seconds.
> 

Great, can i mark this fixed? ;)
I don't really know how to approach this without better STR, or a copy of the places.sqlite file.

The expiration calls that were modified in the patch to bug 386711 are called after a new URL is added to history (this is currently what triggers incremental expiration). The incremental expiration caps the number of history records it expires at 6, so that problems like this do not happen.

Assuming you've been to gmail before, it seems like your action shouldn't trigger expiration.
I did the test once more (although I know it doesn't help much). Startup time counted with stopwatch until the first possible action after the hang:

20070821_1448 build: 1 minute 6 seconds
20070821_1502 build: 1 minute 46 seconds
20071015_2238 build: 1 minute 26 seconds (so there is some improvement)

Maybe one of the other testers can help further. Or one of the voters :).
Keywords: qawanted
Could it have a similar cause as bug 399645? 
(In reply to comment #14)
> Could it have a similar cause as bug 399645? 
> 

I don't see a connection. Why do you think they might be related?
this doesn't need to block M9, as it hasn't been experienced on more than a single machine, we'll see how the feedback is from beta
Target Milestone: Firefox 3 M9 → Firefox 3 M10
also works for me on Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9a9pre) Gecko/2007110605 Minefield/3.0a9pre and on Linux and Mac.

Also debug trunk builds are working fine. Will monitor the Beta Feedback we get for this bug.
Target Milestone: Firefox 3 M10 → Firefox 3 Mx
Target Milestone: Firefox 3 Mx → Firefox 3 M11
This is still happening: 1 minute 26 seconds cold start.
Priority: -- → P1
Until we get more details and can actually reproduce somehow, deprioritizing.
Priority: P1 → P5
Target Milestone: Firefox 3 M11 → Firefox 3 Mx
I'm still having this issue,what could be useful to provide more details ?
Ria / Yani, can you share your places.sqlite with us?

Another thing that might help is to see the storage log.

Try setting the following environment variables

NSPR_LOG_MODULES=mozStorage:5
NSPR_LOG_FILE=log.txt

For more on how to set these variables, see http://www.mozilla.org/quality/mailnews/tests/sea_mn_imap_interop.html#IMAP%20Log%20File
unblocking until we get real STR
Flags: blocking-firefox3+
Priority: P5 → --
Target Milestone: Firefox 3 Mx → ---
Ria, can you still reproduce this?  At this point, I think qawanted is on you.
Yes, still fully reproducible with a new places.sqlite of 10 MB (only the same bookmarks but a new history).
It is bound to one particular system.

NSPR_LOG_MODULES=mozStorage:5
NSPR_LOG_FILE=log.txt

This gives an empty log.txt file.
I found a similar problem on a one year older system, only not as bad. This one had no problem to go to a link after a cold start, but it had very long delay (big hourglass for 15 seconds) to show the location bar autocomplete dropdown. It did the cold startup within 60 seconds.
Both systems have a AMD processor.


Here is some additional info that I am seeing with this bug that may be helpful, and a *workaround* that worked for me at the bottom:

o This happens on my IBM Thinkpad T43, 1.86GHz notebook, 2GB DRAM running Win XP, but doesn't appear to happen(or is very short)to on my Athlon64 3200+ PC, 2GB DRAM, running 64-bit version of Win XP.

o When this happens, it is accompanied by intense disk access for almost the entire duration of the hang.

o When it happens, as noted in the above posts, it happens after a boot, or if I shut down Firefox/Minefield for a while and do other stuff, and then come back and fire up Firefox again.  I assume that is because whatever Firefox cached the first time around needs to be re-cached if it is shutdown too long.

o The buttons are clickable for maybe 1-2 seconds after Firefox comes up before it hangs.

o This also happens on the portable version of Firefox 3, though for a shorter time.

o I first noticed this after adding my Firefox 2 bookmarks, so I decided to do some testing.  I believe I came up with a work around that fixes the problem:

***  workaround ***
Since I believed that this bug was bookmark related I did the following:
1. Backed up my bookmarks.  Actually, they are automatically backed up already.
2. Deleted all of my bookmarks. Then put a single bookmark, Google on my toolbar, so that I would have something to click on.
3. Rebooted, fired up Minefield and was able to click on Google with no problem.  No hang.
4. Restored my bookmarks, using Bookmarks--> Organize Bookmarks.
5. Rebooted and fired up Minefield again. No hang.  Everything works fine.

I also repeated the above procedure with Firefox portable, and it seems fixed as well.

Hope this helps.

And Stan, did you delete your history?
I did delete my history on Firefox 3 Portable, since I have it set up to do that automatically, but did not delete my history with Minefield.
It appears as if I spoke too soon.  Minefield is once again freezing on startup, though not for nearly as long.  Maybe 10 seconds now.  I'm wondering if there is some sort of bookmark cache that is getting corrupted?  Sorry, I don't know the inner workings of Minefield, so it is difficult for me to make an educated guess.  I'll keep this bug posted as to whether the freeze gets longer, stays the same or disappears again.  Does Minefield pre-cache it's bookmarks when it starts up?  I have 25+ RSS feeds on my Toolbar, so is Minefield trying to pre-cache just those feeds?  Or is something else going on?  Let me know if I can help in any way.
I tried comment 27 out, but deleting all bookmarks made only 5 seconds difference. It still took 51 seconds to start up. What makes a lot of difference is deleting all history, but that price is too high.

You're right Ria.  I just deleted my history and Minefield did not hang.  Here are a couple of possibilities that I think could be happening.  First I noted that when Minefield comes up, and I have a lot of history, my hard disk gets exercised vigorously and I notice my network connection also sees a lot of activity.  With no history, I see little to none of either activity.

When I click on my history, I noticed that all the links had the URL icons (favicons?).  So scenario one is that Minefield is contacting all those sites and caching the little icons, and maybe the site info that goes with them.

Scenario two could be that Minefield is vetting all the sites in the history against a Malware Protection and/or Web Forgery Protection database.

The developer should know if ether of these scenarios have are plausable.  I did notice that my CPU is not at 100% when this happens, which leads me to believe that it's an Internet or disk limitation.

I don't understand why when I redid my bookmarks, it temporarily fixed the problem.  Maybe those favicons are being cached as well.
Whiteboard: [Related: 329534, 391836, 393497, 432706]
> bookmarks when it starts up?  I have 25+ RSS feeds on my Toolbar, so is

can you try simply removing all those feeds and see if that helps? (make a backup for sure so you don't loose them)

(In reply to comment #33)
> can you try simply removing all those feeds and see if that helps? 

And please, test it for same days without re-adding them. This bug seems to be a duplicate of Bug 329534 (or of Bug 223476)
Depends on: 223476, 329534
Keywords: qawanted
Whiteboard: [Related: 329534, 391836, 393497, 432706]
(In reply to comment #34)
> This bug seems to be a duplicate of Bug 329534 (or of Bug 223476)

or overlapping of both problems, difficult to assign to one of them

The problem started to happen after the fix of bug 386711 so it can't be a dupe of earlier reported problems I'd think.
In comment 31 I reconfirmed that this bug is only about history, and has no relation with bookmarks or livemarks.
Summary: Trunk hang after every cold start → Trunk hang after every cold start (history)
Can you backup places.sqlite and bookmarks.html and try to clear all private data except browsing history - and except cookies and passwords of course?
I tried deleting places.sqlite and bookmarks.html, both together and separately. It appears that only deleting places.sqlite made any difference, i.e., Minefield did not hang after deleting (actually renaming) the file.  I also tried this with Firefox portable, and had the same results.
I'm running Minefield 3.1b2 and this problem seems to be fixed.  Good job!
Depends on: 464584
I don't know exactly when this changed, but the startup time is now 1 minute, the hang starts now after going to the next page. For this problem I filed Bug 464584.
As I don't know what bug changed the behaviour, resolving Worksforme.
Status: NEW → RESOLVED
Closed: 15 years ago
Resolution: --- → WORKSFORME
No longer depends on: 464584
FYI, I think the problem is totally related to bookmarks, and not history.  I tried deleting my history and it didn't speed up anything.  Only removing my many many bookmarks speeded things up.  Anyway, as I mentioned before, the 3.1 beta does not have this problem.  It smokes!  I only use it now.
Well the problem seemed fixed in the 3.1 beta(Shiretoko).  It was at first, but in the last 3 weeks or so, 3.1 has gotten progressively (I update daily) worse to the point where I think it is now worse than 3.0.  This is very discouraging to me.  Plus now, when I click on a Toolbar bookmark folder, the drop down list only stays open briefley before closing.  Almost no time to click on a link.  Maybe this is a new bug.
Stan, is that in Safe Mode?
(In reply to comment #43)

> Plus now, when I click on a Toolbar bookmark folder, the drop down list
> only stays open briefley before closing.  Almost no time to click on a link. 
> Maybe this is a new bug.

This could be related to bug 473058.
No, not is Safe Mode.
Bug 451915 - move Firefox/Places bugs to Firefox/Bookmarks and History. Remove all bugspam from this move by filtering for the string "places-to-b-and-h".

In Thunderbird 3.0b, you do that as follows:
Tools | Message Filters
Make sure the correct account is selected. Click "New"
Conditions: Body   contains   places-to-b-and-h
Change the action to "Delete Message".
Select "Manually Run" from the dropdown at the top.
Click OK.

Select the filter in the list, make sure "Inbox" is selected at the bottom, and click "Run Now". This should delete all the bugspam. You can then delete the filter.

Gerv
Component: Places → Bookmarks & History
QA Contact: places → bookmarks
You need to log in before you can comment on or make changes to this bug.