555292 - Minefield hangs on twitter.com

Status: RESOLVED WORKSFORME

(Toolkit :: Places, defect)

Description

[Robert [minotaur 11]]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a4pre) Gecko/20100326 Minefield/3.7a4pre
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a4pre) Gecko/20100326 Minefield/3.7a4pre

Minefield hangs for about 10 seconds shortly after opening twitter.com.

Reproducible: Always

Steps to Reproduce:
1. Open http://twitter.com/



I could reproduce this with a new profile only if i copy the places.sqlite file from my normal profile (about 42 MiB) into the new profile. My PC is quite old (Athlon XP 2500+), so i suppose it's less visible with modern Hardware.

Regression range:
works:
1269517894-20100325045134-a2f7186e4379-firefox-3.7a4pre.en-US.win32
hangs:
1269518787-20100325050627-7119a973030d-firefox-3.7a4pre.en-US.win32
Pushlog:
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=a2f7186e4379&tochange=7119a973030d

This seems to be a regression by bug 553489.

Comment 1

[Marco Bonardo [:mak]]

hm, this is pretty strange. well i have a larger DB, but also a faster CPU.

Also the change showed a 10% reduction in cpu usage during tp4...
Will try reproducing.

Comment 2

[Robert [minotaur 11]]

Attachment: places.sqlite test file

I modified my normal places.sqlite file to reduce file size and privacy implications, with this test file minefield also hangs for about 10 seconds on twitter.com.

Comment 3

[Marco Bonardo [:mak]]

hm, how did you reduce the size of the db? there are lot of orphans here, i suppose you removed urls manually from moz_places?
Btw, trying with this profile, i can't reproduce :(

Comment 4

[Robert [minotaur 11]]

That's odd, i tried it on a different PC and it was reproducible there with the test file. I know that the file isn't valid anymore, i thought it wouldn't matter if it's reproducible. To make the test file i used:

DELETE FROM moz_favicons
DELETE FROM moz_inputhistory
DELETE FROM moz_keywords
DELETE FROM moz_items_annos
DELETE FROM moz_annos
UPDATE moz_bookmarks SET title = 'abc' WHERE title IS NOT NULL
DELETE FROM moz_places WHERE url NOT LIKE '%twitter.com%' OR visit_count <= 100
DELETE FROM moz_historyvisits WHERE place_id NOT IN (SELECT id FROM moz_places)

Comment 5

[Robert [minotaur 11]]

Attachment: places.sqlite test file v2

Comment 6

[Robert [minotaur 11]]

I've done some further testing to find reliable STRs.

The second test file is made from my normal places db without manual operations, so it shouldn't contain any orphans. I deleted all history entries (except 3 twitter pages) and all bookmarks through the library. Afterward i set places.history.expiration.interval_seconds to 1, waited until Firefox finished expiring and imported a test bookmarks.html file. I found no differences in the behavior between my normal db, test 1 and test 2.

Builds tested:
a2f7186e4379 and 7119a973030d

Desktop, Amd Athlon XP 2500+
Windows XP SP3
379: works, 30d: hangs
Linux Ubuntu 8.04
379: works, 30d: hangs

Notebook, Intel Pentium 4 Mobile 3,06GHz
Windows XP SP3
379: works, 30d: hangs

Notebook, Intel Core 2 Duo P8400
Windows 7
379: works, 30d: hangs

Comment 7

[Marco Bonardo [:mak]]

i'll try again, but i really can't see where this hang could come from. And it's crazy you can reproduce on 3 boxes while i can't anywhere :\
Will try to reproduce in a VM

Comment 8

[Marco Bonardo [:mak]]

oh, with your second file i've been able to reproduce in a VM, i also registered a recording that i can replay and debug with vmware.
while this happened my computer was pretty busy doing an heavy disk operation (Actually rebuilding iTunes library), but sounds like it took a while more then expected.
Will try to debug it.

Comment 9

[Marco Bonardo [:mak]]

I've not yet found the time to go deeper, at first glance this can't be slow than before, there is still some part of favicon loading that should be made async.

We recently fixed a bug involving a lock contention on visiting pages, is this still reproduceable in current trunk?

Comment 10

[Robert [minotaur 11]]

Yes, i still see the bug.

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a5pre) Gecko/20100505 Minefield/3.7a5pre

test file v2 and my normal places.sqlite file

Comment 11

[Robert [minotaur 11]]

A short update: Since bug 499828 the hang doesn't occur anymore shortly after loading the site, but while loading.

Comment 12

[Marco Bonardo [:mak]]

I think current work on places branch could be fixing this.
You could try a build, get the most recent "places" build for your platform: ftp://ftp.mozilla.org/pub/firefox/tinderbox-builds/

Comment 13

[Marco Bonardo [:mak]]

note that it's better if you use a separate test profile for that, this is experimental.

Comment 14

[Robert [minotaur 11]]

I've compared a current nightly (837d7b346a64) and a places build (080549b4c0f8) on Windows XP with the test file v2 and my normal places file. In all cases i get nearly 100% cpu for 15 to 30 seconds (depending on file and build) while loading twitter. While the nightly is always completely frozen during this time, the places build frequently stays responsive, only a bit laggy. It's then possible to switch tabs, the throbber animation is working and i see the rendering and the animation of the twitter page. Opening the bookmarks or history menu during this time sometimes freezes the places build completely until the cpu usage drops.

Comment 15

[Robert [minotaur 11]]

As Firefox 4 is near and the bug (as described in comment 14 for the places build) is still present for me, i would like to ask if there is a bug who could maybe fix this. If additional information is needed, i would love to help.

Comment 16

[Marco Bonardo [:mak]]

I have never been able to exactly reproduce what you see, there are various issues with twitter, some of which due to hardware acceleration, but from your descriptions looks like we have a query lock contention, most of the stuff that happens on page load today is completely async, opening the menu while this stuff runs could cause contention, but I don't see why twitter would make a difference.
I was able to reproduce other issues that are currently fixed (comment 12) on trunk.
Based on being unable to reproduce, I highly doubt something from current nighrly to final could make a serious difference.

Comment 17

[George Carstoiu]

 Mozilla/5.0 (X11; Linux i686; rv:8.0a1) Gecko/20110809 Firefox/8.0a1

I am unable to reproduce the freeze Robert has reported.

Robert can you please try to reproduce your issue on the latest trunk?

Thanks!

Comment 18

[Robert [minotaur 11]]

I can still reproduce it with a current trunk nightly (20110808 NIGHTLY, f414db34c70b) and the test file v2. But i made a new places.sqlite file with Firefox 5 and only imported my bookmarks, this time the hangs did not reoccur with a growing number of twitter visits. With that places file i also don't get hangs in the nightly. So it's not a problem for me personally anymore.

Comment 19

[B.J. Herbison]

I created a new profile and replaced places.sqlite with the attachment.

Visiting twitter.com caused Firefox 9 to consume 7-8 seconds of CPU time, but it didn't appear to be hung. The page responded to mouse movement and I was able to open other tabs, load other pages, and follow links from the twitter home page.

Tested using the latest Nightly build (Firefox 9).

Comment 20

[Robert [minotaur 11]]

(In reply to B.J. Herbison from comment #19)

Thank you for testing, did you try to open the history or bookmarks menu during the time with high cpu?

Comment 21

[B.J. Herbison]

(In reply to Robert [minotaur 11] from comment #20)
> (In reply to B.J. Herbison from comment #19)
> 
> Thank you for testing, did you try to open the history or bookmarks menu
> during the time with high cpu?

Some menus did respond, but I don't remember which I tried. Typing in the address bar displayed URL suggestions so the history system was not totally blocked. I'll retest with the same profile on Tuesday and report if I find those menus blocked.

Comment 22

[Marco Bonardo [:mak]]

How is FF behaving in current version?

Comment 23

[Gabriela [:gaby2300]]

Resolving WORKSFORME as I've been unable to reproduce this with the latest Nightly and Windows 7. Please reopen with more information if you can still reproduce,