Closed Bug 671558 Opened 13 years ago Closed 11 years ago

Firefox hangs because of SQLite access

Categories

(Toolkit :: Places, defect)

5 Branch
All
Windows 7
defect
Not set
normal

Tracking

()

RESOLVED INCOMPLETE

People

(Reporter: magicandre1981, Unassigned)

Details

Attachments

(2 files)

Attached file Firefox hang dump
User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0
Build ID: 20110615151330

Steps to reproduce:

Browsing the WWW with Firefox 5. I also tried to compact the SQLite DBs, but it doesn't improve the speed.

about:support:

  Allgemeine Informationen

        Name
        Firefox

        Version
        5.0

        User-Agent
        Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0

        Profilordner

          Beinhaltenden Ordner anzeigen

        Aktivierte Plugins

          about:plugins

        Build-Konfiguration

          about:buildconfig

  Erweiterungen

        Name

        Version

        Aktiviert

        ID

        Adblock Plus
        1.3.9
        true
        {d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d}

        Add-on Compatibility Reporter
        0.8.5
        true
        compatibility@addons.mozilla.org

        Anti-Aliasing Tuner
        11.06.06.01
        true
        aatuner@hotmint.com

        Auto Copy
        1.0.1
        true
        {0FED7D55-65D4-47b6-A6DE-9A4ADB55355F}

        Bookmark Duplicate Detector
        1.0.5
        true
        {ba243cb0-b824-4a26-9418-73ee795d9b9d}

        CacheViewer Fx5
        0.8
        true
        {81328583-3CA7-4809-B4BA-570A85818FBB}

        CheckPlaces
        2.6.0
        true
        checkplaces@andyhalford.com

        Dictionary Switcher
        1.3.1
        true
        dictionary-switcher@design-noir.de

        Extended Statusbar
        1.5.8
        true
        {daf44bf7-a45e-4450-979c-91cf07434c3d}

        external IP
        0.9.9.6
        true
        externalip@erik.morlin

        Firefox 4 UI Fixer
        1.4.3
        true
        ff4uifix@nikolakocic.com

        Firefox Sync
        1.7
        true
        {340c2bbc-ce74-4362-90b5-7c26312808ef}

        Flagfox
        4.1.4
        true
        {1018e4d6-728f-4b20-ad56-37578a4de76b}

        FlashGot
        1.3.0.3
        true
        {19503e42-ca3c-4c27-b1e2-9cdb2170ee34}

        Free Download Manager plugin
        1.5.4
        true
        fdm_ffext@freedownloadmanager.org

        Greasemonkey
        0.9.6
        true
        {e4a8a97b-f2ed-450b-b12d-ee082ba24781}

        gui:config
        1.1.4
        true
        guiconfig@slosd.net

        Hide My Ass Proxy Extension
        1.2.3
        true
        extension@hidemyass.com

        Java Console
        6.0.26
        true
        {CAFEEFAC-0016-0000-0026-ABCDEFFEDCBA}

        Linkification
        1.3.9
        true
        {35106bca-6c78-48c7-ac28-56df30b51d2a}

        MAFIAAFire Redirector
        0.7b
        true
        MafiaaFire@mafiaafire.com

        Memory Fox
        7.4
        true
        {E173B749-DB5B-4fd2-BA0E-94ECEA0CA55B}

        MR Tech Toolkit
        6.0.4.9000
        true
        {9669CC8F-B388-42FE-86F4-CB5E7F5A8BDC}

        Nightly Tester Tools
        3.1.5.1
        true
        {8620c15f-30dc-4dba-a131-7c5d20cf4a29}

        NoScript
        2.1.2.1rc1
        true
        {73a6fe31-595d-460b-a920-fcc0f8843232}

        NoSquint
        2.1.2
        true
        nosquint@urandom.ca

        Password Exporter
        1.2.1
        true
        {B17C1C5A-04B1-11DB-9804-B622A1EF5492}

        Redirect Remover
        2.6.4
        true
        {fe0258ab-4f74-43a1-8781-bcdf340f9ee9}

        RSS Icon
        1.0.6
        true
        kitsuneymg@gmail.com

        Status-4-Evar
        2011.06.28.22
        true
        status4evar@caligonstudios.com

        Stylish
        1.2
        true
        {46551EC9-40F0-4e47-8E18-8E5CF550CFB8}

        Tab Mix Plus
        0.3.8.6
        true
        {dc572301-7619-498c-a57d-39143191b318}

        United States English Spellchecker
        5.0.1
        true
        en-US@dictionaries.addons.mozilla.org

        Wörterbuch Deutsch (de-DE), Hunspell-unterstützt
        20110525
        true
        de_DE@dicts.j3e.de

  Modifizierte Einstellungen

      Name

      Wert

        accessibility.typeaheadfind.flashBar
        0

        browser.history_expire_days.mirror
        180

        browser.history_expire_days_min
        45

        browser.places.importBookmarksHTML
        false

        browser.places.importDefaults
        false

        browser.places.leftPaneFolderId
        -1

        browser.places.migratePostDataAnnotations
        false

        browser.places.smartBookmarksVersion
        2

        browser.places.updateRecentTagsUri
        false

        browser.startup.homepage
        about:mozilla

        browser.startup.homepage_override.buildID
        20110615151330

        browser.startup.homepage_override.mstone
        rv:5.0

        browser.tabs.animate
        false

        browser.tabs.closeWindowWithLastTab
        false

        browser.zoom.siteSpecific
        false

        extensions.checkCompatibility
        false

        extensions.checkCompatibility.3.6
        false

        extensions.checkCompatibility.3.6b
        false

        extensions.checkCompatibility.3.6p
        false

        extensions.checkCompatibility.3.6pre
        false

        extensions.checkCompatibility.3.7a
        false

        extensions.checkCompatibility.4.0
        false

        extensions.checkCompatibility.4.0.previous
        true

        extensions.checkCompatibility.4.0b
        false

        extensions.checkCompatibility.4.0p
        false

        extensions.checkCompatibility.4.0pre
        false

        extensions.checkCompatibility.4.2
        false

        extensions.checkCompatibility.4.2a
        false

        extensions.checkCompatibility.4.2b
        false

        extensions.checkCompatibility.4.2p
        false

        extensions.checkCompatibility.4.2pre
        false

        extensions.checkCompatibility.5.0
        false

        extensions.checkCompatibility.5.0a
        false

        extensions.checkCompatibility.5.0b
        false

        extensions.checkCompatibility.5.0p
        false

        extensions.checkCompatibility.5.0pre
        false

        extensions.checkCompatibility.6.0
        false

        extensions.checkCompatibility.6.0a
        false

        extensions.checkCompatibility.7.0
        false

        extensions.checkCompatibility.7.0a
        false

        extensions.checkCompatibility.8.0
        false

        extensions.checkCompatibility.8.0a
        false

        extensions.checkCompatibility.nightly
        false

        extensions.lastAppVersion
        5.0

        general.useragent.extra.microsoftdotnet
        (.NET CLR 3.5.30729)

        gfx.3d_video.enabled
        true

        gfx.direct2d.disabled
        true

        gfx.font_rendering.cleartype.always_use_for_content
        true

        gfx.use_text_smoothing_setting
        true

        keyword.URL
        http://www.google.com/search?ie=UTF-8&oe=UTF-8&q=

        layers.accelerate-all
        true

        mozilla.widget.render-mode
        6

        network.cookie.prefsMigrated
        true

        places.database.lastMaintenance
        1310558706

        places.history.expiration.transient_current_max_pages
        257674

        places.last_vacuum
        1270190442

        print.print_printer
        hp deskjet 5550 series (HPA)

        print.printer_hp_deskjet_5550_series_(HPA).print_bgcolor
        false

        print.printer_hp_deskjet_5550_series_(HPA).print_bgimages
        false

        print.printer_hp_deskjet_5550_series_(HPA).print_command

        print.printer_hp_deskjet_5550_series_(HPA).print_downloadfonts
        false

        print.printer_hp_deskjet_5550_series_(HPA).print_edge_bottom
        0

        print.printer_hp_deskjet_5550_series_(HPA).print_edge_left
        0

        print.printer_hp_deskjet_5550_series_(HPA).print_edge_right
        0

        print.printer_hp_deskjet_5550_series_(HPA).print_edge_top
        0

        print.printer_hp_deskjet_5550_series_(HPA).print_evenpages
        true

        print.printer_hp_deskjet_5550_series_(HPA).print_footercenter

        print.printer_hp_deskjet_5550_series_(HPA).print_footerleft
        &PT

        print.printer_hp_deskjet_5550_series_(HPA).print_footerright
        &D

        print.printer_hp_deskjet_5550_series_(HPA).print_headercenter

        print.printer_hp_deskjet_5550_series_(HPA).print_headerleft
        &T

        print.printer_hp_deskjet_5550_series_(HPA).print_headerright
        &U

        print.printer_hp_deskjet_5550_series_(HPA).print_in_color
        true

        print.printer_hp_deskjet_5550_series_(HPA).print_margin_bottom
        0.5

        print.printer_hp_deskjet_5550_series_(HPA).print_margin_left
        0.5

        print.printer_hp_deskjet_5550_series_(HPA).print_margin_right
        0.5

        print.printer_hp_deskjet_5550_series_(HPA).print_margin_top
        0.5

        print.printer_hp_deskjet_5550_series_(HPA).print_oddpages
        true

        print.printer_hp_deskjet_5550_series_(HPA).print_orientation
        0

        print.printer_hp_deskjet_5550_series_(HPA).print_page_delay
        50

        print.printer_hp_deskjet_5550_series_(HPA).print_paper_data
        9

        print.printer_hp_deskjet_5550_series_(HPA).print_paper_height
        11,00

        print.printer_hp_deskjet_5550_series_(HPA).print_paper_size_type
        0

        print.printer_hp_deskjet_5550_series_(HPA).print_paper_size_unit
        1

        print.printer_hp_deskjet_5550_series_(HPA).print_paper_width
        8,50

        print.printer_hp_deskjet_5550_series_(HPA).print_reversed
        false

        print.printer_hp_deskjet_5550_series_(HPA).print_scaling
        1,00

        print.printer_hp_deskjet_5550_series_(HPA).print_shrink_to_fit
        true

        print.printer_hp_deskjet_5550_series_(HPA).print_to_file
        false

        print.printer_hp_deskjet_5550_series_(HPA).print_unwriteable_margin_bottom
        0

        print.printer_hp_deskjet_5550_series_(HPA).print_unwriteable_margin_left
        0

        print.printer_hp_deskjet_5550_series_(HPA).print_unwriteable_margin_right
        0

        print.printer_hp_deskjet_5550_series_(HPA).print_unwriteable_margin_top
        0

        privacy.clearOnShutdown.cookies
        false

        privacy.clearOnShutdown.formdata
        false

        privacy.clearOnShutdown.history
        false

        privacy.clearOnShutdown.sessions
        false

        privacy.cpd.siteSettings
        true

        privacy.donottrackheader.enabled
        true

        privacy.sanitize.migrateFx3Prefs
        true

        privacy.sanitize.timeSpan
        0

        security.dialog_enable_delay
        0

        security.warn_viewing_mixed
        false

  Grafik

        Karten-Beschreibung
        ATI Radeon HD 5700 Series

        Vendor-ID
        1002

        Geräte-ID
        68b8

        Karten-Ram
        1024

        Karten-Treiber
        aticfx64 aticfx64 aticfx64 aticfx32 aticfx32 aticfx32 atiumd64 atidxx64 atidxx64 atiumdag atidxx32 atidxx32 atiumdva atiumd6a atitmm64

        Treiber-Version
        8.861.0.0

        Treiber-Datum
        5-24-2011

        Direct2D aktiviert
        false

        DirectWrite aktiviert
        false (6.1.7601.17563, font cache n/a)

        WebGL-Renderer
        Google Inc. -- ANGLE -- OpenGL ES 2.0 (ANGLE 0.0.0.611)

        GPU-beschleunigte Fenster
        1/1 Direct3D 9

    


Actual results:

Firefox often hangs and generates 100% CPU usage on 1 CPU core (AMD Phenom II X4 3GHz). I've used ProcDump to generate a hang dump. Here is the WinDbg output:

*** procdump  -h -o firefox D:\Daten\Downloads\_dumps\
*** Hung window detected: 860c82'
Dl Eid Cid     WaitType
-- --- ------- --------------------------
   0   10a0.1478 Critical Section       

WAIT_CHAIN_COMMAND:  ~0s;k;;

BLOCKING_THREAD:  00001478

DEFAULT_BUCKET_ID:  APPLICATION_HANG_BusyHang

0026d2d0 7713b4d4 ntdll!NtWaitForSingleObject+0x15
0026d334 7713b3b8 ntdll!RtlpWaitOnCriticalSection+0x13e
0026d35c 73a46f3b ntdll!RtlEnterCriticalSection+0x150
0026d364 73a46be3 mozsqlite3!winMutexEnter+0xb [e:\builds\moz2_slave\rel-rel-w32-bld\build\db\sqlite3\src\sqlite3.c @ 17445]
0026d370 6b4ef64f mozsqlite3!sqlite3_extended_result_codes+0x13 [e:\builds\moz2_slave\rel-rel-w32-bld\build\db\sqlite3\src\sqlite3.c @ 107339]
0026d390 6b51878d xul!mozilla::storage::prepareStmt+0x14 [e:\builds\moz2_slave\rel-rel-w32-bld\build\storage\src\mozstorageprivatehelpers.cpp @ 315]
0026d3b8 6b66e0a3 xul!mozilla::storage::Statement::initialize+0x5d [e:\builds\moz2_slave\rel-rel-w32-bld\build\storage\src\mozstoragestatement.cpp @ 178]
0026d3d4 6b96a8b6 xul!mozilla::storage::Connection::CreateStatement+0x4e [e:\builds\moz2_slave\rel-rel-w32-bld\build\storage\src\mozstorageconnection.cpp @ 940]
0026d514 739490e9 xul!mozilla::storage::`anonymous namespace'::Vacuumer::execute+0x27cac2
0026d52c 739499ed mozcrt19!arena_malloc_small+0x149 [e:\builds\moz2_slave\rel-rel-w32-bld\build\obj-firefox\memory\jemalloc\crtsrc\jemalloc.c @ 3789]
0026d53c 6b780245 mozcrt19!malloc+0x3d [e:\builds\moz2_slave\rel-rel-w32-bld\build\obj-firefox\memory\jemalloc\crtsrc\jemalloc.c @ 5876]
0026d578 6b728b6d xul!mozilla::storage::`anonymous namespace'::Vacuumer::Vacuumer+0x1f [e:\builds\moz2_slave\rel-rel-w32-bld\build\storage\src\vacuummanager.cpp @ 195]
0026d5ac 6b6e7b8b xul!nsIdleServiceDaily::Observe+0x68 [e:\builds\moz2_slave\rel-rel-w32-bld\build\widget\src\xpwidgets\nsidleservice.cpp @ 98]
0026d67c 6bece3eb xul!nsIdleService::CheckAwayState+0x16c [e:\builds\moz2_slave\rel-rel-w32-bld\build\widget\src\xpwidgets\nsidleservice.cpp @ 426]
0026d688 6b609710 xul!nsIdleService::IdleTimerCallback+0xc [e:\builds\moz2_slave\rel-rel-w32-bld\build\widget\src\xpwidgets\nsidleservice.cpp @ 311]
0026d6b0 6b60995c xul!nsTimerImpl::Fire+0xe0 [e:\builds\moz2_slave\rel-rel-w32-bld\build\xpcom\threads\nstimerimpl.cpp @ 424]
0026d6b8 6b584a22 xul!nsTimerEvent::Run+0x1c [e:\builds\moz2_slave\rel-rel-w32-bld\build\xpcom\threads\nstimerimpl.cpp @ 522]
0026d6dc 6b554a2e xul!nsThread::ProcessNextEvent+0x2e2 [e:\builds\moz2_slave\rel-rel-w32-bld\build\xpcom\threads\nsthread.cpp @ 618]
0026d71c 6b777c7b xul!TimerThread::RemoveTimer+0x2e [e:\builds\moz2_slave\rel-rel-w32-bld\build\xpcom\threads\timerthread.cpp @ 417]
0026d728 6b777c64 xul!MessageLoop::RunInternal+0x11 [e:\builds\moz2_slave\rel-rel-w32-bld\build\ipc\chromium\src\base\message_loop.cc @ 219]
0026d740 7395808b xul!MessageLoop::RunHandler+0x1d [e:\builds\moz2_slave\rel-rel-w32-bld\build\ipc\chromium\src\base\message_loop.cc @ 203]
0026d760 6b777c10 mozcrt19!_VEC_memzero+0x36
0026d780 6b75c12e xul!MessageLoop::Run+0x15 [e:\builds\moz2_slave\rel-rel-w32-bld\build\ipc\chromium\src\base\message_loop.cc @ 177]
0026d78c 6b77875e xul!nsBaseAppShell::Run+0x34 [e:\builds\moz2_slave\rel-rel-w32-bld\build\widget\src\xpwidgets\nsbaseappshell.cpp @ 191]
0026f6e0 6b778792 xul!nsAppShell::Run+0x42 [e:\builds\moz2_slave\rel-rel-w32-bld\build\widget\src\windows\nsappshell.cpp @ 270]
0026f6ec 6b6a9975 xul!nsAppStartup::Run+0x1e [e:\builds\moz2_slave\rel-rel-w32-bld\build\toolkit\components\startup\nsappstartup.cpp @ 219]
0026fa74 00da134c xul!XRE_main+0xdcf [e:\builds\moz2_slave\rel-rel-w32-bld\build\toolkit\xre\nsapprunner.cpp @ 3758]
0026fac0 00da16f2 firefox!wmain+0x34c [e:\builds\moz2_slave\rel-rel-w32-bld\build\toolkit\xre\nswindowswmain.cpp @ 128]
0026fb00 7673337a firefox!__tmainCRTStartup+0x152 [e:\builds\moz2_slave\rel-rel-w32-bld\build\obj-firefox\memory\jemalloc\crtsrc\crtexe.c @ 591]
0026fb0c 7713bf52 kernel32!BaseThreadInitThunk+0xe
0026fb4c 7713bf25 ntdll!__RtlUserThreadStart+0x70
0026fb64 00000000 ntdll!_RtlUserThreadStart+0x1b


Expected results:

Browsing without a hang of the UI.
FOLLOWUP_IP: 
mozsqlite3!winMutexEnter+b [e:\builds\moz2_slave\rel-rel-w32-bld\build\db\sqlite3\src\sqlite3.c @ 17445]
73a46f3b c3              ret

SYMBOL_STACK_INDEX:  3

SYMBOL_NAME:  mozsqlite3!winMutexEnter+b

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: mozsqlite3

IMAGE_NAME:  mozsqlite3.dll

DEBUG_FLR_IMAGE_TIMESTAMP:  4df94446

STACK_COMMAND:  ~0s ; kb

BUCKET_ID:  HANG_mozsqlite3!winMutexEnter+b

FAILURE_BUCKET_ID:  APPLICATION_HANG_BusyHang_cfffffff_mozsqlite3.dll!winMutexEnter
OS: Other → Windows 7
It looks like the places DB is being vacuumed, and nsIdleService::CheckAwayState is involved, which makes it sound like Firefox thinks it's idle.  But André mentioned this problem in a comment on my blog, and added "I see this action during normal usage with Firefox when I browse the web. The Firefox is not idle at this time."

Looking at comment 0... wow, that's a lot of add-ons installed.  I wonder if any of them, or a combination of them, is part of the problem.
I use a lot of them for years (AutoCopy, No redirect,  Linkification, NoScript, ABP and I can't live without them, that's impossible for me to use a browser without them)and I have to add new ones because of the reworked UI in Firefox 4 to get the statusbar and the RSS Feedicon back. I removed some addons and check it again later this day. 

I think the vacuum comes from this:

https://bugzilla.mozilla.org/show_bug.cgi?id=512854

Is there a way to temporary disable the vacuum when Firefox thinks it's idle and only run it on a specified time?
Component: General → Places
Product: Firefox → Toolkit
QA Contact: general → places
Looks more like a bug in the idle service code, if it thinks it's idle when it's not. Vacuum is done once a month, after 5 minutes of idle. if you see it more often, it's most likely a problem in the profile, or a blocker bug.

You can disable vacuum by setting storage.vacuum.last.places.sqlite in the future (its value is number of seconds from the epoch) but I don't think it's needed here. Which value does it have now?
Could you please uninstall CheckPlaces immediately? I highly suspect it.
Also, why do you have Firefox Sync installed? it's integrated in Firefox itself now.
Hi Marco,

this is my profile I'm using for years and so I upgraded from 3.6 to Firefox4 and so I had Sync included. I removed it. Thanks for this.

the value storage.vacuum.last.places.sqlite is 1310848838.

my places.sqlite is 94MB, the urlclassifier3.sqlite is 16MB. 

The performance issue is the function

mozsqlite3.dll!sqlite3VdbeExec

I can see this from the xperf summary table.

I get the high CPU usage when opening new tabs and type in the address fragments and select them from the address bar. 

ntoskrnl.exe!KiSwapContext+0x7a
ntoskrnl.exe!KiCommitThreadWait+0x1d2
ntoskrnl.exe!KeWaitForSingleObject+0x19f
ntoskrnl.exe!KiSuspendThread+0x54
ntoskrnl.exe!KiDeliverApc+0x201
ntoskrnl.exe!KiApcInterrupt+0xd7
mozsqlite3.dll!sqlite3VdbeExec+0xcdc
mozsqlite3.dll!sqlite3_step+0xbc
xul.dll!mozilla::storage::stepStmt+0x26
xul.dll!mozilla::storage::Statement::ExecuteStep+0xaf
xul.dll!`anonymous namespace'::FetchPageInfo+0x205
xul.dll!mozilla::places::AsyncAssociateIconToPage::Run+0x26
xul.dll!nsThread::ProcessNextEvent+0x15a
xul.dll!nsThread::ThreadFunc+0x88
Attached image Xperfview summary table
we have removed some thread contention in vacuum, and actually this could happen while looking at a video online, for example. the idle service doesn't make distinction.
Did you notice hangs lately?
Status: UNCONFIRMED → RESOLVED
Closed: 11 years ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: