1140902 - places.sqlite-wal extremely large and causes firefox to become unresponsive

Status: VERIFIED INCOMPLETE

(Toolkit :: Places, defect)

Description

[neopeeves]

Attachment: sqlite.png

User Agent: Mozilla/5.0 (Windows NT 6.1; rv:36.0) Gecko/20100101 Firefox/36.0
Build ID: 20150305021524

Steps to reproduce:

Casual usage of firefox, single session lasting multiple days without closing


Actual results:

places.sqlite-wal built up an excessive amount of data, causing firefox to become unresponsive or freeze completely. Closing tabs does not remove the data for that page, and firefox runs as if every page open during that session remained open simultaneously. To free the memory, firefox must be fully closed.


Expected results:

Closing or reloading a page/tab for which information is held should free the associated memory. For example, closing a tab using 100k should reduce the memory of firefox.exe by 100k. Leaving a website which has many images cached should remove those images from the cache as well.

Comment 1

[Marco Bonardo [:mak]]

There is only one case where the file can grow much, is that when there is an open transaction that something forget to close. In any other cases we truncate the wal, it can grow but then it gets truncated asap.

300MB is not even that large, I saw worse scenario. I think the unresponsiveness here is sort of unrelated to the size of the wal, we do have performance issues when the user executes very large deletes from history/bookmarks. What were you doing at the time of the freeze?

Which add-ons do you have? Any that might interact with bookmarks or history?

Comment 2

[neopeeves]

(In reply to Marco Bonardo [::mak] -- catching up, delayed answers -- from comment #1)
> There is only one case where the file can grow much, is that when there is
> an open transaction that something forget to close. In any other cases we
> truncate the wal, it can grow but then it gets truncated asap.
> 
> 300MB is not even that large, I saw worse scenario. I think the
> unresponsiveness here is sort of unrelated to the size of the wal, we do
> have performance issues when the user executes very large deletes from
> history/bookmarks. What were you doing at the time of the freeze?
> 
> Which add-ons do you have? Any that might interact with bookmarks or history?

I was browsing theworstthingsforsale. I know that firefox freezes for a moment when deleting history/cache, but this is completely unrelated. I deleted cache after the problem started, to try to reduce firefox's memory. Needless to say, it did nothing and firefox remained at 1,200,000kb

I have no add-ons that affect those, but I have 2 that affect tabs. Bartab heavy and bug489729.

Comment 3

[neopeeves]

Also happens on imgur. Firefox skyrockets to over 1,300,000K, becomes extremely sluggish and unresponsive, and storage/sqlite/other in about:memory is returning a value of -0.33MB

Comment 4

[Marco Bonardo [:mak]]

Can you give me the imgur address to try?
Also, after a backup of the profile, please check coherence of your database
https://addons.mozilla.org/en-US/firefox/addon/places-maintenance/

Comment 5

[neopeeves]

(In reply to Marco Bonardo [::mak] from comment #4)
> Can you give me the imgur address to try?
> Also, after a backup of the profile, please check coherence of your database
> https://addons.mozilla.org/en-US/firefox/addon/places-maintenance/

It's not one address. Browsing through images doubles firefox's memory consumption. I'm guessing because it keeps EVERY image loaded even after leaving the page the image was on, instead of deleting the cached image like it should be doing.

Also, I already looked at that add-on. And as I said in my review. I'm not using mystery buttons. There is NO explanation of what the buttons on that add-on do, and the names of the buttons are in no way helpful. I might as well install a russian add-on and click buttons randomly. I'm not using that add-on until they feel the need to tell people what it does.

Comment 6

[Marco Bonardo [:mak]]

I wrote that add-on, and I'm the owner of this module, so there's nothing really misterious there. It just invokes maintenance code that Firefox should run by itself once a week, but for many users that doesn't happen, due to existing issues.

Comment 7

[neopeeves]

(In reply to Marco Bonardo [::mak] from comment #6)
> I wrote that add-on, and I'm the owner of this module, so there's nothing
> really misterious there. It just invokes maintenance code that Firefox
> should run by itself once a week, but for many users that doesn't happen,
> due to existing issues.

"This helps firefox"

Yeah, not doing it. You tell me (And every one else) EXACTLY what EVERY button does. I'm not going to **** up my firefox because you were too lazy to tell me what the buttons do, and force me to press them randomly and hope I hit the right one before I hit the wrong one. I'm not going to sit here with a notebook, click one option, write down what it does, click the next option, write down what it does, etc. That's YOUR responsibility. I'm not going to install an add-on that has no information other than "It helps" and press random buttons that have no explanation as to what they do. You do not make an add-on and refuse to tell people how to use it. You do not say "Here's 5 buttons. Just click them randomly, and maybe it will do what you want it to"

Comment 8

[camillebarneaud]

Same here Firefox 36 (Win 7 x86) my places.sqlite-wal is larger than 2 Go.
It's linked with the history management because I tried to delete the older entries of my history by different means (even just to delete a month) and it always freeze and I have been unable to delete them for months.
I just tried to delete places.sqlite-wal and yes its size skyrockets when you manage history.

Comment 9

[Marco Bonardo [:mak]]

Regarding huge wal when clearing history, please see Bug 871908.