674475 - places.sqlite reaches 300MB after browsing for several hours with CyberSearch 2.0.8 add-on

Status: RESOLVED FIXED

(Toolkit :: Places, defect)

Description

[NightsoN Blaze]

User Agent: Mozilla/5.0 (Windows NT 6.1; rv:7.0a2) Gecko/20110723 Firefox/7.0a2
Build ID: 20110723042003

Steps to reproduce:

This problem can be reproduced in the lastest Aurora/Nightly builds.

Open firefox, browse for several hours, close all the tabs, wait for about 10 minutes and then open about:memory.

List of installed extensions:
Adblock Plus 1.3.10a.3096
Add to Search Bar 2.0
AlertBox 0.4.2.20110711
AutoProxy 0.4b2.2011041023
CyberSearch 2.0.8
Easy DragToGo+ 1.1.3.3
Encoding And Decoding 2.0
FlashGot 1.3.0.5rc1
KeySnail 1.8.9
lmnpop 20100820
Multi Links 3.0.0.16
Organize Status Bar 0.6.5
Preserve Download Modification Timestamp 2011.03.21.22
ScrapBook 1.4.7
Scriptish 0.1.4a1pre.20110727.0400.git-5b74130
ShowIP 1.1
Simplify Awesome Bar 0.3.7
Space Next 0.22
Tab Utilities 1.2pre
User Agent RG 1.0
userChromeJS 1.3
View Dependencies 0.3.3.2
WebMail Notifier 2.7.9



Actual results:

High memory usage by places.sqlite as showed below. The memory it uses won't go away, sometime even grows slowly without doing any browsing.

490.99 MB (100.0%) -- explicit
├──314.50 MB (64.05%) -- storage
│  └──314.50 MB (64.05%) -- sqlite
│     ├──291.60 MB (59.39%) -- places.sqlite
│     │  ├──283.03 MB (57.64%) -- cache-used
│     │  ├────6.90 MB (01.40%) -- stmt-used
│     │  └────1.68 MB (00.34%) -- (1 omitted)
│     ├───10.70 MB (02.18%) -- other
│     ├────9.53 MB (01.94%) -- urlclassifier3.sqlite
│     │    ├──9.45 MB (01.92%) -- cache-used
│     │    └──0.08 MB (00.02%) -- (2 omitted)
│     └────2.66 MB (00.54%) -- (12 omitted)
├───90.85 MB (18.50%) -- js
│   ├──60.89 MB (12.40%) -- compartment([System Principal])
│   │  ├──33.85 MB (06.89%) -- gc-heap
│   │  │  ├──12.31 MB (02.51%) -- objects
│   │  │  ├──10.00 MB (02.04%) -- arena-unused
│   │  │  ├───9.73 MB (01.98%) -- shapes
│   │  │  └───1.81 MB (00.37%) -- (4 omitted)
│   │  ├──13.38 MB (02.72%) -- mjit-code
│   │  ├───4.46 MB (00.91%) -- string-chars
│   │  ├───3.69 MB (00.75%) -- scripts
│   │  ├───3.68 MB (00.75%) -- object-slots
│   │  └───1.83 MB (00.37%) -- (3 omitted)
│   ├──21.10 MB (04.30%) -- gc-heap-chunk-unused
│   ├───7.02 MB (01.43%) -- compartment(atoms)
│   │   ├──4.13 MB (00.84%) -- string-chars
│   │   ├──2.89 MB (00.59%) -- gc-heap
│   │   │  ├──2.58 MB (00.53%) -- strings
│   │   │  └──0.31 MB (00.06%) -- (6 omitted)
│   │   └──0.00 MB (00.00%) -- (6 omitted)
│   └───1.84 MB (00.37%) -- (5 omitted)
├───83.14 MB (16.93%) -- heap-unclassified
└────2.50 MB (00.51%) -- (3 omitted)

Comment 1

[Marco Bonardo [:mak]]

How large is your places.sqlite? It's possible this is due to an add-on though, if they ATTACH their own db to our connection.

Btw, bug 674210 may help Places memory usage, that's the first reason I filed that bug.

Comment 2

[Nicholas Nethercote [inactive]]

This sounds similar to bug 673409, where a user saw a 1.2GB(!) place.sqlite report with many add-ons installed, and the problem went away when he disabled add-ons.  (AdBlock Plus is the only add-on you have in common, though.)

So, nightson1988:  can you try again in safe mode (http://support.mozilla.com/en-US/kb/Safe%20Mode), which disabled add-ons?  If that fixes the problem, if you are willing to selectively disable the add-ons in order to work out which one (or more) is responsible, that would be extremely helpful.

Comment 3

[NightsoN Blaze]

(In reply to comment #1)
> How large is your places.sqlite? It's possible this is due to an add-on
> though, if they ATTACH their own db to our connection.
> 
> Btw, bug 674210 may help Places memory usage, that's the first reason I
> filed that bug.

10MB

(In reply to comment #2)
> This sounds similar to bug 673409, where a user saw a 1.2GB(!) place.sqlite
> report with many add-ons installed, and the problem went away when he
> disabled add-ons.  (AdBlock Plus is the only add-on you have in common,
> though.)
> 
> So, nightson1988:  can you try again in safe mode
> (http://support.mozilla.com/en-US/kb/Safe%20Mode), which disabled add-ons? 
> If that fixes the problem, if you are willing to selectively disable the
> add-ons in order to work out which one (or more) is responsible, that would
> be extremely helpful.

Thanks for the suggestion.I will give safe mode a try later today.
Is there any way to show the details of the memory held by places.sqlite so I identify the problem more easily?

Comment 4

[Marco Bonardo [:mak]]

300MB is absolutely unnatural with a 10MB database :(

Comment 5

[NightsoN Blaze]

After a lot of tests,I finally find the problematic addon - Cybersearch 2.0.8.
And here is how to reproduce the problem quickly:
1.Install the latest nightly/aurora build.
2.Disable extension compatibility check.
3.Install cybersearch 2.0.8 from http://mac.softpedia.com/get/Internet-Utilities/CyberSearch.shtml
4.Open about:memory
5.Type anything in locationbar and watch the memory usage of places.sqlite build up everytime you type something

Since cybersearch only supports up to 4.0b7pre and it has been discontinued due to change of Google Search API, I think I have to bear with this problem.

Comment 6

[Nicholas Nethercote [inactive]]

Thanks for taking the time to work this out, nightson1988 -- that's extremely useful.

Comment 7

[Marco Bonardo [:mak]]

no reasons to stay unconfirmed, we should take a look at the code for that addon to see what they are doing.

Comment 8

[NightsoN Blaze]

The author of cybersearch updated it yesterday:https://addons.mozilla.org/en-US/firefox/addon/cybersearch/versions/

Unfortunately this bug still remains.I will contact him to take a look at this:D

Comment 9

[Nicholas Nethercote [inactive]]

I see that version 2.3.1 was released September 6, its release notes said "fixed memory bug".  NightsoN Blaze, can you see if this problem has been fixed?  Thanks!

Comment 10

[NightsoN Blaze]

(In reply to Nicholas Nethercote [:njn] from comment #9)
> I see that version 2.3.1 was released September 6, its release notes said
> "fixed memory bug".  NightsoN Blaze, can you see if this problem has been
> fixed?  Thanks!

It was fixed in 2.3.1 but reappeared in the lastest 2.3.4:(

Comment 11

[Jorge Villalobos [:jorgev] (he/him)]

Can you test version 2.3.8? It's the latest available on AMO.

Comment 12

[NightsoN Blaze]

(In reply to Jorge Villalobos [:jorgev] from comment #11)
> Can you test version 2.3.8? It's the latest available on AMO.

Still leaks.

Comment 13

[Jorge Villalobos [:jorgev] (he/him)]

I just sent a message to the devs.

Comment 14

[Marco Bonardo [:mak]]

Looks like this is another leak that may be fixed by bug 722254.

Indeed the add-on does a createInstance on an autocomplete search instead of using a getService.

Comment 15

[Nicholas Nethercote [inactive]]

Can someone with a build that includes the fix from 722254 confirm that it fixes the problem?

Comment 16

[NightsoN Blaze]

(In reply to Nicholas Nethercote [:njn] from comment #15)
> Can someone with a build that includes the fix from 722254 confirm that it
> fixes the problem?

Works for me, Hooray\(^o^)/~

Comment 17

[Nicholas Nethercote [inactive]]

Thanks for confirmation, NightsoN Blaze!

Comment 18

[Kris Maglione [:kmag]]

This is fixed on the version uploaded to AMO. It would be best if these kinds of bugs were left open while it is still possible to get the problems fixed in the add-on some 10 weeks before the core fixes land in a public release.