Closed Bug 639626 Opened 13 years ago Closed 13 years ago

Memory leak when using Firefox 4 and Bugzilla Tweaks (~1MB per tab opened/closed)

Categories

(Core :: XPConnect, defect)

Product:
Core
Component:
XPConnect
Platform:
x86
Windows 7
Type:
defect
Priority:
Not set
Severity:
major

Tracking

()

Status:
VERIFIED DUPLICATE of bug 607601

People

(Reporter: emorley, Assigned: gal)

References

Details

(Keywords: memory-leak, regression)

Attachments

(3 files)

Testcase Bookmarks
107.76 KB, text/html
Details
cc-edges-log (raw)
447.79 KB, application/octet-stream
Details
Reversed CC edges log
860.46 KB, application/x-bzip2
Details
Attached file Testcase Bookmarks — 
Ehsan requested that I file this bug, after mentioning the leak to him.

Summary:
When using the addon Bugzila Tweaks with the latest nightly, memory is not released (according to 'memory in use' in about:memory) when tabs are closed, which after a BMO triaging session (ie opening and closing many BMO tabs), results in high memory usage, that can only be cleared by restarting Fx.

Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b13pre) Gecko/20110303 Firefox/4.0b13pre ID:20110303122446

~ STR:

Baseline usage...

1) Using latest nightly, create a new profile.
2) Open profile and import the bookmarks.html attached to this thread (contains 80 bookmarked BMO bugs, to expedite testing).
3) Middle mouse click on the imported bookmarks folder "Testcase tabs to open", wait for all tabs to finish loading.
4) Right click any tab, use "close other tabs", so now just one tab open.
5) After waiting 30 seconds for memory usage to settle, go to about:memory (in that one remaining tab) and make a note of 'memory in use' value.
6) Repeat steps 3-5 several times, making a note of the memory in use after each iteration.

Results:
- 1st iteration: Memory in use = 84,513,108
- 2nd iteration: Memory in use = 85,933,832
- 3rd iteration: Memory in use = 86,331,438
- 4th iteration: Memory in use = 87,008,932
- 5th iteration: Memory in use = 87,366,468

Using Bugzilla Tweaks 1.8...

Use the same methodology as for the baseline, except inbetween steps 2 and 3, install BugZilla tweaks 1.8 from here:
https://addons.mozilla.org/firefox/187588
...and then restart Fx (just in case going to AMO affected memory usage), before continuing with step 3 onwards from above.

Results:
- 1st iteration: 173,669,404
- 2nd iteration: 254,743,952 
- 3rd iteration: 332,330,176
- 4th iteration: 412,574,016
- 5th iteration: 495,640,840

This shows a consistent ~80MB increase in the about:memory 'memory in use' value, after each set of 80 tabs was opened and then closed; that did not occur when Bugzilla Tweaks was installed. This memory usage can only be freed by restarting Fx.

Reproducibility: 100%

Note: My GFX card is blocklisted (nVidia 6200TC...yes I know :-$); so even though I was using a clean profile, H/W acceleration would have been disabled (layers, the lot).
s/was installed/was not installed

Few extra points:

- The memory leak above only seems to have occurred for the last week or two. I know Bugzilla Tweaks has been updated recently to 1.8; so not sure if it's a regression on m-c or in the addon. Will attempt to find a regression range now.

- The memory leak appears to be made *much* worse when ABP (with filters) and Noscript are also used. By much worse, I mean that after iteration ~2/3; usage would be over 1GB. (This has been happening to me daily and driving me up the wall). However, I need to do further testing with all three addons installed, to try and work out what is going on (ie: is noscript involved, or is it just ABP and bugzilla tweaks? etc etc).
Exactly regression points for the extension and the firefox side would be great.
Assignee: nobody → gal
Bugzilla Tweaks 1.7 & 1.8 are affected; 1.6 is not.
ie: Break on addon side happened between 1.6 and 1.7.

Going to work on Firefox side now, but will take a bit longer.
Ed, your help is very much appreciated.
I'm still finding trying to find some semblance of a regression range (the leak seems even worse as of 2011-01-01 nightly?!), but something I just noticed, which may mean something to you...

After the STR above, if Bugzilla Tweaks is disabled in addons manager, seconds later the mem in use figure in about:memory drops back down to the expected value, without having to restart Fx. I'm presuming that some memory leaks wouldn't be helped by disabling the (jetpack) addon in question, so does this help narrow the cause?
It is also possible to bisect this using the add-on's source code, which is being hosted in an hg repository here: <https://bitbucket.org/ehsan/bugzilla-tweaks>

(In reply to comment #5)
> After the STR above, if Bugzilla Tweaks is disabled in addons manager, seconds
> later the mem in use figure in about:memory drops back down to the expected
> value, without having to restart Fx. I'm presuming that some memory leaks
> wouldn't be helped by disabling the (jetpack) addon in question, so does this
> help narrow the cause?

This might be an indication that the jetpack runtime is holding on to some stuff longer than necessary.  CCing myk, in the hopes that he would CC other jetpack hackers if he deems necessary.
Perhaps time for me to bite the bullet and figure out how to use hg. Presume I can say set up a local repo pointing at the extensions folder, then run hg bisect? Is tortoisehg what people would recommend for using hg on Win?

Anyway, here are the results for the Firefox side...

- Used STR from comment 0 (so Bugzilla Tweaks v1.8)
- Data is in MB; as measured by "memory in use" field from about:memory; 
- The 80 bookmarks were opened and then closed three times per build. The 1st time/iteration I've listed the actual usage, the subsequent times I've just listed the increase/leak amount.

Using m-c nightlies, win32 opt x86:
2011-03-03: 1st iteration = 195 ; subsequent: +69, +75
2011-02-24: 1st iteration = 174 ; subsequent: +93, +74
2011-02-17: 1st iteration = 172 ; subsequent: +77, +64
2010-12-01: 1st iteration = 142 ; subsequent: +50, +39
2010-10-01: 1st iteration = 190 ; subsequent: +135, +136
2010-05-17: 1st iteration = 146 ; subsequent: +93, +96
2010-05-08: 1st iteration = 131 ; subsequent: +107, +95
2010-04-22: 1st iteration = 146 ; subsequent: +94, +90
2010-04-16: 1st iteration = 131 ; subsequent: +92, +108
--
2010-04-15: 1st iteration = 40 ; subsequent: +8, +7
2010-04-14: 1st iteration = 56 ; subsequent: +14, +1
2010-04-14: 1st iteration = 55 ; subsequent: +10, +7
2010-04-07: 1st iteration = 41 ; subsequent: +10, +5
2010-03-07: 1st iteration = 43 ; subsequent: +5, +2
2010-01-05: 1st iteration = 30 ; subsequent: +19, +6

Last good nightly: 2010-04-15 
First bad nightly: 2010-04-16
Pushlog: http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=d9d7de04f75d&tochange=fc3befa99ab7

= Large TM merge, so tested on TM too.

Using tracemonkey nightlies, win32 opt x86:
2010-04-09: 1st iteration = 132 ; subsequent: +94, +92
2010-04-08: 1st iteration = 131 ; subsequent: +92, +92
--
2010-04-07: 1st iteration = 41 ; subsequent: +8, +11
2010-04-05: 1st iteration = 39 ; subsequent: +11, +7

Last good nightly: 2010-04-07 
First bad nightly: 2010-04-08
Pushlog: http://hg.mozilla.org/tracemonkey/pushloghtml?fromchange=aef12a0af84c&tochange=53e381fe0f8b

Much smaller range, hope it helps...
Ehsan, I've:
- Installed HgTortoise
- Cloned the Bugzilla Tweaks repo
- Realised that I can't just drop the files from the repo in the extensions folder as they have to be built with the SDK, duh
- Installed the Jetpack SDK
- Installed python, set up the paths
- Tried to run activate, but it fails with:
ERROR: The system was unable to find the specified registry key or value.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named jetpack_sdk_env

Ideas?
In local build (from m-c repository, on ubuntsu10.04),
    (System monitor MiB) ini 1st 2nd 3rd  (ditto ΔM)
Build from 53e381fe0f8b : 31 206 313 425  (- +175 +107 +112)
Build from 988aee821e1c : 31  82 108 126  (-  +51  +26  +18)
Triggered by:
53e381fe0f8b	Jeff Walden — Bug 550402 - Property readonly bit interferes with setter functions in properties. r=jorendorff
I'll take a look tomorrow, hopefully.  I'm a little bit surprised that patch would have any effect on this, but only a little.  Maybe somehow there's a property being defined with a slot that shouldn't, and it's entraining something, or something.
Finally got the jetpack SDK working; so could narrow the regression range on the bugzilla-tweaks side using hg bisect:

--
Due to skipped revisions, the first bad revision could be any of:

changeset:   37:1a227d923141
user:        Heather Arthur <fayearthur@gmail.com>
date:        Tue Feb 15 11:29:46 2011 -0800
summary:     Move main.js to bug-page-mod.js
https://bitbucket.org/ehsan/bugzilla-tweaks/changeset/1a227d923141

changeset:   38:b76440589008
user:        Heather Arthur <fayearthur@gmail.com>
date:        Tue Feb 15 11:59:54 2011 -0800
summary:     Update to new Addon SDK page-mod and context-menu modules
https://bitbucket.org/ehsan/bugzilla-tweaks/changeset/b76440589008
--

Had to skip 37:1a227d923141 since wasn't in a usable state, given the file rename. 

Hope that helps!
Blocks: 550402
Keywords: regressionwindow-wanted
Sorry, Ed, I was behind my bugmail for a few days, but in the interest of future people reading this bug, here's how you build bugzilla-tweaks from scratch:

git clone git://github.com/mozilla/addon-sdk.git
hg clone https://ehsan@bitbucket.org/ehsan/bugzilla-tweaks
cd addon-sdk
. bin/activate
cd ../bugzilla-tweaks
cfx xpi

After this, you'll have an XPI in the current directory which can be installed in Firefox.

In reply to comment 11, https://bitbucket.org/ehsan/bugzilla-tweaks/changeset/b76440589008 is unfortunately a big change.  You should note that before this change, I used to build bugzilla-tweaks with the old that used to be hosted at http://hg.mozilla.org/labs/jetpack-sdk/.

All of this makes me suspect that it's a bug in the js engine.  Jeff, did you manage to have a look?
Is there anything I can do to help provide more information with this? 

I've had to disable bugzilla tweaks for now due to the leak and I'm really starting to notice it's absence; it used to make bugzilla much more tolerable! 

Is it just not leaking for everyone else/other platforms, or has everyone else disabled it too?

I've had a skim read of https://wiki.mozilla.org/Performance:Leak_Tools but not really sure which method would be a good place to start given what may be leaking. Any ideas? (win32)

Thanks!
Can we use the cycle collection tools to gain some insight here?
Probably. Evaluate this in the Error Console:

window.QueryInterface(Components.interfaces.nsIInterfaceRequestor).getInterface(Components.interfaces.nsIDOMWindowUtils).garbageCollect(Components.classes["@mozilla.org/cycle-collector-logger;1"].createInstance(Components.interfaces.nsICycleCollectorListener))

See https://bugzilla.mozilla.org/show_bug.cgi?id=497808#c63 for some suggestions on debugging with the log.
(In reply to comment #15)
> Probably. Evaluate this in the Error Console:

Entering that in the error console and pressing evaluate didn't give any visual response as to the command having been evaluated; and I can't seem to find a log anywhere.

Is it supposed to work on opt win32 builds or should I be using debug?
It will have dropped a cc-edges-N.log file in the browser's current working directory.
No such luck, imagine UAC intervened. Will try using a build outside of program files.
it shows for me - 183MB :)
on XP, account with admin privileges
Attached file cc-edges-log (raw) — 
Worked when outside of program files, must have just been UAC.

Attaching log. (Log is of opening and then closing one BMO tab with bugzilla tweaks 1.8 installed)
(In reply to comment #15)
> See https://bugzilla.mozilla.org/show_bug.cgi?id=497808#c63 for some
> suggestions on debugging with the log.

So I did run those commands on both Ed's log and mine (too big to upload anywhere), but find-roots.pl always just finishes silently, without outputting anything.
On Windows? The script needs to be tweaked a bit to recognize addresses that don't start with 0x.
Mine (comment 20) was using win32; that would explain why find-roots.pl outputted nothing for me either.
(In reply to comment #22)
> On Windows? The script needs to be tweaked a bit to recognize addresses that
> don't start with 0x.

No, on a Mac.  Should that work fine?
I don't know, I haven't tried on Mac. However the file format and the script are very simple, you should be able to figure it out :-).
I've uploaded a log to dropbox for roc - but it covers more browser activity than just bugzilla with bugzillatweaks enabled. Can offer to someone else, or a reduced testcase if needed.
That shows 139 Bugzilla documents live. It also suggests you had 204 tabs open. Is that true?
Actually:
-- one window with 73 tabs
-- one window with 83 tabs
Is that right?

Unfortunately we actually need a CC log from a debug build to track references through JS :-(. It does look like we have Bugzilla documents which only JS is keeping alive, for example 0C9DD000 in your log. I assume that's a bug in the Bugzilla Tweaks extension doing that.
(In reply to comment #28)
> Actually:
> -- one window with 73 tabs
> -- one window with 83 tabs
> Is that right?

correct

I have a newer, shorter log. but still not from a debug log. don't currently have access to debug
We keep a debug build each day for mozilla-central (and 1.9.2). Look in directories like
 http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2011-03-26-03-mozilla-central-debug/

That's a copy of the last debug build in firefox/tinderbox-builds/$branch-$platform-debug when 3am Pacific comes around.
Not sure if this helps, but this is due to Addon SDK's page-mod module. When I take out the code in Bugzilla Tweaks that uses page-mod there's no memory leak. And if I make a small extension that uses a very simple page-mod the leak occurs.
Attached file Reversed CC edges log — 
This is a log from a debug build.  With about:home being open, I opened one bugzilla tab (this very bug), closed it, waited a few minutes, and entered the logger code into the error console.

I'm not sure if my investigations were correct, but it seems like we're leaking a Sandbox object.  As far as I can tell, this is the only place in which a sandbox is created: <https://github.com/mozilla/addon-sdk/blob/master/packages/api-utils/lib/content/worker.js#L173>

The interesting things about that code seems to be that WorkerGlobalScope and Worker both hold a reference to the window, and WorkerGlobalScope seems to be holding a reference to Worker too.  But these references seem to be broken in the destructor methods...
Alex: any thoughts on what might be causing this?  Perhaps it's related to the remaining leaks in bug 642306?
My analysis in comment 21 makes me believe that bug 642306 might be the culprit here...
Depends on: 642306
Depends on: 607601
The patch in bug 607601 fixes this leak!  Hurray!
Status: NEW → RESOLVED
Closed: 13 years ago
No longer depends on: 642306
Resolution: --- → DUPLICATE
No longer depends on: 607601
Awesome! :-)

One quick question:

Given the platform regression changeset in comment 9; is there also a platform leak here and not just jetpack SDK?
(In reply to comment #36)
> Given the platform regression changeset in comment 9; is there also a platform
> leak here and not just jetpack SDK?

I couldn't say...
Depends on: 607601
There is multiple leaks around page-mod that are in process to be fixed.
The bigest one that lead to full document leak and provoke leaks like 1MB per tab comes from jetpack SDK code. 
There is another leak due to platform code, registered under bug 646575, around sandboxes. But this leak may always had existed!

So to resume, you may consider this bug as being almost jetpack specific.
Depends on: 650649
No longer depends on: 650649
Now that bug 607601 is fixed, is it a good idea for me to release a new version of Bugzilla Tweaks based on the addon-sdk's master branch?
I've re-run my testcase from comment 0 with Bugzilla Tweaks v1.10 - and the leak is no longer present (memory in use at the end of each cycle remains within +/- 1% of it's start value), so the fix to bug 607601 has solved the problem. 

Thanks Alex/Ehsan! :-)
Status: RESOLVED → VERIFIED
In case it's not obvious, I rebuilt 1.10 with a version containing a fix to bug 607601.
(I can't tell yet if there is any connection to what has changed for tweaks) 

I started FF with 2011-04-27 build and tweaks 1.10 enabled (I had been running 2011-04-20 with tweaks disabled) and at some point all my bugzilla tabs went blank, including the tab title.   (~130 tabs total, in 3 windows)
No longer blocks: mlk2.0
No longer blocks: mlk-fx5+
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: