734408 - Vimperator add-on is leaky

Status: RESOLVED FIXED

(WebExtensions :: General, defect)

Description

[Malini Das [:mdas] - Away, not checking bugmail]

After a while of running Nightly and Collusion on my OSX machine, I see *really* slow CC and GC cycles, causing interaction delays lasting 1-2 seconds, occurring every few seconds. GC duration would go over 2000ms and CC duration would climb up to 600-700ms at the worst. In about:compartments, there were a lot of active compartments from websites that were no longer open.

I'm currently on 13.0a1 (2012-03-08), but I had this problem the day before (7th) and had automatically applied an update since then.

Comment 1

[Andrew McCreight [:mccr8]]

The slow GC and CC seem to be due to memory leaks. Her Facebook compartment was over 100MB.

Comment 2

[Justin Lebar (not reading bugmail)]

> After a while of running Nightly and Collusion on my OSX machine

No other extensions, right?

I just played with Collusion 0.16.2 on my Linux box in a clean profile of Nightly.  I browsed to some news sites, opened and closed the Collusion page, and didn't see any compartment leaks.

Comment 3

[Malini Das [:mdas] - Away, not checking bugmail]

I'm using this on an OSX if that makes any difference, with at least 30 other tabs open. The persisting compartments I'm seeing are from facebook, twitter and some news sites, and they persist after closing all tabs to that site and after GC has run

Comment 4

[Jono Xia]

Thanks for bringing this to my attention.

Malini, are the memory leaks happening only while you have the Collusion UI open in a tab? Or does it persist after you close the collusion UI?  Or does it happen even if you never open the UI page?

That will help me figure out what part of the extension code the memory leak might be in.

Comment 5

[Malini Das [:mdas] - Away, not checking bugmail]

Ah, I believe I'm seeing this because I had a newssite that was using some facebook/twitter plug-in, and that might have made the compartment persist as mccr8 suggested to me.

I didn't activate the UI during the first run when I saw this. I just restarted my Nightly instance, closed the newssites, and now I don't see any unnecessary compartments, so I'm pretty sure it's because of fbconnect and what not.

Comment 6

[Justin Lebar (not reading bugmail)]

> Ah, I believe I'm seeing this because I had a newssite that was using some facebook/twitter 
> plug-in, and that might have made the compartment persist as mccr8 suggested to me.

Andrew, can you elaborate on what you think was happening here?

Comment 7

[Andrew McCreight [:mccr8]]

Facebook like buttons are put in a FB compartment, right?  So you could close the FB page but still have FB compartments around.

Comment 8

[Jorge Villalobos [:jorgev] (he/him)]

Yeah, social network widgets are inserted using iframes, so it sounds like those weren't zombie compartments after all.

Please reopen if I got this wrong.

Comment 9

[Andrew McCreight [:mccr8]]

Well, she's still experiencing massive memory usage with the addon, and not experiencing it without the addon. That suggests something is wrong, though it sounds like reproducing it may be difficult.

Comment 10

[Justin Lebar (not reading bugmail)]

> Facebook like buttons are put in a FB compartment, right?  So you could close the FB page but 
> still have FB compartments around.

> http://www.facebook.com/plugins/like.php?

Ah, yes.  Were all the zombies pages with corresponding "like button" frames, or did you see zombies for content origins like nytimes.com or techcrunch.com?

Comment 11

[Jorge Villalobos [:jorgev] (he/him)]

(In reply to Andrew McCreight [:mccr8] from comment #9)
> Well, she's still experiencing massive memory usage with the addon, and not
> experiencing it without the addon. That suggests something is wrong, though
> it sounds like reproducing it may be difficult.

That could be related to the add-on's complexity and not necessarily memory leaks. Malini, can you post the contents of about:memory to see if anything stands out?

Comment 12

[Justin Lebar (not reading bugmail)]

(In reply to Jorge Villalobos [:jorgev] from comment #11)
> (In reply to Andrew McCreight [:mccr8] from comment #9)
> > Well, she's still experiencing massive memory usage with the addon, and not
> > experiencing it without the addon. That suggests something is wrong, though
> > it sounds like reproducing it may be difficult.
> 
> That could be related to the add-on's complexity and not necessarily memory
> leaks. Malini, can you post the contents of about:memory to see if anything
> stands out?

It would be great if we could get the pastebin we looked at yesterday.  Something was extremely wrong; she was using 1.5gb of memory for relatively few compartments.  GC times were upwards of 2s.

This add-on is not that complex.  I'm re-opening because we have a preponderance of evidence that something was wrong.  We can resolve incomplete if we can no longer reproduce.

Comment 13

[Malini Das [:mdas] - Away, not checking bugmail]

Attachment: About:memory read out

I don't have the information from the previous occurrence, but I just ran into his again now. The problem here is that I've been running Nightly for over a day with the Collusion, Vimperator and Bugzilla tweaks add-on. I'm not sure if any of these are the contributors of this problem. I'll copy this profile and run it without these addons, and with collusion only to see what the problem is. It will most likely be a day or two before I'll see the problem!

I've attached the about:memory read out, will attach about:compartments and memchaser log

Comment 14

[Malini Das [:mdas] - Away, not checking bugmail]

Attachment: about:compartments

about:compartments info

Comment 15

[Malini Das [:mdas] - Away, not checking bugmail]

Attachment: memchaser log

Comment 16

[Justin Lebar (not reading bugmail)]

What tabs did you have open when you collected those readouts?

Comment 17

[Justin Lebar (not reading bugmail)]

Also, the about:memory?verbose output would be useful.

Comment 18

[Malini Das [:mdas] - Away, not checking bugmail]

Attachment: list of open tabs

Here's the list of open tabs. Unfortunately, I can't get the about:memory?verbose log for you now, I had to restart the browser

Comment 19

[Justin Lebar (not reading bugmail)]

Okay, there are definitely zombie compartments in there, such as

http://blog.gerv.net/2012/03/you-dont/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+HackingForChrist+%28Hacking+for+Christ%29&utm_content=Google+Reader

http://absolutelymadness.tumblr.com/post/15289955301

https://phonebook.mozilla.org/

Comment 20

[Justin Lebar (not reading bugmail)]

J'accuse Vimperator.

I installed it, visited a few sites, then closed everything except bugzilla, gmail, and gcal.  The copmartments from a twitter button and a news site were still up.  Then I closed all my tabs except about:compartments.  The twitter and news sites' compartments went away, but my bugzilla and Google compartments were still alive.

Comment 21

[Jorge Villalobos [:jorgev] (he/him)]

(In reply to Justin Lebar [:jlebar] from comment #20)
> J'accuse Vimperator.

Kris, can you verify this?

Comment 22

[Kris Maglione [:kmag]]

(In reply to Jorge Villalobos [:jorgev] from comment #21)
> (In reply to Justin Lebar [:jlebar] from comment #20)
> > J'accuse Vimperator.
> 
> Kris, can you verify this?

I don't have anything to do with Vimperator anymore, so I don't have much to say on the matter. It certainly does have a lot of interaction with content and isn't well maintained, or at least wasn't in the year after I left the project, so the chances are good that it does leak.

Comment 23

[Jorge Villalobos [:jorgev] (he/him)]

Adding Vimperator dev.

Malini, can you verify that the leaks are gone after Vimperator is disabled?

Comment 24

[Malini Das [:mdas] - Away, not checking bugmail]

Aha, yes, I can! Thanks for the find!

Comment 25

[Jet Villegas (inactive)]

P3 per Memshrink.

Comment 26

[Jorge Villalobos [:jorgev] (he/him)]

The developer has been notified the listing will be downgraded next week if there's no acknowledgement on his side regarding this bug.

Comment 27

[Segaja]

I'm working with Martin Stubenschrott, but my knowledge about zombie compartments is not that good. I will read some documentation to try to get an idea of the problem.

Comment 28

[Justin Lebar (not reading bugmail)]

> I'm working with Martin Stubenschrott, but my knowledge about zombie compartments is not that good.

We're here to help if you still need it after reading the docs.

Comment 29

[Martin Stubenschrott]

Ok, we seem to have takled at least one big memory leak regarding closed tabs here:

http://code.google.com/p/vimperator-labs/source/detail?r=ba70eee96808681b7e25bc370191a7487a78157b

It's hard to say if that is everything, as of course the code base is quite huge. I still created a first beta version of the upcoming 3.4 version including this bug fix. Can anybody test, if it helps fixing the memory leak?

The XPI is here:

http://vimperator-labs.googlecode.com/files/vimperator-3.4b1.xpi

(I'd like some feedback before "officially" releasing a new version on AMo).

Comment 30

[Justin Lebar (not reading bugmail)]

I couldn't get this version to leak, upon a cursory inspection.  It seems a lot better.  Malini should probably put it through its paces, though, if she has a chance.

Comment 31

[Martin Stubenschrott]

I tried to upload the new version as 3.4 to addons.mozilla.org, but after the file has uploaded 100%, the page says "Validating vimperator-3.4.xpi" but that for at least 2h now. Is something broken on AMo?

Comment 32

[Jorge Villalobos [:jorgev] (he/him)]

The validator has been having problems recently. Please try again, and if the problem continues, please file a bug (https://bugzilla.mozilla.org/enter_bug.cgi?product=addons.mozilla.org) and attach the file you're trying to upload.

Comment 33

[Jorge Villalobos [:jorgev] (he/him)]

Actually, this is probably bug 739106.

Comment 34

[Martin Stubenschrott]

Thanks for the info, I resubmitted 3.4 and it worked flawlessly now.

Comment 35

[Andrew Williamson [:eviljeff]]

Memory leak appears to be fixed in Vimperator 3.4 so marking as fixed.