1106833 - Memory leak on x64Nightly with Pushbullet Addon

Status: VERIFIED FIXED

(Core :: General, defect)

Description

[Mayank Bansal]

Attachment: t2.txt

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:37.0) Gecko/20100101 Firefox/37.0
Build ID: 20141201030207

Steps to reproduce:

in my normal profile, with e10s disabled,  I kept open a gmail tab open , and left firefox idle for some time.

At startup, the memory use was ~400MB. After about 2 hours, memory use was ~700mb, and after 5 hours from then, memory use was 1.2GB.


I am attaching the memory reports of the 2 hour, and the 5 hour reports.
Both these reports were with Nightly being completely idle.  I use Nightly x64

Comment 1

[Mayank Bansal]

Attachment: t5.gz

memory usage after 5 hours

Comment 2

[Mayank Bansal]

both these reports are after minimizing memory in about:memory

Comment 3

[Mayank Bansal]

After disabling all my extensions, i couldnt reproduce this.
So this is definitly a addons-extensions issue.

I will try to narrow down which addon causes this.

Comment 4

[Mayank Bansal]

Confirmed it is the Pushbullet Addon. You can get the latest version from here  https://addons.mozilla.org/en-US/firefox/addon/pushbullet/


STR:

1. Create a new profile
2. Install the Pushbullet addon
3. Let Nightly be idle for 8-9 hours.


Memory usage is huge. In my case, it went from ~200MB to ~800MB overnight.  Doing minimize memory had no effect.



Attaching the memory report after 8 hours, and doing minimize memory.

Comment 5

[Mayank Bansal]

Attachment: about:memory report

Comment 6

[Mayank Bansal]

tentatively ni? ing you, as you seem the best person to take a look at this.

Comment 7

[Nicholas Nethercote [inactive]]

Thank you for working out which add-on is the cause; that's extremely helpful.

The most important thing in the memory reports is this:

> ├──404.17 MB (50.15%) ── heap-unclassified

I've started a DMD run which will give us more insight into this memory usage.

And I've CC'd kmag, who is the add-on leak expert.

Comment 8

[Mayank Bansal]

3 things i forgot to mention :

1. This is with e10s disabled. I havent checked if the leak persists in e10s mode.
2. You might have to sign on the addon with your google account.
3. I enabled the  "enable universal copy paste" in the addons options. You might have to, too.

Comment 9

[Kris Maglione [:kmag]]

I briefly tested this with the instructions from comment 4 and comment 8, but after a couple of hours I saw no significant increase in memory use, so I suspect that there are some other necessary steps.

I also gave the code a quick look and didn't see anything glaringly problematic. Except that enabling the universal copy & paste causes the add-on to make 5 useless HTTP requests to localhost per second and, if I'm reading the code right, doesn't actually work (at least for the purposes of reporting clipboard changes to its servers).

However, looking at your memory report, this is probably relevant, given the amount of memory consumed by the requests.js module.

Did you do anything else in the browser before leaving it to idle? Did you do anything else on other devices which would cause the Pushbullet servers to interact with the add-on? Can you also try reproducing this with universal copy & paste disabled?

Thanks.

Comment 10

[Mayank Bansal]

In Nightly->Options->Options->Privacy, i made two changes :

1. Enabled the "I dont want to be tracked"
2. Chose "Use custom settings for history" , and then
 "Accept cookies from third party" = never .


The "never accepting cookies from third party" will cause the addon to not sign you in. Which means you can sign in the addon from the pushbullet.com website, but in the actual addon you will remain unsigned. 
To sign in, you have to temporarily enable third party cookies and then sign in the addon.
After that, i disabled third party cookies again.  I remain signed in the addon, AND i dont get third party cookies.

Could this be one of the reasons of the leak ?




i use the internet through a wall, which will log me out after two hours.

To check whehter the addon leaks with "universal copy paste", i will leave the browser open tonight.

Comment 11

[Nicholas Nethercote [inactive]]

I also couldn't reproduce the problem. I'll wait for the results from comment 10 before trying again.

Comment 12

[Mayank Bansal]

So i left Nighty idle for the night.
I disabled universal copy, and disabled third party cookies. And the memory leak didnt happen.


So i think the leak is due to a combination of disabling third party cookies, and enabling universal copy-paste.

Comment 13

[Mayank Bansal]

These appear to be better STR:

1. Install Nightly x64. Disable e10s.
2. Install Pushbullet addon.
3. In the addon, sign in your google account.
4. From the addon settings, enable Universal copy
5. From the Nightly->Options->Privacy,  set "Allow third party cookies" to never.

Comment 14

[Kris Maglione [:kmag]]

I really wish you hadn't changed two variables at once.

In any case, I suspect that third-party cookies are not relevant. Given the memory usage in the requests module and the dependence on that setting, it looks like this is probably a leak in that module.

However, I still can't reproduce this myself, so I think we're still missing some information. The copy and paste feature attempts to make requests to a program running on localhost. Are you running a separate Pushbullet app? Are you opening any pages in the same session before you leave Firefox to idle?

Also, can you possibly try to reproduce with a Pushbullet account which isn't active on any other devices?

Comment 15

[Mayank Bansal]

(In reply to Kris Maglione [:kmag] from comment #14)
 Are you running a separate Pushbullet app? 
> 

i have the Android Pushbullet app on my android phone, and a Pushbullet extension for Google Chrome on desktop. 


> Are you opening any pages in the same session before you leave Firefox to idle?

No. On startup, as soon as a visible window appears, i open a newtab.



> Also, can you possibly try to reproduce with a Pushbullet account which
> isn't active on any other devices?

Will try.

For the next try, i will just enable universal copy, with a account that isnt activated anywhere.

Comment 16

[Mayank Bansal]

Attachment: memory-report.json.gz

In a new profile, I signed on with a new ID that isnt activated anywhere else, and enabled the universal copy feature.

Memory use increased from ~150 to ~ 350 in 2 hours.
Attaching the memory report

Comment 17

[Kris Maglione [:kmag]]

OK, it looks like I'm able to reproduce part of this on nightly and Aurora now. As far as I know, I haven't changed anything, but I'm now seeing heap-unclassified grow steadily. I'm still not seeing any growth in the add-on compartments, though, and that's something I'm just as worried about.

I'll leave these instances running for a few hours and see what happens.

Comment 18

[Kris Maglione [:kmag]]

Hm. This is strange. While your memory reports show growth in heap-unclassified and the Pushbullet add-on's zone, I'm seeing growth of heap-unclassified and the SDK loader's compartment.

I want to be sure we're testing with the same versions. Can you give me the build date from the About dialog, and the contents of about:support (click 'Copy text to clipboard')?

Comment 19

[Kris Maglione [:kmag]]

Attachment: about:memory dump

Comment 20

[Mayank Bansal]

Attachment: aboutsupport.txt

https://hg.mozilla.org/mozilla-central/rev/c1f6345f2803

Comment 21

[Mayank Bansal]

You dont even have to log in with a account on Pushbullet. Just installing the addon, and enbabling universal copy will leak memory.

Comment 22

[Kris Maglione [:kmag]]

Hm. That about:support output has a lot of other add-ons installed and enabled, Pushbullet disabled, and e10s enabled. I was hoping for the output of the profile you were testing with so that I could look for relevant differences.

Comment 23

[Mayank Bansal]

Attachment: aboutsupport.txt

https://hg.mozilla.org/mozilla-central/rev/c1f6345f2803

Comment 24

[Kris Maglione [:kmag]]

OK, thanks for the info.

I can't reproduce this reliably. For the moment, I'm going to contact the add-on authors about fixing the HTTP polling issues, which should solve the problem with this add-on.

I'll file a different bug for determining why that behavior causes leaks.

Comment 25

[Mayank Bansal]

Attachment: memory report without signing in the addon

memory report without signing in the addon. Just installed it, and enabled universal copy.

Comment 26

[Kris Maglione [:kmag]]

A new version of Pushbullet has been published which does not poll as aggressively.

Can you please verify that this solves the problem?

Thanks.

Comment 27

[Mayank Bansal]

Will test it.
And thanks for taking this up!

Comment 28

[Mayank Bansal]

Is there any way to speed up time in Nightly ? As in can i send some external signal/command to Nightly to speed up the time, so that the leaks that happen overnight happen in a few minutes ?

Comment 29

[Andrew McCreight [:mccr8]]

Unfortunately not.

Comment 30

[Mayank Bansal]

OK. I guess this would have helped diagnose slow leaks in Firefox.

Anyway, i will test this tonight, and report tomorrow.

Comment 31

[Mayank Bansal]

(In reply to Kris Maglione [:kmag] from comment #26)
> A new version of Pushbullet has been published which does not poll as
> aggressively.
> 
> Can you please verify that this solves the problem?
> 
> Thanks.

Verified fixed after leaving Nightly idle overnight.

Comment 32

[Kris Maglione [:kmag]]

Great. Thanks for all your testing.