Closed Bug 1326095 Opened 7 years ago Closed 7 years ago

Content windows leak if content policies block XMLHttpRequests

Categories

(Core :: DOM: Core & HTML, defect)

defect
Not set
normal

Tracking

()

RESOLVED WORKSFORME
Tracking Status
firefox51 --- affected
firefox52 --- affected
firefox53 --- affected
firefox54 --- affected

People

(Reporter: pcunger, Assigned: mccr8)

References

(Blocks 1 open bug)

Details

(Whiteboard: see comment 24 for STR)

Attachments

(6 files, 1 obsolete file)

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36 OPR/42.0.2393.94

Steps to reproduce:

modest browsing (max 6 tabs open at one time) leads to extreme memory use. I'm currently up to 2,313,000 KB, and it keeps growing as I watch it (in the time it has taken to write this bug, it has grown to 2,329,000, and FF has--for all intents and purposes--been idle). Lenovo X230, Win 7-64, SSD, 8 GB RAM, Intel 4000 HD Graphics. I'm posting this using Opera out of desperation. I like FF--used it for years--but I'm getting ready to drop it...


Actual results:

switching from tab to tab leaves fragments of previous pages; ultimately, FF bogs down and goes 'Not Responding'; computer also hangs for minutes at a time while FF sorts out its brain...


Expected results:

memory use would remain minimal and stable
Hi Paul,

I tested this issue on Windows 7 x64 with FF 50 and FF 53.0a1 and I can't reproduce it. 

Could you please try to:

1. Please test if the issue can be reproduced in the safe mode of Firefox: https://support.mozilla.org/en-US/kb/troubleshoot-firefox-issues-using-safe-mode

2. Please download the Firefox Nightly from here: https://nightly.mozilla.org/ and retest the problem.

3. If you still have the issue please create a new profile, you have the steps here:https://support.mozilla.org/en-US/kb/profile-manager-create-and-remove-firefox-profiles?redirectlocale=en-US&redirectslug=Managing-profiles#w_starting-the-profile-manage



If nothing from the above helped you, can you please capture a performance profile? You can get more info about how to install and use the Cleopatra add-on (that helps you get the performance profile) by going to:
https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Profiling_with_the_Built-in_Profiler
Flags: needinfo?(pcunger)
(In reply to ovidiu boca[:Ovidiu] from comment #1)
> Hi Paul,
> 
> I tested this issue on Windows 7 x64 with FF 50 and FF 53.0a1 and I can't
> reproduce it. 
> 
> Could you please try to:
> 
> 1. Please test if the issue can be reproduced in the safe mode of Firefox:
> https://support.mozilla.org/en-US/kb/troubleshoot-firefox-issues-using-safe-
> mode
> 
> 2. Please download the Firefox Nightly from here:
> https://nightly.mozilla.org/ and retest the problem.
> 
> 3. If you still have the issue please create a new profile, you have the
> steps
> here:https://support.mozilla.org/en-US/kb/profile-manager-create-and-remove-
> firefox-profiles?redirectlocale=en-US&redirectslug=Managing-
> profiles#w_starting-the-profile-manage
> 
> 
> 
> If nothing from the above helped you, can you please capture a performance
> profile? You can get more info about how to install and use the Cleopatra
> add-on (that helps you get the performance profile) by going to:
> https://developer.mozilla.org/en-US/docs/Mozilla/Performance/
> Profiling_with_the_Built-in_Profiler

Thanks for testing, ovidiu boca.

Safe mode may have helped a bit? At one point, with four tabs open, memory was climbing, but then I clicked a link on one of the pages and it stopped. So now I guess I need to work through my add-ons to see which one is the problem? Ugh...

I'm not sure I want to go the 'nightly' route: "Enjoy at your own risk" isn't what I look for in a browser... :-(

I may reset my profile, but I'm not sure what I'll lose; if it's resetting things, I'm going to lose something, right?
Hi,

Thanks for sending feedback, Firefox Nightly is:
 Nightly - Under heavy development. Least stable/secure. First tests of new changes/features; some changes/features introduced in Nightly may be removed before Release and other versions. Only for testing. Should only be used by very experienced users/testers.

You can install Nightly and give it a try and then uninstall it, but like you said if in safe mode you saw some changes it will be a good idea to try and disable one by one the add-ons installed on your profile and see witch one is causing the problem.
I don't know what happened, but after FF being really stable for quite a while, now I have to restart it every 1-2 days, or even twice a day.  Memory usage shoots to 2GB rapidly, and then it becomes very laggy and unresponsive, hanging often for 500-2000mS until restarted.  This has started somewhere in the Firefox 48 / 49 / 50 time frame, not exactly sure when.  And it happens on Linux (openSUSE x64) and Win7 x64.  This is the closest matching bug report I could find.  *Just one time* closing all tabs caused memory to drop like a rock.  But all other times, closing all tabs to a single new empty tab, still stuck at 2GB (right now, 3.2GB).  Top notch custom gaming system w/16GB system ram.
Hi Jayrusman,

Can you please capture a performance profile? You can get more info about how to install and use the Cleopatra add-on (that helps you get the performance profile) by going to:
https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Profiling_with_the_Built-in_Profiler
Flags: needinfo?(jayrusman)
I have exactly the same issue as in the comment 4, so much for now, I'll try to paste some more technical info later.

Firefox 50.1.0 32bit, Win 7-64bit SP1, Intel Core i3-2100, 4GB RAM DDR3, Samsung EVO 850 SSD, Intel 2000 HD Graphics, nothing reasource eating running in the background.
Thanks  marcino245 for your feedback, can you please try the steps from comment 1 and if nothing changes please try to capture a cleopatra profile, you have the steps in comment 5. Thank you.
As for now some debug logs with Firefox eating 1.3GB RAM with a single tab.
I closed all other tabs, leaving just about:support tab, and waited 5 minutes, and it eats 1.3GB RAM, it seems that Firefox doesn't free it's RAM memory.
Mike can you please take a look? Thanks
Flags: needinfo?(mconley)
According to the about:memory report in the 7z file, several hundred MB are being held by "ghost windows", likely by an add-on.

marcino245: you have a number of enabled add-ons. I recommend temporarily turning them off to see if that reduces the amount of memory Firefox uses over time. If so, then slowly start introducing them back again until you identify the one that is causing the leak. Unfortunately, that's the best technique we have right now to identify leaking add-ons. :/
Flags: needinfo?(mconley) → needinfo?(marcino245)
> According to the about:memory report in the 7z file, several
> hundred MB are being held by "ghost windows", likely by an add-on.

Commenter from #4 here, this is what I saw, lots memory locked up in "ghost windows".  I have disabled all add-ons and am searching for the offender.  Problem is, disabling ABP has lead to having to kill Firefox when I bump into one of those pages that give endless password-req'd dialog box (rendering Firefox useless) while claiming PC is infected ... can't Firefox limit password dialog popups like other dialog popups?
(In reply to ovidiu boca[:Ovidiu] from comment #7) &&
(In reply to Mike Conley (:mconley)  - PTO on Jan 20th from comment #11)

Thank you guys, yes I know, I'll do that, I just wanted to create some debug logs before, I also now did run "Minimize memory usage" button from about:memory tab, and created memory-report_after_minimize_memory_usage.json.gz, which I included into the updated  attachement. After the comparision it seems that it regained only 73MB. Now I'll restart Firefox and conduct the investigation as you guys suggested.
Attachment #8828304 - Attachment is obsolete: true
Hey jayrusman, I have noticed your comment #12 just right now because you have started writing your comment when I was already in the process of writing my comment #13. It's easier for you now to deduce which add-on cause the memory leak, because you have list of my add-ons, so probably an add-on causing the memory leak will be the same one we both have, so could you please past from about:support a list of your add-ons too? Thanks.
It seems that Bug 1330860 - Memory Leak is a dupe of this bug.
I have traced it, the add-on causing ghost windows & memory usage growing is Adblock Plus 2.8.2 (with EasyList enabled), that's Bug 1281728, in that bug Wladimir Palant has declared that the bug has been already fixed, but it doesn't seems so, but if it does, it means that the bug has returned.
Flags: needinfo?(marcino245)
I agree, on FF Win7x64 I [had] many add-ons, but the offending FF on Linux only has ABP and Private Tab, and I'm guessing Private Tab is extremely simple.
OK, therefore it seems that this bug should be closed as a duplicate of Bug 1281728, and we should reopen https://issues.adblockplus.org/ticket/4203.
Component: Untriaged → Add-ons
Product: Firefox → Tech Evangelism
Version: 50 Branch → Trunk
Summary: extreme memory use → Ghost windows with Adblock Plus
(In reply to marcino245 from comment #18)
> OK, therefore it seems that this bug should be closed as a duplicate of Bug
> 1281728, and we should reopen https://issues.adblockplus.org/ticket/4203.

This being another memory leak somehow related to Adblock Plus doesn't make it a duplicate. This is definitely an entirely different issue.

I see entries in CC log like the following:

> 0x7f5865997a00 [rc=2] FragmentOrElement (xhtml) img id='site-logo' http://example.com/
> > 0x7f587dc67430 Preserved wrapper
>
> 0x7f587dc67430 [gc] JS Object (HTMLImageElement)
> > 0x7f5865997a00 UnwrapDOMObject(obj)

There are no further entries relating to these addresses and GC log shows 0x7f587dc67430 being a root:

> 0x7f587dc67430 G Preserved wrapper
>
> 0x7f587dc67430 G HTMLImageElement <no private>
> > 0x7f586013bfa0 G group
> > 0x7f5881a2d2e0 G shape

From what I can tell, there is no way to get further information from the logs - e.g. what kind of wrapper that is and why it is rooted. If somebody knows more than me, any hints are welcome. As it stands now, I cannot really see how an extension can possibly produce a leak like that and will have to proceed by eliminating functionality which is quite tedious.
> From what I can tell, there is no way to get further information from the logs - e.g. what kind of wrapper that is and why it is rooted.

You should run find_roots.py on the GC log with the -bro option. (And I should really set that to be the default...) By default, it includes things like C++ roots which are not very interesting most of the time.
I sort of gave up on find_roots.py a while ago, it has issues with large log files (365 MB this time). But I will try it...
See Also: → 1334367, 1335482
I didn't manage to get find_roots.py to complete without exhausting my memory but I successfully tracked this issue down to blocking XMLHttpRequests via content policies (only reproducible if E10S is disabled). This can be reproduced if all filters in Adblock Plus are disabled and the custom filter "*$xmlhttprequest,domain=youtube.com" is added instead - any YouTube video page will leak then (the video itself will be broken but that's expected). The leak seems to be somewhere in the Firefox code, I will try to create a minimal extension to reproduce this without Adblock Plus.

For reference, I can reproduce in Nightly once E10S is disabled.
With the attached minimal extension the issue can be reproduced without having Adblock Plus installed.

Steps to reproduce:

1. In Nightly, make sure that "Enable multi-process Nightly" is switched off in Preferences and xpinstall.signatures.required is set to false in about:config.
2. Install attached extension.
3. Open about:memory, click "Minimize memory usage", check "verbose" and click "Measure" - there should be no ghost windows under explicit / window-objects / top.
4. Open https://www.youtube.com/watch?v=NxGzTBXyDbw (random YouTube video), let the page load. The video doesn't start, this is expected. Close this
5. Now close the YouTube video again.
6. On about:memory click "Minimize memory usage," then "Measure."

Expected results:

The YouTube video page doesn't appear explicit / window-objects / top, it has been garbage collected correctly.

Actual results:

The YouTube video page still appears under explicit / window-objects / top, first as detached, then as ghost window (reproduced in Firefox 51 and current Firefox 54.0a1 nightly on Ubuntu). This memory leak isn't in the extension - it does nothing but block all XMLHttpRequests via content policies. Something in the way Firefox processes blocked XMLHttpRequests is causing this leak.
Status: UNCONFIRMED → NEW
Component: Add-ons → DOM
Ever confirmed: true
Flags: needinfo?(pcunger)
Flags: needinfo?(jayrusman)
Product: Tech Evangelism → Core
Summary: Ghost windows with Adblock Plus → Content windows leak if content policies block XMLHttpRequests
Whiteboard: see comment 24 for STR
Thanks for the STR. That sounds suspiciously like some other XHR-related leaks we've had. Kris, do you have time to look at this?
Flags: needinfo?(kmaglione+bmo)
When I was looking into the other leaks, I did come across a few places where the lack of cleanup code made me suspicious. In particular:

http://searchfox.org/mozilla-central/rev/d20e4431d0cf40311afa797868bc5c58c54790a2/dom/xhr/XMLHttpRequestMainThread.cpp#2087-2094

we don't null out the mChannel property or clear the channel's notification callbacks, which point to the XHR object and create a reference cycle. There are a few other places that seem to have similar issues.
Flags: needinfo?(kmaglione+bmo)
Assignee: nobody → kmaglione+bmo
uBlock implements blocking via HTTP observers, from what I know only some requests are blocked via content policies, so the behavior there is expected to be different. It's likely the same bug however.
> uBlock ghost-windows.7z

I can't reproduce with uBlock Origin + other repro steps (disabling multiprocess, etc.) I can reproduce immediately with the minimalist add-on provided at #24.

The data in "uBlock ghost-windows.7z" shows that many other extensions were present. If the goal was to find out whether the issue also exists with uBO, then only uBO should have been running -- since I can't reproduce as with the same repro steps as the minimalist extension, this suggests a memory leak issues could also be present in one of the other extensions in the report.

The fact that I could not reproduce with uBO might be a hint that the issue is specific to blocking the XHR through nsIContentPolicy -- uBO blocks through HTTP observer. Confirmation of this could help narrow where to look in Firefox code for the leak.
Hello, you are right, I have 8 add-ons enabled currently, I have been just more supposing that uBlock Origin also has been affected like AdBlock Plus than definitely claiming it, it has just seemed to me that the most likely it is uBlock Origin being affected. Sorry for not signaling it and taking your time, anyway you have done good work, it is good to know that uBlock is (most likely) not affected. As for the long 48h Firefox webbrowsing session I'm currently running, I am having at the moment 3 ghost windows, the 3rd ghost window has the same url like the 2 previous ones. It seems that Firefox is still generating a little leak with some add-on(s), but this leak is not having a noticeable impact on browsing at the moment, but after another several days of webbrowsing, the ammount of ghost windows might grow up to the level that the impact might begin to be noticeable. But you are right, there is no sense anymore in keeping running the session with 8 add-ons wherein we don't know which one(s) is/are being affected, and also the 48h session has already proved that whatever the leak is now, it is a slow and little leak compared to the leak with AdBlock Plus enabled, at least I can comfortable browse internet.
Thanks to all who have taken this seriously! I appreciate your hard work.

Just wondering: I switched to BluHell adblocker (https://addons.mozilla.org/en-US/android/addon/bluhell-firewall/?src=api) some time ago--many months before reporting this issue. Before that I /had/ used AdBlock Plus. Anyway, do you think BluHell is of the same architecture as AdBlock Plus? Would it cause the same kinds of problems?
(In reply to paul from comment #31)
> Just wondering: I switched to BluHell adblocker
> (https://addons.mozilla.org/en-US/android/addon/bluhell-firewall/?src=api)
> some time ago--many months before reporting this issue. Before that I /had/
> used AdBlock Plus. Anyway, do you think BluHell is of the same architecture
> as AdBlock Plus? Would it cause the same kinds of problems?

Yes, possibly. It also uses content policies, so it is potentially affected. It has a hardcoded list of domains that it blocks, and the question is mostly whether these will match any problematic XMLHttpRequests (my testing indicated that not every XMLHttpRequest blocked results in a memory leak but I didn't investigate further).
Blocks: GhostWindows
This looks like a dupe of bug 1336811. When I apply the patch from bug 1336811, I can no longer reproduce the leak by following the steps in comment 24.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → DUPLICATE
If you could confirm that the issue you were seeing with AdblockPlus is fixed in Nightly (it probably requires the 2/21 Nightly or newer) I'd appreciate it. I want to make sure the initial issue is really fixed.
Flags: needinfo?(trev.moz)
> If you could confirm that the issue you were seeing with AdblockPlus is fixed in Nightly

For myself I tried nightly Feb 22 Win64 with ABP, it *seems* better so far.  It peaked around ~1GB (sum of 3 processes) so far that I noticed, and it drops to 300MB with all the tabs closed, and I did blast through a bunch of sites on purpose.  I did get one notice that "ABP may be slowing nightly down" (triumphrat.net motorcycle forum) but I don't have any UI performance complaints, it seems snappy.
Attached file ghost_windows.7z
The ghost windows are still there - I today have run a several hours session on Nightly 53.0a2 (2017-02-22) (32bit) on the same profile with the same several add-ons enabled and with multi-process enabled by default too and I have had 19 ghost windows which most probably would grow with time.
(In reply to marcino245 from comment #36)
> The ghost windows are still there - I today have run a several hours session
> on Nightly 53.0a2 (2017-02-22) (32bit) on the same profile with the same
> several add-ons enabled and with multi-process enabled by default too and I
> have had 19 ghost windows which most probably would grow with time.

Could you please file a new bug for your issue and needinfo me or link to it here? We'll have to figure out which addon is causing the issue. Thanks.
> Could you please file a new bug for your issue and needinfo me or link to it
> here? We'll have to figure out which addon is causing the issue. Thanks.
Flags: needinfo?(marcino245)
I have been aware of you comment 37 and have not been ignoring it, just decided to conduct my own investigation without filing a new bug so hurry: I have disabled all add-ons except AdBlock Plus, restarted the browser (it self-updated to 53.0a2 20170223004018) and run a new browser session, and after a dozen of hours the ghost windows still appear, 6 ghost windows already and incerasing with time. Now the question is: is it still the same bug not fixed completely or it is another "Firefox + AdBlock = ghost windows" bug with the same symptoms? I remember that someone has said that the issue occurs only with multi-process disabled, now I have proved that the same symptoms are present with multi-process enabled too. So now you guys should decide if you still want me to fil a separate bug or not.
Flags: needinfo?(marcino245)
Fixing the typos: "aware of you" = aware of youR", "it self-updated = "it HAS self-updated".
I have noticed that even if I have AdBlock Plus disabled on some websites (ABP icon grayed out), I still see ghost windows's urls of these websites in about:memory.
OK, I have enabled back my set of add-ons + switched from AdBlock Plus to uBlock Origin, restarted the browser and most of ghost windows have dissapeared, but after several hours still single ghost windows are generated, it seems that's still the main bug not completely fixed. 

Also an another strange thing is that this time the single ghost window's url is...about:blank, wherein I have not visited about:blank manually in this session nor in any session, but as I remember in Firefox when a pop-up window appears while webbrowsing on some websites, first about:blank is loaded on the fly in the pop-up window and then it is quickly switched with the real content (webpage), thus this is most probably how about:blanks have even appear in Firefox. 

And again like I have already stated in my previous post, about:blank is not blocked by any ads blocker (uBlock in this case) and I think no add-on works on about:blank, however still it might be that add-ons (mostly ads-blockers) still process the about:blank even if they don't block it or don't do any action on it.


──0.04 MB (00.02%) -- ghost/window(about:blank)
│   │      ├──0.04 MB (00.01%) -- js-compartment(https://accounts.google.com/ServiceLogin?service=mail&passive=true&rm=false&continue=https://mail.google.com/mail/&ss=1&scc=1&ltmpl=default&ltmplcache=2&emr=1&osid=1, about:blank)
│   │      │  ├──0.03 MB (00.01%) -- classes
│   │      │  │  ├──0.03 MB (00.01%) -- class(Function)/objects
│   │      │  │  │  ├──0.03 MB (00.01%) ── gc-heap
│   │      │  │  │  └──0.00 MB (00.00%) ── malloc-heap/slots
│   │      │  │  └──0.01 MB (00.00%) -- class(<non-notable classes>)/objects
│   │      │  │     ├──0.00 MB (00.00%) ── malloc-heap/slots
│   │      │  │     └──0.00 MB (00.00%) ── gc-heap
│   │      │  └──0.00 MB (00.00%) ++ sundries
│   │      ├──0.00 MB (00.00%) -- dom
│   │      │  ├──0.00 MB (00.00%) ── other
│   │      │  └──0.00 MB (00.00%) ── element-nodes
│   │      └──0.00 MB (00.00%) ── style-sheets

1 (100.0%) -- ghost-windows
└──1 (100.0%) ── about:blank

 1 ── ghost-windows
(sorry, I have forgotten to uncover ++sundries, and also btw I am attaching the about:support log)
I'll install ABP and see if I can reproduce the ghost windows. Does it seem like any sites in particular are coming up more often, like YouTube or something else? What version of ABP are you using?

You are right about about:blank: it is always loaded when you open a new tab, then the actual page is loaded. I haven't seen that kind of ghost window before, where the window and the compartment don't match.
Assignee: kmaglione+bmo → continuation
Status: RESOLVED → REOPENED
Depends on: 1336811
Resolution: DUPLICATE → ---
It doesn't seem that there are any particual websites on which the ghost-windows appear more often, the ghost windows might appear randomly on any website for. ex. in my case: allegro.pl (polish eBay equivalent), tvn24.pl (polish news), youtube. ABP 2.8.2.
Depends on: 1343353
Quoting my comment 42 I can add that I have now switched back from uBlock Origin to AdBlock Plus (while still keeping the other several add-ons enabled) and have ran a 12 hours test session on 53.0a2 20170303004003, and the ghost windows are gone, except the single about:blank ghost window as in comment 42, which is rarely appearing and dissapearing, but doesn't affect neither the browser's performance nor memory usage. I can say that the ghost windows issue with AdBlock Plus has been fixed at 90% by one of recent Developer Editions, and I think that there are only some cosmetic chunks remaining, at the moment it looks like:

├───31.67 MB (10.17%) -- window-objects
│   ├──17.01 MB (05.46%) ++ top(about:memory, id=262)
│   ├───7.71 MB (02.48%) -- top(chrome://browser/content/browser.xul, id=5)
│   │   ├──5.63 MB (01.81%) -- active
│   │   │  ├──5.53 MB (01.78%) ++ window(chrome://browser/content/browser.xul)
│   │   │  └──0.10 MB (00.03%) -- window(about:blank)
│   │   │     ├──0.09 MB (00.03%) -- js-compartment([System Principal], about:blank)
│   │   │     │  ├──0.08 MB (00.03%) -- classes
│   │   │     │  │  ├──0.07 MB (00.02%) -- class(Function)/objects
│   │   │     │  │  │  ├──0.07 MB (00.02%) ── gc-heap [2]
│   │   │     │  │  │  └──0.00 MB (00.00%) ── malloc-heap/slots [2]
│   │   │     │  │  └──0.01 MB (00.00%) -- class(<non-notable classes>)/objects
│   │   │     │  │     ├──0.01 MB (00.00%) ── malloc-heap/slots [2]
│   │   │     │  │     └──0.00 MB (00.00%) ── gc-heap [2]
│   │   │     │  └──0.01 MB (00.00%) -- sundries
│   │   │     │     ├──0.01 MB (00.00%) ── malloc-heap [2]
│   │   │     │     └──0.00 MB (00.00%) ── gc-heap [2]
│   │   │     ├──0.01 MB (00.00%) -- dom
│   │   │     │  ├──0.01 MB (00.00%) ── other [4]
│   │   │     │  ├──0.00 MB (00.00%) ── element-nodes [2]
│   │   │     │  └──0.00 MB (00.00%) ── event-targets [2]
│   │   │     ├──0.00 MB (00.00%) ── style-sheets [2]
│   │   │     └──0.00 MB (00.00%) ── property-tables [2]
│   │   └──2.08 MB (00.67%) ++ js-zone(0x12db8800)
│   └───6.95 MB (02.23%) ++ (4 tiny)

1 (100.0%) -- ghost-windows
└──1 (100.0%) ── about:blank

 1 ── ghost-windows
Thanks for the update. I'm going to close this bug for now. Please file a new bug and needinfo me if you see a new issue with ghost windows.
Status: REOPENED → RESOLVED
Closed: 7 years ago7 years ago
Flags: needinfo?(trev.moz)
Resolution: --- → WORKSFORME
Fantastic work, marcino245! I'm very grateful for people like you. :-) Looking forward to when this gets released!
Component: DOM → DOM: Core & HTML
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: