Multiple gigabytes of tiny detached windows
Categories
(Core :: Performance, defect)
Tracking
()
People
(Reporter: Anon2057, Unassigned)
References
(Blocks 1 open bug)
Details
Attachments
(2 files)
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:125.0) Gecko/20100101 Firefox/125.0
Steps to reproduce:
Using various sites for long periods of time. keep Firefox open for multiple days at a time.
It cannot be troubleshooted in a short period. The ram balloons over a period of days to the crash occurring by around a week.'
original Addon list:
'Improve Youtube!'
600% sound volume
7TV
Auto Tab Discard
Better TTV
Libredirect
LiveTL
Privacy Badger
Tampermonkey (No scripts enabled)
Ublock
User-Agent Switcher and Manager
i'd say about 40 tabs, mix of:
Danbooru
Youtube
Kbin
Reddit
Amazon
game8.co
libreddit
Addon List on second test:
Ublock
600% sound volume
Currently running in troubleshoot mode, too soon to say how memory is currently.
I have whitelisted only youtube.com due to recommendations from other sites, but this was done during second test, memory still increased.
I have closed processes in about:processes. I noticed that GPU in particular remained at 1 GB. It is currently running at half of that.
I originally started looking into this 3 months ago. I have changed hardware since then and issues persist.
Actual results:
Memory usage increased. If ignored Firefox starts to skip on videos. If ignored further Firefox will hang and crash.
Skipping starts after a day or so, crash will usually be a week at the maximum
Expected results:
Memory should not have increased to this point. When reopening even if all open tabs are made active, the memory will be 1/2 to a quarter of the size.
I hope you use Ublock Origin and not Ublock. But lets see if it is an issue with "the bad Ublock"
|
||
Comment 2•7 months ago
|
||
The Bugbug bot thinks this bug should belong to the 'Core::Performance' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.
(In reply to Amanita from comment #1)
I hope you use Ublock Origin and not Ublock. But lets see if it is an issue with "the bad Ublock"
Yes Ublock Orgin
|
|
||
Comment 4•7 months ago
|
||
This bug was moved into the Performance component.
:Anon2057, could you make sure the following information is on this bug?
- For slowness or high CPU usage, capture a profile with http://profiler.firefox.com/, upload it and share the link here.
- For memory usage issues, capture a memory dump from
about:memoryand attach it to this bug. - Troubleshooting information: Go to
about:support, click "Copy raw data to clipboard", paste it into a file, save it, and attach the file here.
If the requested information is already in the bug, please confirm it is recent.
Thank you.
I will see if I can do this. I may be awhile as i'm trying to see if I can run it to the point of another crash. So far the memory is at around 6GB but the amount of tabs open hasn't changed much.
But so far I can run the profiler without any issues.
Stopped it today. It is now at 13GB
7GB up from the previous post.
I did the "save memory reports" as well, not sure if that was the correct option there. I'll keep the Firefox open in this state until tomorrow, but it usually crashes when it reaches 15-16GB.
|
||
Comment 9•6 months ago
|
||
Content process 14080 has 4 GB of detached window objects. Andrew, any idea what could be going on here?
Also, interesting to note from the profile that the GC is running a GCMajor every few seconds that is about 3s long and not freeing any memory. Maybe we need to reduce the rate at which this occurs if free'ing memory is hopeless?
|
|
||
Comment 10•6 months ago
|
||
You mentioned in comment 0 you started the browser in troubleshoot mode, was this still a problem in that case? I suspect there is a bad addon interaction here keeping a lot of these windows alive since you have so many active.
|
Reporter | |
Comment 11•6 months ago
|
||
(In reply to Denis Palmeiro [:denispal] from comment #10)
You mentioned in comment 0 you started the browser in troubleshoot mode, was this still a problem in that case? I suspect there is a bad addon interaction here keeping a lot of these windows alive since you have so many active.
The profiles and memory-reports are from while running in troubleshoot mode, so it occurred even when no addons were active.
|
||
Updated•6 months ago
|
|
|
||
Comment 12•6 months ago
|
||
A detached window means it is a DOM window that is not currently being displayed. Though it isn't a ghost window. I'm not sure exactly what the difference is. The process 14080 is very strange looking. There's almost 4GB of detached windows, but each individual page is no more than a few MB. There's a few MB of JS (probably the basic class stuff) and a tiny amount of DOM. Usually window leaks hold alive a ton of stuff, so I'm not sure why we're holding alive only this tiny residue of each page. Maybe the page has some kind of small iframe that is getting refreshed and it leaks the old one? It could be a problem with the webpage leaking stuff, or maybe it is doing something unusual that is triggering a leak in Firefox.
I'm not really sure what we can do here. I think the usual next step would be to try to reproduce the issue (not when there are so many detached windows, but when there are a small number of them) and get GC and CC logs. Note that these logs contain a huge amount of browser data (like all strings). For that, we'll need to know what page is leaking so badly.
|
||
Comment 13•6 months ago
|
||
Oops, forgot to click submit.
(In reply to Denis Palmeiro [:denispal] from comment #9)
Also, interesting to note from the profile that the GC is running a GCMajor every few seconds that is about 3s long and not freeing any memory. Maybe we need to reduce the rate at which this occurs if free'ing memory is hopeless?
Given that it says the GC heap is at 3.87GB, I'm guessing we're thrashing while nearing the max GC heap size (which is normally quite a bit less than the full memory usage). We could stop GCing in this situation, but we should probably figure out what we want in this sort of situation. It's clearly headed toward an out of memory crash eventually, and it seems like we should be more aggressive about giving up and crashing at least this process. The GC thrashing is making the observed behavior in the meantime much worse, so if the earlier crashing still allows this misbehavior, then we ought to do something about suppressing the GCs.
Oh, and looking at the memory report, the full GC heap size is right at 4GB (out of 7GB total, which is a higher fraction than I'd expect).
7,076.03 MB (100.0%) -- explicit
4,032.94 MB (100.0%) -- js-main-runtime-gc-heap-committed
Note that this is all about making the symptoms of the real underlying problem less bad. The underlying problem is the ever-climbing memory usage. Which seems to be coming from the 37,681 window objects and 3783 user realms. As long as those are building up, we can only make the eventual failure cause a bit less mayhem.
The profiles and memory-reports are from while running in troubleshoot mode, so it occurred even when no addons were active.
The memory report I'm looking at lists a bunch of addons active. Hm... oh, maybe those are system addons? They show up in the profile too. Yes, they must be the builtin search providers and things.
Can you share what URL is loaded in the largest content process? (The one shown in about:memory after doing "Measure", listed on the right hand side just under "Main Process".)
|
|
Reporter | |
Comment 14•6 months ago
|
||
webIsolated=https://youtube.com (pid 29612)
When I look at what is below it, it seems to show multiple youtube pages, and not just one of them, so i'm not sure which specifically it is pointing to.
This is the first part of it, but it goes on much longer. Let me know if you need more or anything in particular.
webIsolated=https://youtube.com (pid 29612)
Explicit Allocations
1,054.33 MB (100.0%) -- explicit
├────529.89 MB (50.26%) -- window-objects
│ ├──183.05 MB (17.36%) -- top(none)
│ │ ├──105.79 MB (10.03%) -- detached
│ │ │ ├──104.35 MB (09.90%) -- window(https://www.youtube.com/watch?v=e9xzs2BDZ-g)
|
|
Reporter | |
Comment 15•6 months ago
|
||
If it's of any help, I think I also still have the non-anonymonized memory report, because I think I downloaded both on that day.
This is the first one that shows on that one:
webIsolated=https://gamespot.com (pid 14080)
Explicit Allocations
7,074.38 MB (100.0%) -- explicit
|
|
||
Comment 16•6 months ago
|
||
(In reply to Anon2057 from comment #15)
webIsolated=https://gamespot.com (pid 14080)
Explicit Allocations
7,074.38 MB (100.0%) -- explicit
Ah! Yes, that's the one.
|
|
Reporter | |
Comment 17•6 months ago
|
||
Let me know if you need anything else from that tree
|
||
Updated•3 months ago
|
|
|
||
Comment 18•2 months ago
|
||
Removing perf impact flag and adding to memory investigation meta bu - please continue to provide additional details to help us remedy any possible issues
|
|
Reporter | |
Comment 19•2 months ago
|
||
If it helps, one thing I noticed is that over long periods of time, the about:cache never seems to free up. It also seems to frequently use the memory cache as little as possible unless the disk cache is disabled entirely.
I've extended the disk cache multiple times, I've tried running on memory cache exclusively (after a RAM upgrade). For the most part every change makes it last longer but going back to the cache no matter how high I make it it eventually caps out. It doesn't crash currently but that's after quadrupling the cache, and now firefox is going as far as 20+ GB before I decide to reboot it due to just slowing down.
I even noticed just now that even though there is a maximum storage size, the storage in use is over 4 times greater than it currently.
As of current:
Number of entries: 542677
Maximum storage size: 4194304 KiB
Storage in use: 17557275 KiB
Storage disk location: none, only stored in memory
|
||
Comment 20•2 months ago
|
||
(In reply to Anon2057 from comment #19)
If it helps, one thing I noticed is that over long periods of time, the about:cache never seems to free up. It also seems to frequently use the memory cache as little as possible unless the disk cache is disabled entirely.
I've extended the disk cache multiple times, I've tried running on memory cache exclusively (after a RAM upgrade). For the most part every change makes it last longer but going back to the cache no matter how high I make it it eventually caps out. It doesn't crash currently but that's after quadrupling the cache, and now firefox is going as far as 20+ GB before I decide to reboot it due to just slowing down.
I even noticed just now that even though there is a maximum storage size, the storage in use is over 4 times greater than it currently.
As of current:
Number of entries: 542677
Maximum storage size: 4194304 KiB
Storage in use: 17557275 KiB
Storage disk location: none, only stored in memory
Can you tell me what the value of browser.cache.memory.capacity is in about:config?
|
|
Reporter | |
Comment 21•2 months ago
|
||
It is currently 4194304, as noted by the max storage size
This is after various changes to it to try to make it a bit less laggy and crash less often. The defaults are much lower and when disk cache is available it will never use memory cache even if disk cache is maxed out.
|
|
||
Comment 22•2 months ago
|
||
Thanks for the response! I could see the information that you posted from about:cache but that value can also be set to be calculated at run-time based on total system memory, which also happens to be the default, so I wanted avoid assumptions regarding where that value was coming from.
I am wondering if there could be an overflow or unsigned/signed comparison issue here since 4194304 KiB is 4294967296 bytes which is exactly one more byte than can be counted using a 32-bit unsigned integer so that it what I am looking into right now.
|
|
||
Comment 23•25 days ago
|
||
Just to follow up briefly here, the memory cache referred to in "about:cache" seems to not be what you might expect it to be (it's not quite what I was expecting at least). Primarily it stores data from responses that are marked as "no-store". Data in this cache is generally kept around for only as long as is needed then is quickly flushed from the cache (usually due to expiration). In my own experiments, I can hardly find cases where it gets even up to 1000KiB before getting cleared out.
The limit specified by "maximum storage size" is also a soft limit - a purge is scheduled when we exceed that limit but it does not happen right away, so it is not unexpected to momentarily exceed that limit (which I was able to readily reproduce locally). There is, however, additionally a bug when the limit is set close to 4Gb where we effectively never think that we've exceeded the limit, which explains why you're able to exceed your limit by so much and stay there.
I am intrigued by the fact that enough data shows up and stays in the cache long enough for that broken limit-check to even be relevant though. That would seem to imply that something else strange is going on too. Like either so much no-store data shows up so quickly that we don't have time to purge it, or that the limit-checking and/or cache purging are broken for some other reason and we aren't properly cleaning things out.
@Anon2057: Just to be clear though - you were seeing the excessive memory usage before you increased browser.cache.memory.capacity, is that correct? Can you try reverting that value back to its default (-1 I believe)? I don't know that that changes anything but it likely isn't helping and the bug related to limit-checking when it's high introduces one more variable into the mix.
|
|
Reporter | |
Comment 24•25 days ago
|
||
Data in this cache is generally kept around for only as long as is needed then is quickly flushed from the cache (usually due to expiration). In my own experiments, I can hardly find cases where it gets even up to 1000KiB before getting cleared out.
That...doesn't seem right. Are you sure this isn't site dependent? I've had it capped out for weeks. Did you keep about:cache open long enough for the disk cache to show up? I noticed if the Disk cache is set to a large amount, it doesn't immediately show on the about:cache page, so it may appear that there's less than there actually is. Mine is particularly large so after about 5m the disk cache showed up on the page (currently 33991805 KiB in use)
This is after re-enabling disk cache after our previous discussion. Memory cache seems to only get used in excess if disk cache is disabled.
There is, however, additionally a bug when the limit is set close to 4Gb where we effectively never think that we've exceeded the limit, which explains why you're able to exceed your limit by so much and stay there.
What does the system do if your maximum storage size greatly exceeds 4GB?
Just to be clear though - you were seeing the excessive memory usage before you increased
Yes, all the changes made were done to alleviate the issue. Currently I can usually run firefox until it starts to frustrate me, but on the original settings Firefox was simply locking up completely and crashing.
Currently, i'm trying running Youtube in Chrome, and all my other tabs in Firefox (no other tabs in chrome). I'm noticing a large improvement. The RAM usage in chrome has remained stable, but firefox has been a lot of stable too. Are any tests being done with how Youtube uses the cache?
I can revert the settings if you still want after checking this.
|
|
Reporter | |
Comment 25•25 days ago
|
||
I checked some entries in the cache and noticed this:
170028463:https://www.youtube.com/youtubei/v1/live_chat/get_live_chat?prettyPrint=false
O^partitionKey=%28https%2Cyoutube.com%29, 2077 bytes 0 bytes 1 2024-10-30 17:25:26 2024-10-26 20:53:53
It looks like it should have expired like 4 days ago, but there's a significant number of these. Do you know why they would still be present?
there's hundreds of this entry, the cache entries are almost impossible to navigate it drags the entire PC down
|
|
||
Comment 26•24 days ago
|
||
Thank you for following up again! Those extra, expired entries are definitely suspicious but are perhaps consistent with that cache being "broken" in some way. My expectation is that there is more wrong here than just the bug where the threshold is effectively ignored when it is set high enough but I'd like to eliminate at least that variable so please do go ahead and reduce that value (probably restart your browser afterward too) and let's see if, when your memory issues come back, your cache looks to be behaving correctly or not.
|
|
Reporter | |
Comment 27•24 days ago
|
||
I've reverted my settings, i'll clear the needinfo when the issue resumes.
|
|
Reporter | |
Comment 28•23 days ago
|
||
So yesterday it was around 4GB for most of the day. Today it was 6GB. It skyrocketed by this time to around 11GB (noticed as the videos started to get choppy)
Caches are currently at
Memory
Number of entries: 6
Maximum storage size: 32768 KiB
Storage in use: 56 KiB
Storage disk location: none, only stored in memory
Disk
Number of entries: 30356
Maximum storage size: 1048576 KiB
Storage in use: 1047871 KiB
As mentioned previously, memory cache largely gets unused if Disk cache is turned on (the default)
|
|
Reporter | |
Comment 29•23 days ago
|
||
Confirmed in a different browser just to make sure it wasn't the person I was watching, firefox is definitely periodically skipping currently.
|
|
||
Comment 30•20 days ago
|
||
Thank you for the update! It is definitely a useful piece of information to see that the memory issues still occur even without the memory cache being crazy.
YouTube and gamespot have been mentioned in previous comments - when you were able to reproduce the problem this last time, were you just on those sites? And were you opening multiple simultaneous tabs? Watching different videos on different urls? Scrolling through video on a single url? Watching a livestream? Was there any other behavior that you can think? I'm really hoping that I can get to the point of reproducing this at some point, and it's entirely possible that I just have something setting/behavior different that we don't yet realize is relevant.
Also, if you wouldn't mind, the next time that you get the high memory usage (let's say at least 10Gb without there being an obvious), can you grab another memory report? I'd be interested to know, for example, if we still see large amounts of "detached window objects" somewhere.
|
|
Reporter | |
Comment 31•20 days ago
|
||
|
|
Reporter | |
Comment 32•20 days ago
|
||
Here you go
Of repeated tabs are usually
Danbooru images
Youtube (both videos and livestream)
On a day where nothing in particular is going on i'll usually go through youtube and reddit. If a livestream is scheduled for later in the day it'll be open in the background so I don't forget about it. Sometimes several livestreams.
Sometimes I'll bounce from videos in the same tab, but normally i'll open a bunch of tabs and then go through them. I do close most of them once i've gone through the video unless it's one of the earlier mentioned livestreams.
I'll also usually check a couple galleries on danbooru once a day to see if anything has changed, which usually involves opening a bunch of tabs, checking the images, favoriting some, and closing.
Youtube is generally the only one where the tab sometimes stays up afterwards.
Description
•