High memory usage when searching on https://docs.sqlalchemy.org
Categories
(Core :: Performance: General, defect)
Tracking
()
People
(Reporter: mozbug, Unassigned)
Details
Attachments
(3 files, 1 obsolete file)
User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:84.0) Gecko/20100101 Firefox/84.0
Steps to reproduce:
On some occasions, memory usage sky-rockets (GBs of usage) while loading some content.
One location that I can reliably reproduce this, is the search function on sqlalchemy docs. For example:
https://docs.sqlalchemy.org/en/14/search.html?q=asyncio&check_keywords=yes&area=default
Actual results:
While the search results are loading in, the memory usage sky-rockets. Within about 5 seconds the memory usage is 2+ GB, at which point my system has run out of memory and freezes, eventually crashing the tab.
Expected results:
A manageable amount of memory usage, that doesn't result in the tab crashing.
I recall this also happening with the recommended videos on the side of Youtube videos. It's less significant, but still required around 1-2GB of free RAM to successfully load the page (memory usage drops once loaded). I have simply blocked those elements with adblock since then.
|
||
Comment 2•5 years ago
|
||
Hey Sam,
I tried reproducing this on the latest versions of Firefox Nightly Nightly 86.0a1 (2020-12-28), beta 85.0b4 and release 84.0.1 but the memory usage didn't go past 400 mb for us.
Can you test the issue while in Safe Mode? You can find helpful info here : https://support.mozilla.org/en-US/kb/troubleshoot-firefox-issues-using-safe-mode .
Also a fresh new profile could help. You can find more about creating a new profile here : https://support.mozilla.org/en-US/kb/troubleshoot-and-diagnose-firefox-problems#w_6-create-a-new-firefox-profile .
If possible, you can test this issue on the nightly build as well. Download the build from : https://www.mozilla.org/en-US/firefox/nightly/all/ .
I tried safe mode and a new profile, both times remained 100% reproducible.
Opening the link in the report resulted in my system RAM usage rising from 4-5GB to around 7GB at which point it ran out of memory and froze.
This is the standard Firefox that is installed on Ubuntu (i.e. from apt). Currently v84, but I've had the issue for many months if not years. So, it's not specific to v84.
|
|
||
Comment 4•5 years ago
|
||
Setting a component for this issue in order to get the dev team involved.
If you feel it's an incorrect one please feel free to change it to a more appropriate one.
Sam, can you please create a Gecko profile via https://profiler.firefox.com/? Thanks.
Also the a saved memory report from about:memory would be helpful. Please attach one to this bug.
During the issue? That's going to be challenge, I have about a 5 second window before the entire system freezes...
OK, I'm not sure exactly what you mean by a Gecko profile. I enabled the menu button, selected "Firefox Platform" and started recording. I then opened the sqlalchemy link in a new tab, closed it just before it froze my system, then clicked the capture button. In the view it presented I click upload, then download. I'm attaching the file it gave me, is this what you want?
I also used the measure and save option in about:memory, while it was loading that page again, so hopefully that contains enough to show the issue.
|
Reporter | |
Comment 10•5 years ago
|
||
|
|
Reporter | |
Comment 11•5 years ago
|
||
Also, how do I disable the profiler again?
|
||
Comment 12•5 years ago
|
||
The memory report doesn't show high memory use, is that perhaps taken after some memory has been released already?
Or where do you see high memory use?
|
|
Reporter | |
Comment 13•5 years ago
|
||
I clicked the button to save the report and it completed a couple of seconds before the system froze due to running out of memory. The system usage must have been 2-4GB at the time I pressed the button.
I have no idea what's in the report, but I was watching the system RAM usage while the page was loading and it was definitely high when I clicked the save button.
Btw here is the gecko profile as attached on comment 9: https://share.firefox.dev/2YBAZA9
As it looks like there is a lot of activity going on in graphics. But not sure how this is related to a possible high memory usage.
Sam, could you maybe profile again but change the settings before? Therefor click the small arrow button next to the toolbar icon, and within the setting enable JS and native allocations.
|
|
Reporter | |
Comment 15•5 years ago
|
||
I can try JS allocations tomorrow, but native allocations is disabled.
|
|
Reporter | |
Comment 16•5 years ago
|
||
Took me 3 attempts to get this one. With JS allocations:
https://share.firefox.dev/2NNZHuR
I have a hard time to actually load that profile. Most of the time it stalls during load. But finally I was able to open it. Sadly also this profile doesn't show high memory usage for any of the uploaded web content processes.
So here some suggestions:
- Can you please test it again with a fresh Firefox profile without any extensions installed?
- When you see high memory usage in about:memory, save a measurement with anonymize turned off.
- When profiling in that fresh profile please submit everything and not anonymizing the data.
|
||
Comment 18•5 years ago
|
||
Be aware that not anonymizing the memory usage will include the URLs of every tab you have open. It is okay to upload an anonymized report. It sounds like the issue was that the memory usage wasn't high. Unfortunately, in situations where the memory usage suddenly spikes up about:memory reports are not useful.
Sorry when it was unclear, but step 2 was meant to be run for the fresh profile. It's clear which web page this bug is about, so there wouldn't be a need to anonymize the data.
|
||
Comment 20•5 years ago
|
||
The profile in comment 16 is a bit bizarre - I see in the network track for the parent process that the SQL Alchemy site is being loaded (and I see the screenshots for it), but I can't find the associated content process (having made all of them visible). Perhaps the data from the content process ran out of buffer space and we overwrote it? Pretty strange.
At any rate, what I do see are other sites loaded in the background that are making this analysis kind of difficult. I see YouTube, ig.com, document360.io, GitHub, SalesForce...
If this is something that is easily reproducible, is it possible to capture it:
- On a new profile
- Without any other background tabs or windows opened
- As soon as the memory usage spikes
I think (hope) that will help us find the signal amongst the noise here and figure out what's happening.
|
|
Reporter | |
Comment 21•5 years ago
|
||
OK, completely fresh .mozilla/firefox/ directory. I repeated the profiler again. The memory spiked to 4+ GB before I closed the tab, I then waited for the memory to drop again before stopping the profiler.
https://share.firefox.dev/3tBixG6
I also tried the about:memory thing again, but it seems that when testing completely fresh (and no other applications open) I had enough memory for it to load successfully and the memory spike was very quick, so I was probably unable to to catch the memory spike. From this though, it appeared that I even if I pick a search term with a large number of results, the memory actually spikes only at the beginning of the load (still around 2+GB in this test) before dropping back down, and then continues to load more results without any further issues.
|
|
Reporter | |
Comment 22•5 years ago
|
||
(In reply to Sam Bull from comment #21)
OK, completely fresh .mozilla/firefox/ directory. I repeated the profiler again. The memory spiked to 4+ GB before I closed the tab, I then waited for the memory to drop again before stopping the profiler.
When I was checking this profile I noticed that the memory bar is missing from each of the processes. As I got informed it requires a build config that is not set for release / beta builds. Sorry to ask again, but could you maybe record it again while running a Firefox Nightly build?
|
||
Comment 24•5 years ago
|
||
I think I am able to see what is happening here, although it didn't OOM for me.
If you just enter a very simple search term, like just trying the letter a, the site will keep searching and grow the list indefinitely, thus the memory sky-rocks.
Here's a profile https://share.firefox.dev/3twXF2u ( I enabled memory allocation tracking ). I don't see any GC/CC in my profile, the site was busy doing the search I believe. Looks like the majority of the memory usage was created the rAF callback. Can someone with more advanced profiler analysis skills take a look?
I actually tried myself and for me it's more a high CPU load when doing a search for a. While the parent process shows an around 40% CPU usage the web content process has 100% CPU load all the time. Memory usage doesn't really grow that much for me.
Here my profile: https://share.firefox.dev/2MSArmT
As it looks like the function css as run by FrameRequestCallback causes lots of time spend in nsCounterList::SetScope. But maybe that's expected given a lot of search results are getting added to the page? Emilio, what do you think?
The same function is also causing most of the memory allocations for me. And I think that's similar to Sean's profile.
|
||
Comment 26•5 years ago
•
|
||
Yeah so this is kinda sad. The page has some sort of "animation" which relies on for some reason setting then unsetting overflow: hidden on each <li> element that appends to the DOM. This is done from here:
0 S.Animation<.prefilters<(e = "[object HTMLLIElement]", t = "[object Object]", n = "[object Object]") ["https://docs.sqlalchemy.org/en/14/_static/jquery.js":2:61155]
this = [object Object]
1 ft(o = "[object HTMLLIElement]", e = "[object Object]", t = "[object Object]") ["https://docs.sqlalchemy.org/en/14/_static/jquery.js":2:60470]
2 a([function], "[object Object]") ["https://docs.sqlalchemy.org/en/14/_static/jquery.js":2:63163]
this = [object HTMLLIElement]
3 dequeue(e = "[object HTMLLIElement]", t = ""fx"") ["https://docs.sqlalchemy.org/en/14/_static/jquery.js":2:35164]
this = function(e,t){return new S.fn.init(e,t)}
4 queue/<("0", "[object HTMLLIElement]") ["https://docs.sqlalchemy.org/en/14/_static/jquery.js":2:35624]
this = [object HTMLLIElement]
5 each(e = "[object Object]", t = [function]) ["https://docs.sqlalchemy.org/en/14/_static/jquery.js":2:2975]
this = function(e,t){return new S.fn.init(e,t)}
6 each(e = [function]) ["https://docs.sqlalchemy.org/en/14/_static/jquery.js":2:1453]
this = [object Object]
7 queue(t = ""fx"", n = [function]) ["https://docs.sqlalchemy.org/en/14/_static/jquery.js":2:35529]
this = [object Object]
8 animate(t = "[object Object]", e = "5", n = [function], r = "undefined") ["https://docs.sqlalchemy.org/en/14/_static/jquery.js":2:63278]
this = [object Object]
9 S.fn[e](e = "5", t = [function]) ["https://docs.sqlalchemy.org/en/14/_static/jquery.js":2:64490]
this = [object Object]
10 displayNextItem() ["https://docs.sqlalchemy.org/en/14/_static/searchtools.js":276:19]
this = [object Window]
11 query(query = ""asyncio"") ["https://docs.sqlalchemy.org/en/14/_static/searchtools.js":311:4]
this = [object Object]
12 performSearch(query = ""asyncio"") ["https://docs.sqlalchemy.org/en/14/_static/searchtools.js":146:11]
this = [object Object]
13 init() ["https://docs.sqlalchemy.org/en/14/_static/searchtools.js":79:15]
this = [object Object]
14 anonymous([function]) ["https://docs.sqlalchemy.org/en/14/_static/searchtools.js":513:9]
this = [object HTMLDocument]
15 e() ["https://docs.sqlalchemy.org/en/14/_static/jquery.js":2:30004]
16 l/</t<() ["https://docs.sqlalchemy.org/en/14/_static/jquery.js":2:30306]
this = [object Window]
So changing an element to be overflow: hidden from overflow: visible changes the layout box structure. Since they're changing it to a <li> element, which has a CSS counter, destroying its layout box dirties the counter list structure. So we end up recomputing it over and over since the page also reads out styles, so forces us to do the recalc basically sync.
There's no particularly easy fix from our side. We could try to make counter updates more incremental maybe, or maybe detect box reconstruction more easily and update the list in place... none of those are particularly easy.
I bet this regressed with bug 288704.
Emilio could the above also be the cause of the high memory usage? Actually this was the original reason why the bug has been filed.
Also regarding the animation stuff, it sounds like an issue our webcompat team could maybe get in contact with those folks so that improvements directly for the website could be made.
|
|
Reporter | |
Comment 28•5 years ago
|
||
Note that the search is implemented by Sphinx, so if you have an easy fix it could be done there. That'd be good enough for me.
I previously opened an issue about this there:
https://github.com/sphinx-doc/sphinx/issues/8562
|
|
||
Comment 29•5 years ago
|
||
I don't think so, or at least I don't see how. We're creating and destroying a lot of objects internally, but that's just wasted work, shouldn't grow memory usage. It might be that the thousands of jQuery animations are leaking or what not, but that seems also somewhat unlikely.
I sent an Sphinx fix here: https://github.com/sphinx-doc/sphinx/pull/8868
|
|
Reporter | |
Comment 30•5 years ago
|
||
The fix in Sphinx has resolved the issue on docs.sqlalchemy.org for me. I can no longer see any notable increase in memory during load, and the search results are a lot faster. Thanks for that.
As I can no longer reproduce it, I'll close the issue, but feel free to reopen if you're still thinking of making changes to Firefox.
(Although I did have a similar experience with the recommended sidebar on Youtube videos, which I've simply blocked with ublock since then).
Description
•