ArrayBuffers do not trigger OOM and can cause system to become unresponsive
Categories
(Core :: JavaScript: GC, enhancement, P3)
Tracking
()
People
(Reporter: benzakhar, Unassigned)
References
(Blocks 1 open bug)
Details
Attachments
(1 file)
|
639 bytes,
text/html
|
Details |
Steps to reproduce:
When I develop applications for the web I have memory leaks and I prefer firefox to just kill the webpage preferable with some stacktrace or debugging info so I can debug my code. Instead firefox allows the leak to go out of hand. Many websites also have memory leaks infowars.com is one of them, just let firefox be idle while having some tabs open and watch the memory usage progressively go up.
Actual results:
Firefox on desktop just eats memory and allows missbehaving code to eat system resources and produces a frustrating experience.
Expected results:
Just kill the website or tab and console.error some details for me so I may diagnose and fix my code. The lack of killing such websites is also annoying when I casually browse the web for personal use as people have buggy code or websites just use unreasonable amount of resources.
Ideally any website that uses more than 1GB of memory should ask me for permissions. And when CPU is concerned any website that utilizes too much CPU cycles while being in background should ask me for permissions as well.
The memory usage ideally to be calculated by incorporating all non-javascript data like images, javascript data, strings, array buffers across all threads, and GPU resources such as texture buffers, shaders (estimate it based on pre-compiled string size should be fine), data buffers, and anything additional you can think of.
There should be an absolute hard limit, just query the system for total physical RAM. When the total limit of anything start reaching this limit, I don't care how much virtual memory there is, start killing tabs, biggest memory usage first as those are usually the culprits.
Due to the way firefox utilizes virtual memory setting ulimits is not sufficient for me and doesn't work well.
Thank you.
|
||
Comment 1•1 years ago
|
||
The Bugbug bot thinks this bug should belong to the 'Core::Graphics' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.
|
||
Comment 2•1 years ago
|
||
- Can you attach a testcase to this bug that reproduces this issue OR can you share link to publicly visible page that reproduces this issue?
- Can you type about:support in the Firefox browser (where you are testing your web applications) , and paste its contents to this bug
Fascinating, I did a simple test by loading up an array, and firefox reports out of memory. I have never seen that before. I'm going to think harder how to make a sample that circumvents this. There is clearly something wrong and I don't know what it is.
The following code bellow results in out of memory in the dev console. However once that happens I have to restart firefox to test this again. Additional the memory consumed seems permanent and firefox never gives it back to the OS, even after I closed the window. Opening a new window and firefox refuses to load this file and I can't even right cilck to open devconsle. Using the shortcut F12 results in unusable devconsole. The more memory you have the longer it will take.
I'm using Firefox 123.0.1 (64-bit) snap package for Ubuntu
➜ bigmemweb cat index.html
<!DOCTYPE html>
<html>
<body>
<div id="bytes"></div>
<div id="megs"</div>
<script>
let everything = [];
let mem = [];
setInterval(function() {
mem.push("hello world123456789123456789");
let total = 0;
for (let i = 0; i < mem.length; i += 1) {
if (mem[i].length < 1024*1024)
mem[i] = mem[i] + 'h';
else
mem[i] = mem[i].substr(10);
total += mem[i].length;
}
document.getElementById("bytes").innerHTML = total + ' bytes';
document.getElementById("megs").innerHTML = (total/(1000*1000)) + ' megs';
}, 1);
</script>
</body>
</html>
|
|
||
Comment 5•1 years ago
|
||
|
|
||
Comment 6•1 years ago
|
||
Can confirm the memory increase. Memory will not increase initially, but maybe after 30-45 seconds it will start rising quickly.
https://share.firefox.dev/3VoKwcp
I am classifying under JS:GC, cc'ing :sfink and :jonco , but feel free to change that.
|
|
||
Comment 7•1 years ago
|
||
This also repros on Google Chrome, so I am guessing there is something unconvenional about this code.
The code just eats memory, it simulates a memory leak on the web developers side, not on firefoxes side. That's why I labeled it as a feature request, I want my faulty code to not force me to restart firefox.
After some more investigation I noticed ArrayBuffers can bypass the out of memory error. Becareful with the bellow webpage. It takes a long time, about 10min, it will use up all your memory, then it will use up your swap, and your whole system will slow down potentially to a halt. I closed the window before that happened so I don't know, but my whole system slowed down drastically once I saw my swap space neared 100%.
Fascinating that once closed, firefox reclaimed the memory back to the OS. Also fascinating it took so long. I prefer firefox to kill this. Compared to the above I never got an out of memory error.
➜ bigmemweb cat index.html
<!DOCTYPE html>
<html>
<body>
<div id="bytes"></div>
<div id="megs"</div>
<script>
let everything = [];
let mem = [];
let bigMem = [];
setInterval(function() {
mem.push("hello world123456789123456789");
bigMem.push(new Uint8Array(1024*1024));
let total = 0;
for (let i = 0; i < mem.length; i += 1) {
if (mem[i].length < 1024*1024)
mem[i] = mem[i] + 'h';
else
mem[i] = mem[i].substr(10);
for (let j = 0; j < 1000; j += 1) {
bigMem[i][j*1024] = j*bigMem.length;
}
total += mem[i].length;
total += bigMem[i].byteLength;
}
document.getElementById("bytes").innerHTML = total + ' bytes';
document.getElementById("megs").innerHTML = (total/(1000*1000)) + ' megs';
}, 1);
</script>
</body>
</html>
This could also be considered a bug cause no misbehaving website should be allowed to interfere with my system or firefox in this way. It's a bit subjective. In theory a malicious person could optimize the above code and force people opening links to it to restart their computer. I find firefox on iOS & Android does a much better job handling this, maybe due to those OSes being very pedantic with memory use.
Most of the time it's bugs, and especially for development I want to set a tiny hard limit and just get a stack trace & details so I can avoid it, and this will help people make better websites. After some sleep I have realized the best way is to do it like permissions, but only present it when the website consumes above a threshold, and offer user to not just stop the script, but kill the tab and all memory associated, with an option to save a default higher memory limit. I think that will make everyone the most happiest. Then no one has to write new code to handle memory permissions.
|
||
Comment 10•1 year ago
|
||
Making this a setting which can be managed per domain if something which would require a lot more coordination.
While I agree with the principle of the idea, I can offer a mediocre alternative.
The GC has a setting which is named javascript.options.mem.max (you can set it by visiting about:config), which let you choose how much memory should be allocated by the garbage collector before emulating an Out-Of-Memory issue in the JavaScript engine.
|
|
||
Updated•1 year ago
|
|
Reporter | |
Comment 11•1 year ago
|
||
I set javascript.options.mem.max to 1000, I assume its unit is MB. My example above with array buffers surpasses that limit and keeps on going. I have tried restarting firefox, and double checked about:performance that it indeed is reported to surpass that limit too, beyond 2GB with no errors in devconsole.
|
|
||
Comment 12•1 year ago
|
||
Thanks for the feedback, yes, this sounds likely as the setting is most related to the GC and array buffers are tiny GC-object holding large malloc-ed arrays.
The other way I know to limit the memory usage of Firefox and all its sub-processes is using to systemd:
systemd-run --unit=firefox --user --scope -p MemoryMax=6G -- firefox
|
|
Reporter | |
Comment 13•1 year ago
|
||
If you take the example above and remove the string memory and just have the array buffers. It's clear that the firefox memory snapshot in inspect element does not capture memory contributed by the malloced data in ArrayBuffers. Should this be another ticket?
|
|
Reporter | |
Comment 14•1 year ago
|
||
I renamed ticket to be something that is more actionable based on what the cause is. Also I noticed that in inspect element -> memory -> memory snapshot does not capture the memory contribution of malloc-ed data of array buffers. This makes analyzing something like tensorflow js, wasm, and array buffer optimizations more difficult.
|
|
Reporter | |
Comment 15•1 year ago
|
||
Reporting in that after setting javascript.options.mem.max to 1000 for both my personal computer and work. It leads to a more pleasant browsing experience.
Observe the code bellow. Very interesting that firefox reports the size of this array buffer when created the way bellow correctly in inspect element -> memory tab.
// take memory snapshot
canvas = document.createElement("canvas");
canvas.width = 4000;
canvas.height = 4000;
context = canvas.getContext("2d");
context.fillRect(3000, 3000, 1000, 1000);
context.getImageData(0, 0, 4000, 4000);
// take memory snapshot
image = null;
context = null;
canvas.width = 0;
canvas.height = 0;
canvas = null;
// take snapshot
// notice that memory appears to still be referenced
Also it appears that firefox does not report the memory associated with the canvas buffer itself. For example if you try to remove the getImageData(), I could not see it's associated memory when taking memory snapshots, even if I add them to document.body. Perhaps it's on the GPU, in such a case my opinion it should still contribute to either GC budget or perhaps GPU memory budget to eventually lead to OOM.
Description
•