1897875 - Virtual memory allocation keeps growing while using Firefox with hardware acceleration on, eventually causing the GPU process to crash

Status: RESOLVED DUPLICATE of bug 1897006

(Core :: Graphics, defect)

Description

[caskets-06-mystery]

Attachment: memory-report-anonymized.json.gz

I have had some issues after updating to FF 126.0 where the window UI appears to disappear for about a second before reappearing (as in, the window, page, and browser UI becomes blank for a second then reappears, similar what appears for a split second while Firefox is opening). Checking about:crashes shows that FF crashed.

This happened 3 times to me in the past 4 days. Only the first 2 times produced a crash report in about:crashes.

At the time Firefox crashes (all 3 times), I see an event in the Windows Event Viewer under the system logs that says the following (or similar):

Windows successfully diagnosed a low virtual memory condition. The following programs consumed the most virtual memory: firefox.exe (4892) consumed 48357220352 bytes, vmmemWSL (22580) consumed 939347968 bytes, and firefox.exe (4736) consumed 878108672 bytes.

When leaving Firefox open for a few hours and then looking at the process with the giant "commit size" in Windows' Task Manager, that process's PID corresponds to the GPU process indicated in about:memory, which also seems to be the process that crashed. (After the crash, the GPU process automatically reopens with a much lower commit size.) That "commit size" for the GPU process as measured by Windows Task Manager seems to go up quite quickly when opening a YouTube video and does not go back down when it is closed.

I assume Windows killed the Firefox GPU process due to the high virtual memory usage. I'm not sure the crash reports themselves are that meaningful if that is the case, but they are below.

This issue does not happen anymore when hardware acceleration is disabled (in Firefox Settings > General > Performance, unchecked "Use recommended performance settings" and unchecked "Use hardware acceleration when available").

Steps to reproduce:

1. To be able to observe Firefox's virtual memory usage, open Task Manager, go to the Details tab, then filter the processes by Firefox. If necessary, right click one of the column names, choose "Select columns," and check "Commit size."
2. Open Firefox.
3. Go to https://www.youtube.com
4. Click on any video.
5. Go back to Task Manager and check the "commit size" of the Firefox GPU process. (On my system, it goes up significantly.)
6. From that video, click on any of the recommended videos.
7. Check Task Manager again - the commit size goes up even more.
8. Repeat step 6 until eventual crash. (Or just use Firefox normally.)

GPU details:
AMD Radeon RX 6600
AMD Adrenalin 24.5.1 - full install (The first crash I experienced was with Adrenalin 24.4.1 though, where I did not experience the issue before updating to Firefox 126 even while Adrenalin 24.4.1 was installed. I tried updating the driver to fix this but the behavior was the same.)

---

Crash report: https://crash-stats.mozilla.org/report/index/eda250a3-b878-40d8-bafa-925370240519

Reason: EXCEPTION_STACK_OVERFLOW

Top 10 frames:

0  mozglue.dll  mozilla::Maybe<unsigned long long>::Maybe(mozilla::Nothing)  mfbt/Maybe.h:388
0  mozglue.dll  MozJemallocPHC::malloc(unsigned long long)  memory/build/PHC.cpp:1346
0  mozglue.dll  je_malloc(unsigned long long)  memory/build/malloc_decls.h:51
1  xul.dll  alloc::alloc::alloc(core::alloc::layout::Layout)  /rustc/25ef9e3d85d934b27d9dada2f9dd52b1dc63bb04/library/alloc/src/alloc.rs:98
1  xul.dll  alloc::alloc::Global::alloc_impl(core::alloc::layout::Layout, bool)  /rustc/25ef9e3d85d934b27d9dada2f9dd52b1dc63bb04/library/alloc/src/alloc.rs:181
1  xul.dll  alloc::alloc::impl$1::allocate(alloc::alloc::Global*, core::alloc::layout::La...  /rustc/25ef9e3d85d934b27d9dada2f9dd52b1dc63bb04/library/alloc/src/alloc.rs:241
1  xul.dll  hashbrown::raw::alloc::inner::do_alloc(alloc::alloc::Global*, core::alloc::la...  /rust/deps/hashbrown-0.14.3/src/raw/alloc.rs:15
1  xul.dll  hashbrown::raw::RawTableInner::new_uninitialized<alloc::alloc::Global>(alloc:...  /rust/deps/hashbrown-0.14.3/src/raw/mod.rs:1752
2  xul.dll  hashbrown::raw::RawTableInner::fallible_with_capacity(alloc::alloc::Global*, ...  /rust/deps/hashbrown-0.14.3/src/raw/mod.rs:1790
2  xul.dll  hashbrown::raw::RawTableInner::prepare_resize(alloc::alloc::Global*, hashbrow...  /rust/deps/hashbrown-0.14.3/src/raw/mod.rs:2869

---

Crash report: https://crash-stats.mozilla.org/report/index/c1a59763-b979-495a-a6e5-e5d000240519

Reason: EXCEPTION_ACCESS_VIOLATION_WRITE

Top 10 frames:

0  ntdll.dll  RtlpWaitOnCriticalSection
1  ntdll.dll  RtlpEnterCriticalSectionContended
2  ntdll.dll  RtlEnterCriticalSection
3  ntdll.dll  RtlSleepConditionVariableCS
4  KERNELBASE.dll  SleepConditionVariableCS
5  amdxx64.dll  amdxx64.dll@0x97202
6  ntdll.dll  RtlSetLastWin32Error
7  amdxx64.dll  amdxx64.dll@0x9740e
8  amdxx64.dll  amdxx64.dll@0xccb1a7
9  amdxx64.dll  amdxx64.dll@0xccb201

---

There's also an anonymized report from about:memory attached from Firefox when it had about ~12 hours uptime (this was a few minutes before a crash, by chance).

Comment 1

[Sotaro Ikeda [:sotaro]]

It looks similar to Bug 1897006 or Bug 1847453.

caskets-06-mystery, can you check if the problem happen with latest nightly?

Comment 2

[caskets-06-mystery]

Looks like this is fixed in the latest Nightly (20240521094553). I tried going to a bunch of YouTube videos and the commit size measured by Task Manager didn't go above about 1 GB. (I didn't try for the length of time it would normally take to crash, but given that the commit size isn't infinitely leaking I doubt it would crash.)

Comment 3

[Sotaro Ikeda [:sotaro]]

Great! Thank you for the checking.