Closed Bug 1828587 Opened 1 year ago Closed 11 months ago

Animated theme produces steady memory leak in occluded or minimized windows [leaking render-texture-hosts via RecvAddSharedSurface]

Categories

(Core :: Graphics: WebRender, defect, P1)

Firefox 112
defect

Tracking

()

VERIFIED FIXED
114 Branch
Tracking Status
relnote-firefox --- 112+
firefox-esr102 --- unaffected
firefox112 + verified
firefox113 + verified
firefox114 + verified

People

(Reporter: kuriosevwon, Assigned: emilio)

References

(Blocks 2 open bugs, Regression)

Details

(Keywords: memory-leak, regression)

Attachments

(13 files)

Attached file memory-report.json.gz

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/112.0

Steps to reproduce:

Opened firefox, tabs restored w/ Youtube/Videos immediately | OR | Opened Firefox with no tabs loaded with video, then opened Twitch VOD | OR | Opened Firefox with no tabs loaded with video, then opened Kick VOD | OR | Opened a separate Firefox Profile and watched a video on there

Actual results:

In all cases above, seemingly, when Video related content is loaded after 30 or so minutes Firefox will consume all RAM. With Youtube at a rate of 13 megabytes a second. With other video providers at different rates. This RAM usage is not correctly reported in Task Manager, but is visible in Resource Monitor under Firefox. I don't have to watch anything. Just have the Video tab loaded and stopped the video and it will continue to rise even when buffer percentage is reached, as if some infinite cycle going on. Firefox seems to run normally for the most part, sometimes delayed input. RAM is freed when Firefox is closed. When closing Firefox, video seems to trail on for a second or two before shutting down, as if it is unloading a bunch of data in the background first. https://i.imgur.com/L46Fubn.png

Expected results:

RAM usage reflects tabs loaded and memory gets unloaded or stays steady depending on usage. Normal use around 3-6 gigs depending on workload. Not rising until 100% of memory is reached (22 gigs of in-use memory).

Let me know if you need any more data. Happy to help!

The Bugbug bot thinks this bug should belong to the 'Core::Performance' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Component: Untriaged → Performance
Product: Firefox → Core

This bug was moved into the Performance component.

:kuriosevwon, could you make sure the following information is on this bug?

  • For slowness or high CPU usage, capture a profile with http://profiler.firefox.com/, upload it and share the link here.
  • ✅ For memory usage issues, capture a memory dump from about:memory and attach it to this bug.
  • Troubleshooting information: Go to about:support, click "Copy raw data to clipboard", paste it into a file, save it, and attach the file here.

If the requested information is already in the bug, please confirm it is recent.

Thank you.

Flags: needinfo?(kuriosevwon)

Requested about:support data dump

Flags: needinfo?(kuriosevwon)

https://share.firefox.dev/3GY1G8s Requested profile capture. Not that the application really runs slow. Just memory usage keeps growing and overflows to physical drives.

Also, additional community discussion can be found at https://www.reddit.com/r/firefox/comments/12q6m1z/firefox_hoarding_massive_amount_of_memoryram/

Hello, thank you for your report! Can you try the following steps:

  • Download VMMap.
  • Reproduce the issue.
  • Navigate to about:processes and identify the process ID for the GPU process, it should show as GPU (XXXX) where XXXX is the process ID.
  • Launch VMMap64.exe and select the Firefox GPU process by process ID.
  • Capture a screenshot of the VMMap window that opens and share the screenshot here.
  • Ideally, if you can also click File, Save As to save the data into a file, then share that file with me at yjuglaret@mozilla.com, that is even better.
  • In the VMMap window, if you click Tools, Empty Working Set, does this temporarily solve the issue?
Attached image Aelux_VMMap.png
Attached file Aelux_firefox.mmp

Hi Yannis,

I believe that I am experiencing the same bug and the cause of the bug I have found is simply having a window minimized, possibly with media on it I am unsure of that detail. And for me it clears itself out when I focus on that window again but doing that every 20 minutes or simply having to make sure no windows are minimized ever isn't ideal.

I have performed your last steps if you wish to check mine and have more data on this. See attached.

Additionally emptying the working set didn't resolve the issue and as per usual, I need to open any minimized windows

[Tracking Requested - why for this release]: might be a big deal according to Yannis

Mine was in fact v112.0.1 and for me I didn't notice it until this update

The attached about:memory shows massive amounts of committed memory which are unused. That's why Task Manager doesn't show it, the memory has been committed but hasn't been touched so it's not backed by physical memory. Unfortunately we'll end up running out of commit space anyway which will eventually lead to a crash, it seems we have a commit-space leak somewhere.

One more piece of information: the commited memory appears in the main process and in the GPU process in similar amounts, this could be related to passing data between the two.

Here's our likely culprit from the report:

28,605.56 MB (47.38%) ── render-texture-hosts

It seems we're leaking textures that have been shared from the main process to the GPU process (the mappings are read/write in the main process and read-only in the GPU process). We need to get hang of somebody from the graphics team.

Confirming the issue.

Status: UNCONFIRMED → NEW
Ever confirmed: true
Duplicate of this bug: 1829179

I'm reproducing on my machine by watching a YouTube video. On Windows commit-space creeps up continuously in both the main process and GPU process. Closing the tab containing the video releases all the committed memory in the main process but not in the GPU process which remains at the same level.

FYI I'm running a bisection, if someone else who knows the graphics code better than me wants to do a more direct investigation just go ahead and don't wait for me.

Severity: -- → S2
Priority: -- → P1

Is the minimized window a requirement? I wonder if it is bug 1768495? That involves the compositor and minimized/occluded windows.

(In reply to Gabriele Svelto [:gsvelto] from comment #15)

Here's our likely culprit from the report:

28,605.56 MB (47.38%) ── render-texture-hosts

These textures are stored in mozilla::wr::RenderThread::mRenderTextures in case you need to track them.

See Also: → 1828793
See Also: → 1820789
See Also: → 1819442
See Also: → 1818982

Having exact same issue, can confirm it occurs when a window that has recently played media (for example a YT video that has ended) is staying minimized. Task manager doesn't show the massive RAM leak, but software such as Process Explorer pinpoints the leak to firefox. When about:memory is run I'm getting massive leakage in the web-renderer section.
https://bugzilla.mozilla.org/show_bug.cgi?id=1828793 for reference.

I've not been able to reproduce reliably on my machine, can you share the exact steps that lead to the problem (including the video causing it in case it only happens on some videos rather than on others)?

Flags: needinfo?(nasko235679)

Also, the original reporter had several YouTube- and Twitch-related extensions, is that also the case?

I have tried without any extensions at all and still encounter my issue, and then on that Reddit thread the original poster shared a lot of people say they have the issue without any Add-ons as well.

Thanks. If anyone experiencing this issue consistently has some time on their hands it would be super-helpful if they could run mozrgression for us:

  • Install mozregression-gui from here: https://github.com/mozilla/mozregression/releases
  • Launch it
  • Click on the scissors icon in the top left (Run a new bisection)
  • In the wizard leave everything to its default values except for "Repository" which needs to be mozilla-central
  • Click next, this will take you to the "Profile selection" page
  • Click next again, this will test using a new profile and won't pollute your main Firefox profile
  • In the "Build selection" page set "Last known good build" to release 111 and "First known bad build" to release 112 then click next
  • mozregression will download and launch a version of Firefox, try reproducing the problem. If you can reproduce close Firefox and click on the "Bad" button in mozregression window, if the problem is gone click on the "Good" button
  • Following this procedure should eventually lead you to the version that first contained the problem, attach this information here.

If anybody can do this it would help us immensely in identifying the issue.

Duplicate of this bug: 1828963
Duplicate of this bug: 1828793

The bug is marked as tracked for firefox112 (release), tracked for firefox113 (beta) and tracked for firefox114 (nightly). We have limited time to fix this, the soft freeze is in 13 days. However, the bug still isn't assigned.

:fdoty, could you please find an assignee for this tracked bug? If you disagree with the tracking decision, please talk with the release managers.

For more information, please visit auto_nag documentation.

Flags: needinfo?(fdoty)

(In reply to Gabriele Svelto [:gsvelto] from comment #24)

Also, the original reporter had several YouTube- and Twitch-related extensions, is that also the case?

(In reply to Gabriele Svelto [:gsvelto] from comment #23)

I've not been able to reproduce reliably on my machine, can you share the exact steps that lead to the problem (including the video causing it in case it only happens on some videos rather than on others)?

It just happened again. So here is what happens:

1.I have multiple monitors. I have multiple firefox instances (windows) open with a few tabs in each window. I minimize one of the windows on one monitor and that's the window that starts to leak memory. SHOCKINGLY this time there was no video in the window that was leaking memory. There was reddit (I use the classic grid so no videos autoplay and there are no video players in the open tab), my gmail and this bug's url. Once I maximized the window the memory leak stopped and memory and page file went back to normal.

In terms of extensions I have YouTube Enhancer, AdBlocker Ultimate, uBlock Origin, betterTTV and Return YouTube Dislike.
Hope this helps.

Flags: needinfo?(nasko235679)
See Also: → 1827186

Edit: No more required thanks to comment 45.

Windows only: Are able to reproduce the issue with this custom build? If yes, please navigate to about:memory while the memory is still high, and click Measure. That should create a report.txt file on your Desktop. If you could then share that file here as an attachment, that should help us diagnose this issue. Thank you!

Note: This is custom behavior that is specific to this custom build. I am not asking for the contents of about:memory. The report.txt file will contain specific information that is relevant for the investigation.

Duplicate of this bug: 1827696
No longer duplicate of this bug: 1827696
See Also: → 1827696
Duplicate of this bug: 1828075

Bug 1828075 has the interesting detail that they are seeing the issue with a second instance of Firefox opened, but minimized.

(In reply to Andrew McCreight [:mccr8] from comment #34)

Bug 1828075 has the interesting detail that they are seeing the issue with a second instance of Firefox opened, but minimized.

That does make me suspect strongly about bug 1768495 again fwiw... Can someone repro without another window minimized or occluded by another window?

See Also: → 1828287

Attached the report with your custom build, also worth mentioning I too have multiple screens and multiple windows open when getting the issue.

Aelux_memory-report looks similar to the other one.

Parent process:
4,927.30 MB ── resident
4,472.57 MB ── shmem-allocated
4,472.57 MB ── shmem-mapped

GPU process:
4,541.99 MB (47.95%) ── render-texture-hosts
└──4,480.06 MB (47.30%) -- images/mapped_from_owner
├──4,473.41 MB (47.23%) -- pid=39604
│ ├──4,461.33 MB (47.10%) ── image(640x360, compositor_ref:2, creator_ref:1)/decoded-nonheap [5076]

  4,666.75 MB ── resident
      0.00 MB ── shmem-allocated
  4,505.23 MB ── shmem-mapped

(In reply to Aelux from comment #37)

Attached the report with your custom build, also worth mentioning I too have multiple screens and multiple windows open when getting the issue.

We need the report.txt from your Desktop, not the about:memory report please. Thanks!

Flags: needinfo?(thedragongamers)

I'm moving this to WebRender, as the leak seems like some sort of graphics thing.

Component: Performance → Graphics: WebRender
Flags: needinfo?(fdoty)

Hmm I don't seem to get that file created, it is the button under show memory reports?

actually one sec, it went to my other drive's "desktop" which isn't normally visible to me, I'll do a fresh one and upload

Bug 1816026 seems like a possible cause. For those who can reproduce this can you try setting gfx.webrender.dcomp-video-sw-overlay-win=false in about:config, restarting Firefox and checking whether you can still reproduce?

See Also: → 1816026

The 3 reports I've seen with the issue (the two in this bug plus the one in bug 1828075) have an entry in the GPU process under explicit/gfx/webrender/swgl, for whatever that's worth, so I assume that means they are all using SWGL.

Flags: needinfo?(thedragongamers)

(In reply to Jeff Muizelaar [:jrmuizel] from comment #43)

Bug 1816026 seems like a possible cause. For those who can reproduce this can you try setting gfx.webrender.dcomp-video-sw-overlay-win=false in about:config, restarting Firefox and checking whether you can still reproduce?

It is still happening with that set and after a restart of FF sadly

Thank you for the report.txt file (comment 45). This suggests that you have 9146 entries that were added to mozilla::wr::RenderThread::mRenderTextures through the following path, which are still alive:

xul!mozilla::wr::RenderThread::RegisterExternalImage+0xfe [/builds/worker/checkouts/gecko/gfx/webrender_bindings/RenderThread.cpp @ 891]
xul!mozilla::layers::SharedSurfacesParent::Add+0x17a [/builds/worker/checkouts/gecko/gfx/layers/ipc/SharedSurfacesParent.cpp @ 243]
xul!mozilla::layers::CompositorManagerParent::RecvAddSharedSurface+0x23 [/builds/worker/checkouts/gecko/gfx/layers/ipc/CompositorManagerParent.cpp @ 266]
xul!mozilla::layers::PCompositorManagerParent::OnMessageReceived+0x88d [/builds/worker/workspace/obj-build/ipc/ipdl/PCompositorManagerParent.cpp @ 306]
xul!mozilla::ipc::MessageChannel::DispatchAsyncMessage+0x71 [/builds/worker/checkouts/gecko/ipc/glue/MessageChannel.cpp @ 1800]
xul!mozilla::ipc::MessageChannel::DispatchMessage+0x15c [/builds/worker/checkouts/gecko/ipc/glue/MessageChannel.cpp @ 1729]
xul!mozilla::ipc::MessageChannel::RunMessage+0xf6 [/builds/worker/checkouts/gecko/ipc/glue/MessageChannel.cpp @ 1526]
xul!mozilla::ipc::MessageChannel::MessageTask::Run+0x70 [/builds/worker/checkouts/gecko/ipc/glue/MessageChannel.cpp @ 1632]
xul!nsThread::ProcessNextEvent+0x58c [/builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp @ 1233]
xul!NS_ProcessNextEvent+0x68 [/builds/worker/checkouts/gecko/xpcom/threads/nsThreadUtils.cpp @ 477]
xul!mozilla::ipc::MessagePumpForNonMainThreads::Run+0xfe [/builds/worker/checkouts/gecko/ipc/glue/MessagePump.cpp @ 330]
xul!MessageLoop::RunHandler+0x50 [/builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc @ 375]
xul!MessageLoop::Run+0x57 [/builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc @ 357]
xul!nsThread::ThreadFunc+0xe6 [/builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp @ 393]

This sounds a lot like an S1.

Severity: S2 → S1

I'm adding another bit of info: I've tried reproducing on my machine with dual monitors but couldn't. I've tried playing video in a separate window on the second monitor, minimizing it, restoring it, etc... and couldn't trigger that runaway behavior. However I noticed a difference between the memory report I get and the ones in the reports from the affected users: if I look at the entries under gfx > webrender > images/mapped_from_owner > pid > image(...)/decoded-nonheap I always have compositor_ref: 1 and creator_ref: 0. In their reports compositor_ref is always >1, sometimes 10+. I don't know how that happens but maybe it's part of the reason why I'm not seeing the problem with the same setup.

So, taking the info from comment 47 it seems we're adding more consumers to those surfaces here. If you dig into that function you'll see that it bumps the mConsumers variable here. So the question is: why does this surface have more than one consumer and could that cause the leak?

As much as I love watching youtube videos for work, this doesn't repro on nightly for me, even matching the webrender-compositor:disabled, dcomp-present:disabled config. (Win11 Nightly114)
Leaking texture hosts is a great lead.

Summary: Firefox consuming massive amount of RAM, seemingly related to HTML Video → Firefox consuming massive amount of RAM, seemingly related to HTML Video [leaking render-texture-hosts via RecvAddSharedSurface]
See Also: 1828793

I am also on W10 still, not sure if that matters? just noticed that about Gilbert's comment and not sure if the other people with the issue are as well?

Aelux, I tested Fx112 (Release) & Fx114 (Nightly) on Win10 64 22H2 with an NVIDIA 2080Ti (driver 531.18) using one and two monitors, and could not reproduce. So, hard to say that the specific version of the OS is actually a factor.

Windows only: Here is a new custom build which could help us explore further the idea from comment 50. Similar to the one from comment 31: Please navigate to about:memory with this custom build while the memory is still high, and click Measure. That should create a report.txt file on your Desktop which could help us diagnose this issue. Thank you!

This new custom build should help us visualize which calls to AddConsumer and DeleteConsumer did happen on the surfaces that are still alive. We may be able to deduce which ones did not happen, for example by comparing with what happens on computers that don't reproduce the issue.

Flags: needinfo?(thedragongamers)
Flags: needinfo?(thedragongamers)

To potential reproducers: note comment 44 suggests swgl might be necessary to reproduce. The pref gfx.webrender.software should turn that on.

(In reply to Timothy Nikkel (:tnikkel) from comment #56)

To potential reproducers: note comment 44 suggests swgl might be necessary to reproduce. The pref gfx.webrender.software should turn that on.

These are my config settings around that and they are unmodified, it claims
gfx.webrender.software=false
gfx.webrender.software.d3d11=true
gfx.webrender.software.d3d11.upload-mode=4
gfx.webrender.software.opengl=false

Could you please attach your about:support (I guess the "Copy raw data to clipboard") as an attachment? I think that might indicate what is going on with your WebRender. Thanks.

Flags: needinfo?(thedragongamers)
Flags: needinfo?(thedragongamers)

Thanks! There are a few things I noticed in that log that sounds relevant to some of the discussion around multiple monitors and refresh rates.

"name": "WEBRENDER_COMPOSITOR",
"status": "unavailable",
"log": [{"type": "default", "status": "available"},
{"type": "runtime", "status": "unavailable",
"failureId": "NVIDIA_REFRESH_RATE_MIXED",
"message": "No DirectComposition usage"}]

"name": "WEBRENDER_DCOMP_PRESENT",
"description": "WebRender DirectComposition",
"status": "blocked",
"log": [{"type": "default", "status": "available"},
{"type": "env", "status": "blocked",
"failureId": "NVIDIA_REFRESH_RATE_MIXED",
"message": "Monitor refresh rate too high/mixed }]

Just as a note: On a downgraded version I am using for the moment as a workaround, the graphics section of that page appears to be identical including things you mentioned such as mixed refresh rate and no composition usage. I am curious how that looks on different machine using 112.0.1 however

I noticed that the original posters windows build was also NT and not sure if that is some correlation or not? are those of you that can't reproduce running standard builds? I know that NT builds require manual installation of a media codec pack or some such whereas it is baked into regular installs. Maybe some change with the new version of FF that uses something that us on NT don't have or that isn't updated? (just wild speculation)

I'm running Windows 11 Pro 22H2 perfectly standard OS, I have no idea why the bug reporter identified it as Windows 10 / NT.

If anyone that can reproduce this would like to try using Firefox Nightly and report back if the problem is reproducible, that would also be a helpful data point. Nightly can be downloaded from https://www.mozilla.org/en-US/firefox/channel/desktop/. But make sure to use a separate profile when Nightly starts up so that there is no risk to your existing profile.

@Andrew McCreight [:mccr8]

Framerate does not seem to be the issue., I just set refresh rate to the same values which resulted in no error. Same issue.

Using Yannis Nightly custom build, comment 54, framerate of displays set to 60hz to avoid "NVIDIA_REFRESH_RATE_MIXED" failure. Memory issue persists.

A few ideas to explore if you have the time:

  • Try turning on the pref gfx.webrender.compositor.force-enabled in the about:config page - this may directly fix the issue but it may cause flickering issues when a browser window is on the border between monitors on nvidia drivers if a high refresh rate monitor is present (we currently disable the compositor to keep this flickering from occuring on affected systems with an nvidia gpu and high refresh rate and regular refresh rate monitors side by side, but not using the compositor can cause video playback code to behave very differently, which might be at the heart of this memory issue).
  • Try installing Firefox Nightly - it is two major versions ahead of the current stable and if we happen to have fixed something in that time range it would be good to confirm (and it would give you a workaround of sorts as you can sync your account in Nightly to your regular release channel browser if you wish).
  • Try mozregression if you have a few hours to spend on this issue - it runs several different versions of Firefox with a temporary profile (i.e. don't expect it to stick around, it's not going to touch your regular browser experience) and prompts you to reproduce the issue in each and tell it whether the issue is present ('bad') or works fine ('good'), it would then tell you exactly which date and time the firefox code changes happened that broke this for you. Knowing this date and time when things broke would let us immediately fix the issue.

If I am able to repro this issue I would be following these same steps to figure out what is going on, but I have not been able to reproduce it yet, so your help is much appreciated if you wish to spend the time :)

So as soon as I tried to use mozregression the issue vanished and the test builds I was using was using my profile but I disabled or removed all addons..or so I thought, an addon that doesn't show as an addon, not even in all these debug logs we provided is a theme. In my case, Dark-Space-Theme which is a highly recommended one on the store. Low and behold I see a bug on the git for that theme and they have linked an old issue for it. They recommend simply making the gif they are using for the theme into an animated png.

https://addons.mozilla.org/en-US/firefox/addon/nicothin-space/
https://github.com/nicoth-in/Dark-Space-Theme/issues/11
https://bugzilla.mozilla.org/show_bug.cgi?id=1786247

It is still interesting to me that the theme only broke with 112.0.1
why did it work fine before that update?
is it localised to just this theme or other animated ones using the same technique? and is this why more than just one person reported it?
is there just my issue which could be unrelated and a second similar more serious one? or is everyone experiencing the same one as me?

Additionally: using gfx.webrender.compositor.force-enabled didn't resolve the issue so it doesn't seem like that is related.

I can confirm that using that dark space theme and opening 50 maximized windows causes a steady rise in committed readwrite pages in the address space section of the main process report in about:memory, it barely shows up on Windows task manager in process view and the Firefox task manager, but the memory view in task manager shows a steadily rising amount of memory usage, if I hold alt-tab and let it cycle through all tabs the memory usage goes entirely back to normal.

What this tells me is that the WebRender thread isn't looking at these hidden windows and dealing with consuming pending pictures pushed by the main process.

(In reply to Avwon from comment #66)

Created attachment 9329780 [details]
firefox_about-support-nightly.txt

Using Yannis Nightly custom build, comment 54, framerate of displays set to 60hz to avoid "NVIDIA_REFRESH_RATE_MIXED" failure. Memory issue persists.

(Custom builds are branded as Nightly but this was based on 112 Release.)

Steps to Reproduce for my example in comment #69:

  • Enable theme https://addons.mozilla.org/en-US/firefox/addon/nicothin-space/
  • maximize window
  • hold Ctrl-N (or Cmd-N on Mac) until you have some hundreds of Firefox windows all overlapping the same monitor
  • open Task Manager in Windows (ctrl-shift-escape), switch to Performance tab, select the Memory tab in that
  • watch memory usage steadily grow for as long as you like
  • hold alt-tab until it has cycled through previews of all of the Firefox windows
  • watch memory usage return to normal and start climbing again

Deactivating the theme and then using alt-tab to cycle all windows will return Firefox to normal memory usage and does not grow over time.

It's not clear to me if this can be done with paused videos in hidden windows instead of animated gifs, but we should try to confirm that, I wasn't able to repro on videos in my testing but maybe there's a trick to it.

DAMN!!! You guys are crazy. I'm using the exact same theme. You actually found it ya crazy bastards. Okay, I guess the solution is switching to another theme?

Attached file consumers.txt

For what it's worth, here are the symbolized stacks from comment 55 explaining why these surfaces are still alive. It seems that only half of the calls to add consumers originating from WebRenderBridgeParent::RecvUpdateResources have a matching call to mozilla::layers::WebRenderBridgeParent::ObserveSharedSurfaceRelease. Note that there are two stacks for WebRenderBridgeParent::RecvUpdateResources: one going through this path and the other through that path, so the total number of calls is the sum of the two paths.

Edit: For more clarity, this should probably read as something like: we added 3650 surfaces, for which, out of the two "things" (threads? processes?) that registered additional consumers for them, only one has successfully executed the code path to unregister those consumers through ObserveSharedSurfaceRelease, hence the underlying textures for these surfaces have not been released.

(In reply to Emilio Cobos Álvarez (:emilio) from comment #35)

That does make me suspect strongly about bug 1768495 again fwiw... Can someone repro without another window minimized or occluded by another window?

At this point though bug 1768495 sounds very suspicious like [:emilio] suggested. Maybe pausing the compositor while we are not active prevents it from doing the job that triggers the path to reach ObserveSharedSurfaceRelease?

(In reply to nasko235679 from comment #72)

DAMN!!! You guys are crazy. I'm using the exact same theme. You actually found it ya crazy bastards. Okay, I guess the solution is switching to another theme?

For the moment that is the best workaround, but we will do our best to fix this with Firefox 112.0.2.

That's a good idea considering this is one of the most used themes (and highest rated ones).

Ayy, glad I got it!
I would also suggest that in the fix it is made so themes which actually exist under the add-on tab show up in logs like all these support output files we generated because someone would have found it much sooner than me if we all could see right away we all had one "add-on" left that we all had in common that wasn't disabled

Has STR: --- → yes
Regressed by: 1768495
See Also: → 1786247
Summary: Firefox consuming massive amount of RAM, seemingly related to HTML Video [leaking render-texture-hosts via RecvAddSharedSurface] → Animated theme produces steady memory leak in occluded windows [leaking render-texture-hosts via RecvAddSharedSurface]

:bradwerth, since you are the author of the regressor, bug 1768495, could you take a look?

For more information, please visit auto_nag documentation.

Flags: needinfo?(bwerth)

As the compositor might be paused. We don't really want to queue a bunch
of paints that are never going to be visible.

Assignee: nobody → emilio
Status: NEW → ASSIGNED

An alternative approach would be to throttle rather than pause the compositor, which is what Wayland does.

This backs out the regressing bug. Backout applied cleanly, so even
though a targetted change in CanonicalBrowsingContext would've probably
been enough, just an automated backout seems preferable.

This backs out the regressing bug. Backout applied cleanly, so even
though a targetted change in CanonicalBrowsingContext would've probably
been enough, just an automated backout seems preferable.

Okay, so we decided to revert the offender on beta/release, and land the "proper" fix on nightly (since it's not risk-free):

Flags: needinfo?(bwerth)

Comment on attachment 9329810 [details]
Bug 1828587 - [release] Back out bug 1768495. r=smaug,bradwerth

Beta/Release Uplift Approval Request

  • User impact if declined: See comment 0. When users have animated themes and one window is occluded or minimized, we'll appear to leak memory until the window is painted (which might be never).
  • Is this code covered by automated tests?: No
  • Has the fix been verified in Nightly?: No
  • Needs manual test from QE?: Yes
  • If yes, steps to reproduce: comment 71 has reliable STR
  • List of other uplifts needed: none
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): Backs out regressing bug.
  • String changes made/needed: none
  • Is Android affected?: No
Attachment #9329810 - Flags: approval-mozilla-release?
Flags: qe-verify+
Pushed by ealvarez@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6f2d406b412b
Don't paint or refresh animated images on throttled refresh drivers. r=smaug

Comment on attachment 9329811 [details]
Bug 1828587 - [beta] Back out bug 1768495. r=smaug,bradwerth

Beta/Release Uplift Approval Request

  • User impact if declined: See comment 0. When users have animated themes and one window is occluded or minimized, we'll appear to leak memory until the window is painted (which might be never).
  • Is this code covered by automated tests?: No
  • Has the fix been verified in Nightly?: No
  • Needs manual test from QE?: Yes
  • If yes, steps to reproduce: Comment 71
  • List of other uplifts needed: none
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): Cleanly reverts regressing patches.
  • String changes made/needed: none
  • Is Android affected?: No
Attachment #9329811 - Flags: approval-mozilla-beta?
Blocks: 1829488
Blocks: 1829492
Blocks: 1829493
Blocks: 1829494

Comment on attachment 9329811 [details]
Bug 1828587 - [beta] Back out bug 1768495. r=smaug,bradwerth

Approved for 113.0b7

Attachment #9329811 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Summary: Animated theme produces steady memory leak in occluded windows [leaking render-texture-hosts via RecvAddSharedSurface] → Animated theme produces steady memory leak in occluded or minimized windows [leaking render-texture-hosts via RecvAddSharedSurface]

Comment on attachment 9329810 [details]
Bug 1828587 - [release] Back out bug 1768495. r=smaug,bradwerth

Approved for 112.0.2 dot release

Attachment #9329810 - Flags: approval-mozilla-release? → approval-mozilla-release+
Status: ASSIGNED → RESOLVED
Closed: 11 months ago
Resolution: --- → FIXED
Target Milestone: --- → 114 Branch

Darn, now I feel terrible for the misdirection. All of my profiles had different themes for me to differentiate them. My barebones profile also had a animated theme to it. Thank you for isolating the issue Aelux and for the devs work on this.

Darn, now I feel terrible for the misdirection.

Don't :) It is part of the investigate process!

If folks want to test, 113.0b7 is live on beta with the change

QA Whiteboard: [qa-triaged]
Duplicate of this bug: 1829818
No longer duplicate of this bug: 1829818
Duplicate of this bug: 1827139
Duplicate of this bug: 1829818
See Also: → 1829868

(In reply to Sylvestre Ledru [:Sylvestre] from comment #96)

If folks want to test, 113.0b7 is live on beta with the change

Thank you, I have used it and it appears to be resolved.

I may have ran into a separate performance issue yesterday but will confirm. I was doing some other things as well at the time which might be the actual cause.

I managed to reproduce the issue described here using steps from comment 71 and an old Nightly build from 2023-04-20. I also verified that the memory remains at a constant rate using latest Nightly 114.0a1, Firefox 113.0b8 and Firefox 112.0.2 across platforms (macOS 13, Ubuntu 22.04 and Windows 10).

Flags: qe-verify+
Duplicate of this bug: 1829868
See Also: 1829868
Duplicate of this bug: 1823847
See Also: → 1820841
Duplicate of this bug: 1827841
No longer duplicate of this bug: 1827841
See Also: → 1830753

I had Firefox beta version 113.0b7 installed to fix the issue for me since it was pointed out. Since then my Firefox has automatically updated to version 114.0b3 and this exact animated theme/gpu memory issue has returned. I can provide logs again if needed.

avwon, can you please file a separate bug so that this would get the right attention?

(In reply to Julien Wajsberg [:julienw] from comment #106)

avwon, can you please file a separate bug so that this would get the right attention?

Thanks, reported: https://bugzilla.mozilla.org/show_bug.cgi?id=1834246
Sorry, for late report, a bit busy :)

See Also: → 1834246

comment 73 indicates that this has the exact same presentation at the issue being tracked in Bug 1830753. What we've landed here, though useful, wasn't sufficient to fix the issue. Let's use Bug 1830753 to build the rest of the fix.

Duplicate of this bug: 1786940
See Also: → 1836699
Blocks: 1838350
Regressions: 1843205
See Also: → 1845413
Regressions: 1843700
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: