1834246 - Memory Leak due to Animated Theme reemerged in beta 114.0b6 after update from 113.0b7

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, defect)

Description

[Avwon]

Attachment: memory-report.json-3.gz

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/114.0

Steps to reproduce:

Install Firefox 113.0b7 to use fix of previously reported issue for the same bug. https://bugzilla.mozilla.org/show_bug.cgi?id=1828587
After a while I received an automatic update to 114.0b6 which I didn't really take note off until my previous v112 issue came back with my animated themes overrunning my RAM.
Open Firefox 114.0b6 with any animated theme, for example 'ANIMATED | Neurons', and reportedly having one window occluded, causes memory leak.

Actual results:

Memory leak rises at steady rate when animated theme enabled. No rise when disabled.

Expected results:

Dedicated memory to GPU render of animated frames (render-texture-hosts) needs to be released as necessary.

Fix should have been already applied to 114 branch as far as I am aware, odd to have it come back.

Comment 1

[Yannis Juglaret [:yannis] | PTO back Oct 16]

Thanks Avwon for this early notice. I can reproduce the issue in Firefox 114b7 with Windows 11 22H2 using the steps from bug 1828587 comment 71.

In my case I do the following:

• open 40 maximized windows with Dark Space theme on top of one another;
• sort processes by Working Set (KB) in Resource Monitor;
• GPU process is listed first with both Firefox 113.0.1 and Firefox 114b7;
• the Working Set value stabilizes at ~4.7GB in 113.0.1 (this is the expected behavior), but it continues to grow infinitely in 114b7 (this is bad behavior) provided that at least one of the Firefox windows is visible (I stop at ~10 GB).

This issue was fixed for 112 and 113 by a backout, and for 114 we took in a patch instead. The patch appears to not be enough to fix the issue, or subsequent work may have brought it back.

Comment 2

[Yannis Juglaret [:yannis] | PTO back Oct 16]

Attachment: memory-reports-113.0.1-114b7.zip

Attaching my own memory reports from 113.0.1 and 114b7 which were recorded on the same machine following the same steps, so they can be used to diff. The diffing shows +6.7GB in 114b7 gfx (because I stopped my test at ~10GB of Working Set), resulting from render-texture-hosts again:

6,654.57 MB (100.0%) -- gfx
└──6,654.57 MB (100.0%) -- webrender
   ├──6,552.37 MB (98.46%) -- textures
   │  ├──6,522.69 MB (98.02%) ── render-texture-hosts
   │  └─────29.69 MB (00.45%) ++ (4 tiny)
   └────102.20 MB (01.54%) -- images/mapped_from_owner/pid=NNN
        ├───94.92 MB (01.43%) ── image(640x360, compositor_ref:2, creator_ref:1)/decoded-nonheap [108]
        └────7.27 MB (00.11%) ++ (15 tiny)

Comment 3

[Emilio Cobos Álvarez (:emilio)]

Why is nightly not affected?

Comment 4

[Emilio Cobos Álvarez (:emilio)]

(Or it is and somehow QA didn't catch it in bug 1828587?)

Comment 5

[Kestrel]

It's reproducible on latest Nightly 115.0a1 (2023-05-22) and 114.0a1 (2023-04-23) builds so it appears it was never fixed on Nightly.

Comment 6

[Brad Werth [:bradwerth]]

This may be a duplicate of Bug 1830753. I'm building a fix there that definitely will impact memory usage of animated themes with at least one window occluded and one visible. Bug 1830753 is reported as macOS-specific, but from looking at the root cause, it's likely to be happening on all platforms. The fix, when ready, will address the issue for all platforms.

Comment 7

[Bogdan Maris, Desktop Test Engineering]

(In reply to Kestrel from comment #5)

It's reproducible on latest Nightly 115.0a1 (2023-05-22) and 114.0a1 (2023-04-23) builds so it appears it was never fixed on Nightly.

I can confirm this as well, I was the QA who verified bug 1828587 and it looks like I've missed this on Nightly 114. Both the one I've tested at that time 2023-04-26 and the one with the patch https://treeherder.mozilla.org/jobs?repo=mozilla-central&revision=aeded8c07780ebdc913d0a465d1a55f37318b2ef, also latest Nightly 115.0a1 (2023-05-22) and latest Beta 114.0b7 they are all affected.

Comment 8

[Timothy Nikkel (:tnikkel)]

Should we back out bug 1768495 from beta to buy ourselves some time to fix this (at least the part of the problem that bug 1768495 made worse)?

Comment 9

[Brad Werth [:bradwerth]]

(In reply to Timothy Nikkel (:tnikkel) from comment #8)

Should we back out bug 1768495 from beta to buy ourselves some time to fix this (at least the part of the problem that bug 1768495 made worse)?

I think we shouldn't, because that's a good fix that causes two bad side effects in Bug 1830753:

1. An external image is added every frame of the animation.
2. The external image is never released.

I'd rather fix side effect #2 by landing D178551, and then build a fix for side effect #1 as a follow-up. I'll make this argument in the review comments for D178551.

Comment 10

[Timothy Nikkel (:tnikkel)]

Attachment: Bug 1834246. Back out bug 1768495 from beta. r?emilio,bradwerth

Comment 11

[Timothy Nikkel (:tnikkel)]

Comment on attachment 9335752 [details]
Bug 1834246. Back out bug 1768495 from beta. r?emilio,bradwerth

Beta/Release Uplift Approval Request

• User impact if declined: memory leak with occluded window and theme with animated image
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): backout to what we are already have on release
• String changes made/needed:
• Is Android affected?: No

Comment 12

[Timothy Nikkel (:tnikkel)]

ni moved from bug 1830753. I think this ni is just to get this backout approved/landed.

Comment 13

[Pascal Chevrel:pascalc]

Comment on attachment 9335752 [details]
Bug 1834246. Back out bug 1768495 from beta. r?emilio,bradwerth

Approved for 114 beta 9, thanks.

Comment 14

[Pascal Chevrel:pascalc]

https://hg.mozilla.org/releases/mozilla-beta/rev/a77cb72b0b19

Comment 15

[Timothy Nikkel (:tnikkel)]

I'm gonna mark this as fixed as this was strictly to track the issue in 114 beta and we have several bugs open to fix the underlying problem. We might have to back out from beta again depending on how long the fix takes but we can file new bugs for that.

Comment 16

[Bogdan Maris, Desktop Test Engineering]

Using Firefox 114.0b9 it looks like this is fixed by the back out for me on all 3 platforms (Windows 10, Ubuntu 18.04 and macOS 13). Test duration was about 30 minutes.

Windows 10:

• 78,794,128 B (01.14%) - render-texture-hosts (constant) - memory usage ~2.000 MB the entire time, total memory percentage 53% entire time.

MacOS 13:

• 65,755,160 B (01.20%) - render-texture-hosts (constant) - memory usage at 5.000 MB the entire time, total memory used stayed constant as well.

Ubuntu 18.04:

• 99,220,468 B (04,27%) - render-texture-hosts (constant) - memory usage at 1.100 MB the entire time, total memory percentage 41% all entire time.

Comment 17

[Timothy Nikkel (:tnikkel)]

Attachment: Bug 1834246. Back out bug 1768495 from beta 115. r?emilio,bradwerth

Comment 18

[Phabricator Automation]

Comment on attachment 9337398 [details]
Bug 1834246. Back out bug 1768495 from beta 115. r?emilio,bradwerth

Revision D179909 was moved to bug 1836699. Setting attachment 9337398 [details] to obsolete.