1999636 - [HDR] VRAM leak in webgl apps with gfx.color_management.mode = 1 and gfx.wayland.hdr = true

Status: UNCONFIRMED

(Core :: Graphics: CanvasWebGL, defect)

Description

[benjamin.j.grant]

Attachment: about:support from affected profile

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:144.0) Gecko/20100101 Firefox/144.0

Steps to reproduce:

• create a fresh profile and set gfx.color_management.mode = 1 and gfx.wayland.hdr = true
• use a WebGL app (i tried this demo app i made for school and Google Maps in satellite mode while panning around)

Actual results:

VRAM usage constantly increases until it overflows into system memory and eventually the system freezes

Expected results:

VRAM usage should go up only enough to accommodate the application and remain roughly steady thereafter (this is observed with either or both of the flags left default)

Comment 1

[benjamin.j.grant]

Attachment: VRAM usage graph

This graph shows VRAM usage on my GPU over time

• the sharp rise is when i was running a WebGL app in the affected profile
• the plateu is when i closed the WebGL tab (so the VRAM use stopped going up but it remained high)
• the falloff is when i closed Firefox
• the later blip is launching an unaffected Firefox profile, running a WebGL app, then closing it (all using minimal VRAM)

Comment 2

[BugBot [:suhaib / :marco]]

The Bugbug bot thinks this bug should belong to the 'Core::Graphics: CanvasWebGL' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 3

[Ashley Hale [:ahale]]

NeedInfo - do you have theories on why this changed with gfx.wayland.hdr? I know the GLBlitHelper side of this code but I'll need to dig through it to see where the bug is, so any tips would be appreciated.

Comment 4

[Martin Stránský [:stransky] (ni? me)]

Yes, HDR rendering uses different 'native compositor' implementation and uses external textures for WebGL surfaces. There may be a bug in surface recycling. The easiest way is to run Firefox on terminal with MOZ_LOG="Dmabuf:5" env variable and check how dmabuf surfaces are used/recycled.
We recycle dmabuf framebuffers for regular WebRender.

Comment 5

[Martin Stránský [:stransky] (ni? me)]

I'm not sure if gfx.color_management.mode = 1 is related at all - no idea what this pref does.

Comment 6

[Ashley Hale [:ahale]]

gfx.color_management.mode only affects QCMS handling of untagged images, it's unrelated (though it may affect GLBlitHelper), but yes the dmabuf recycling is what I assumed must be going wrong here, I'll have to check if we're properly recycling dmabuf surfaces in WebGL when using GLBlitHelper to color-correct and convert the framebuffer.

Comment 7

[Martin Stránský [:stransky] (ni? me)]

(In reply to Ashley Hale [:ahale] from comment #6)

I'll have to check if we're properly recycling dmabuf surfaces in WebGL when using GLBlitHelper to color-correct and convert the framebuffer.

I think that's correct - we use that for regular WebGL/WebRender patch too without this bug. I think the problem is in texture host HDR rendering code.

Comment 8

[Martin Stránský [:stransky] (ni? me)]

(In reply to Martin Stránský [:stransky] (ni? me) from comment #7)

(In reply to Ashley Hale [:ahale] from comment #6)

I'll have to check if we're properly recycling dmabuf surfaces in WebGL when using GLBlitHelper to color-correct and convert the framebuffer.

I think that's correct - we use that for regular WebGL/WebRender patch too without this bug. I think the problem is in dmabuf texture host HDR rendering code.

Comment 9

[Martin Stránský [:stransky] (ni? me)]

Actually gfx.color_management.mode does the difference here - I see the dmabuf memleak only when it's set.

Comment 10

[Jesper Jensen]

I think I'm seeing this same issue, except I'm not setting gfx.color_management.mode (it's 2 by default). Like the other reporter here I'm seeing a steadily increasing VRAM allocation, but unlike him I'm also seeing hangs in my window manager (sway), with perf reporting most of the time is being spent in AMDGPU TTM sorting buffer objects.

I've recorded a couple of WAYLAND_DEBUG=1 traces (with and without HDR enabled) to track down the issue, and it looks like it's caused by firefox failing to call wl_buffer.destroy after wlroots reports wl_buffer.release in the HDR codepath.

Comment 11

[Jesper Jensen]

Correction. I am not seeing the issue on the reporters websites unless I set gfx.color_management.mode=1. I am however seeing a very similar (but slower) issue on a different website: https://campedersen.com/brep-kernel with gfx.color_management.mode=2.

It's unclear to me if these are the same problem, but they are similar.

Comment 12

[Jesper Jensen]

I think I've narrowed down the issue to the SwapChain::Acquire method where the colorspace seems to be messed up. In my example, the canvas is asking for SRGB, but the SwapChain has Display, that causes the SwapChain to flush its pool and allocate a new surface. This causes the buffer to destroy the surface, which has been imported into the Renderer attached to one of it's surfaces as an "externalBuffer", where it will now sit unused forever.
BUT, that's not all. The SwapChain, after having flushed the old pool, will now try to create a new shared surface to put into the pool. It asks the factory for a surface with the given colorSpace, which the factory (at least if you're not rendering to metal) then ignores because it doesn't support ColorSpaces. Thus you get a Display, which you then pass back to the caller, which gives it back to your pool, where the whole thing repeats again.

All this means you can repro it pretty easily by drawing enabling HDR rendering, and drawing a triangle with webgl where you set gl.drawingBufferColorSpace. I've attached a repro.

Comment 13

[Jesper Jensen]

Attachment: triangle.html

A reproducer

Comment 14

[Ashley Hale [:ahale]]

Thank you for the investigation, this makes it very actionable, I've not familiarize myself with the buffer handling in WebGL yet so I didn't guess this was the mechanism but we need to look into this on all platforms as I think we have similar leaks in other situations caused by the same issue.

Comment 15

[Todd]

(In reply to Jesper Jensen from comment #11)

Correction. I am not seeing the issue on the reporters websites unless I set gfx.color_management.mode=1. I am however seeing a very similar (but slower) issue on a different website: https://campedersen.com/brep-kernel with gfx.color_management.mode=2.

It's unclear to me if these are the same problem, but they are similar.

I'm also having this issue, it's a very gradual increase (on the scale of hours/days) that eventually manifests in stuttering due to GPU vram being at capacity. Tools like amdgpu_top pin the blame on the compositor (sway/wlroots in my case), but I had suspected firefox was to blame due to it all being freed when the application closes. I'd hate to step on the toes of another issue so maybe this should be broken out into a separate report?