Video -> WebGL Texture update causes massive memory leak

RESOLVED FIXED

Status

()

Core
Canvas: WebGL
RESOLVED FIXED
8 years ago
7 years ago

People

(Reporter: Charles J. Cliffe, Assigned: bjacob)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(URL)

Attachments

(2 attachments)

(Reporter)

Description

8 years ago
User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.70 Safari/533.4
Build Identifier: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.3a5pre) Gecko/20100610 Minefield/3.7a5pre

Just testing a video texture rendering in a 3D context for a demonstration -- after running through once with updateTexture() enabled Minefield is using about 1GB of Real Memory, then a second or third refresh will cripple my system as it will use multiple gigs of real and virtual memory.  Only way to recover memory is to exit Minefield completely.   There is a hidden <video> tag with a video playing in the background used to display on the billboard on top of the building.  If you disable the gl.texImage2D call in the updateTexture() function the memory leak will not occur.


Reproducible: Always

Steps to Reproduce:
1.Open Page above with WebGL enabled build
2.Watch resources balloon out of control
3.Attempt to recover sanity ;)
Actual Results:  
Scene plays choppily and Minefield becomes utterly crippled along with the rest of the system.

Expected Results:  
Scene plays smoothly and everyone is happy.

My web server sends video mime-types incorrectly so I have used the original source links in the HTML <video> tag -- you will notice an external connection to the video files although this does not affect the leak (it does the same with local copy).
Status: UNCONFIRMED → NEW
Ever confirmed: true
Severity: critical → normal
(Assignee)

Comment 1

8 years ago
What page are you talking about? Did you forget to paste a link?
(Assignee)

Comment 2

8 years ago
Oh! nevermind. I didn't see the URL.
(Assignee)

Comment 3

8 years ago
Hm, the scene plays smoothly for me here in linux/x86-64/nvidia driver, but indeed, memory usage increases by +400M roughly during the playback of the video inside of the 3D scene. Normally I'd valgrind it but the leaks from bug 568761 comment 4 are too much noise :/
(Reporter)

Comment 4

8 years ago
This appears to also affect just uploading the contents of a canvas or other source, what appears to be happening is the call to texImage2D is creating some internal framebuffer before it uploads to the GPU memory, this buffer should either be re-used and collected (ideally) or cleared every upload but it appears to be just leaking instead.

Comment 5

8 years ago
Yeah I am experiencing exactly the same thing here on Linux. I ran my System Monitor and grabbed a screen-shot so you could watch the fun: http://code.bocoup.com/webgl/WebGL-mem-texture-overload.gif
OS: Mac OS X → All
Hardware: x86_64 → All
(Assignee)

Comment 6

8 years ago
Thanks for the info, and yes the graph is pretty... conclusive :) there isn't much code anyway in TexImage2D, so it has _got_ to be an easy fix, let me try to do that before the beta1...
(Assignee)

Comment 7

8 years ago
ok, the leak is found, it's in the flipY part of DOMElementToImageSurface(), indeed if I comment that out the leak disappears (and the video is upside down).

Vlad, do you see new why I hate reference counting ;-)

I am removing this flipY part because the flipping is best done on the fly in the previous loop, so we avoid allocating a temporary surface and traversing the buffers twice.
(Assignee)

Comment 8

8 years ago
Created attachment 452320 [details] [diff] [review]
image flipY: fix memleak and work in 1 pass

Here's a patch reimplementing the flipY thing in DOMElementToImageSurface, so that the flipping in the same pass as the pixel conversion, not as a separate pass. It also fixes the memleak discussed here: memory usage stays constant (after reasonable initial increase) during the playback of the movie, in the scene linked to here.

There's one comment I didn't understand in the original code:

-        PRUint8* src = surf->Data();
-        PRUint8* dst = surf->Data();
-
-        // this wants some SSE love
-
-        for (int j = 0; j < height; j++) {
-            src = surf->Data() + j * surf->Stride();
-            // note that dst's stride is always tightly packed

Since both src and dst are coming from the same 'surf', I guess that both are tightly packed then? So why did this code mention and rely on the tight packing for dst and not for src? Afraid I'm missing something. I tested in many scenes and my code seems to be working, though.
Assignee: nobody → bjacob
Status: NEW → ASSIGNED
Attachment #452320 - Flags: review?(vladimir)
Comment on attachment 452320 [details] [diff] [review]
image flipY: fix memleak and work in 1 pass

nope, you're not missing anything -- it's referring to the stride, we assume that dst is packed, but we don't make any assumptions about src (which is why we multiply stride by the y index).  We could, though, if it would help.
Attachment #452320 - Flags: review?(vladimir) → review+
OK. Thanks for the super quick reviewing, this is currently on the tryserver. 

There's no need to make assumptions on packing because that doesn't simplify the inner loop.
@ Charles:

Did you notice that the movie is flashing, in Minefield? It seems to intermittently show different, blueish movie frames for a fraction of a second, then the movie continues as normal. I also get a lot of error messages (see JS error console):

EnableVertexAttribArray: index out of range
VertexAttribPointer: index out of range - -1 >= 16
DisableVertexAttribArray: index out of range

Here the -1 is actually the max GLuint value, 0xffffffff. Any idea where these out of range indices are coming from?

Other than that, the scene is playing smoothly.
(Reporter)

Comment 12

8 years ago
Nope, those are all new problems -- haven't noticed this problem at all in my tests, the video always played smooth on the first run and then just choppy on the second run as it ate memory, at no point did it tear, change color or cause warnings/errors while rendering.
Created attachment 452361 [details] [diff] [review]
image flipY: fix memleak and work in 1 pass, updated

Just for the record, a little update, was not correctly handling the 24bpp case.

Carrying forward Vlad's r+.

Doesn't make any difference wrt the glitches and error messages mentioned in my previous comment.
Attachment #452361 - Flags: review+
http://hg.mozilla.org/mozilla-central/rev/8259b0b9337a
Status: ASSIGNED → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED

Comment 15

8 years ago
Works for me here on Linux: http://code.bocoup.com/webgl/WebGL-mem-texture-underload.gif
Hehe, thanks :)
(Reporter)

Comment 17

8 years ago
Thanks everyone for the quick response, got my audio build up to date and working great here, no visual glitches to report!
ok, let me still describe the glitches I'm observing in case this might ring a bell. Certain frames of the video (e.g. one every 10 frames, but it's very irregular) are rendered blended with the same frame upside-down and light-blueish (as if e.g. the blue channel was filled with 255). In other words, there always is the correct frame, the glitch is that in addition to it, a "bad" frame is sometimes being rendered too and blended with it.

I am using the proprietary nvidia driver on linux x86-64. Given the level of attention that nvidia pays to this platform, this could be a driver bug.
That doesn't sound like a driver bug, it sounds like the decoder is writing to the buffer and doing some in-between-frames work while we're trying to read from it.
Duplicate of this bug: 545909
You need to log in before you can comment on or make changes to this bug.