589809 - [D2D] More corruption and lockups in the UI

Status: RESOLVED FIXED

(Core :: Graphics, defect)

Description

[Benjamin Smedberg]

+++ This bug was initially created as a clone of Bug #586909 +++

The attachment shows the results, shortly afterwards it can crash (though it fails to submit a report) or the UI locks up and has to be manually closed.

I see this still on nightly builds, and the symptoms are, after 2-4 hours of browser use:

* text areas stop drawing their borders and backgrounds
* images stop drawing in all cases
* crash, with crash reporter, but the crash reports are empty/useless

I'm running the browser in a debugger to try and catch first-chance exceptions: I found bug 589805, but I don't *think* that is related.

Comment 1

[Martijn Wargers (dead)]

(In reply to comment #0)
> +++ This bug was initially created as a clone of Bug #586909 +++
> 
> The attachment shows the results, 

Which attachment?

Comment 2

[Benjamin Smedberg]

I'm getting first-chance exceptions _com_error with the following stack:

 	KernelBase.dll!_RaiseException@16() 	
 	msvcrt.dll!__CxxThrowException@8() 	
 	d3d10_1core.dll!ThrowFailure() 	
 	d3d10_1core.dll!CDevice::CreateTexture2D_Worker() 	
 	d3d10_1core.dll!CDevice::ID3D10Device1_CreateTexture2D_() 	
 	d3d10_1core.dll!NMultithread::CDevice::CreateTexture2D() 	
>	xul.dll!cairo_d2d_surface_create(device=0x051a4c00, format=CAIRO_FORMAT_ARGB32, width=0x00000000, height=0x00000000)  Line 3479	C++

This is caught by ::CreateTexture2D_Worker, but is it possible that this is causing a resource leak within the d3d10 code?

Comment 3

[Benjamin Smedberg]

Attachment: Firefox virtual memory report

Here is a virtual memory report I created from a process stopped at OOM. There is a *lot* of heavy fragmentation, e.g. 

  400000	    1000	MEM_COMMIT
  401000	    f000	MEM_FREE
  410000	    1000	MEM_COMMIT
  411000	    f000	MEM_FREE
  420000	    1000	MEM_COMMIT
  421000	    f000	MEM_FREE
  430000	    1000	MEM_COMMIT
  431000	    f000	MEM_FREE

(This is one page allocated, 15 pages free). I'm trying to figure out the least terrible way of figuring out who is allocating those pages.

Comment 4

[Loïc Yhuel]

(In reply to comment #3)
> Created attachment 468438 [details]
> Firefox virtual memory report
> 
> Here is a virtual memory report I created from a process stopped at OOM. There
> is a *lot* of heavy fragmentation, e.g. 

This is what I reported in the original bug : I see leaks of backbuffers, and leaks of 4KB pages. The crash happens when the biggest free virtual memory becomes smaller than a few MB.
I see many of these 4KB pages allocated when resizing the window. The fragmentation here is normal : 16 pages = 64KB, which is the alignment of Windows virtual memory.

Comment 5

[Loïc Yhuel]

Attachment: Fix D2D texture leak, and workaround to allow runnig PIX

The IDXGISurface in _cairo_d2d_get_buffer_texture wasn't released, so the reference count on surface->surface increased. With this change the virtual memory leaks I saw when scrolling and resizing window are fixed.

Workaround to allow running PIX :
When we do "newSurf->backBuf->QueryInterface(&newSurf->surface);", PIX returns the original interface instead of the hooked one, so I swapped newSurf->backBuf and newSurf->surface, to get the two interfaces in the reverse order. In this case PIX returns hooked interfaces which will work when used as argument of hooked methods.

Comment 6

[Bas Schouten (:bas.schouten)]

Comment on attachment 468628 [details] [diff] [review]
Fix D2D texture leak, and workaround to allow runnig PIX

Great job, let's see if this fixes all our last issues!

Comment 7

[Jeff Muizelaar [:jrmuizel]]

This fixes it for me. I'll check it in.

Comment 8

[Jeff Muizelaar [:jrmuizel]]

http://hg.mozilla.org/mozilla-central/rev/a8f18c4dbc50
http://hg.mozilla.org/mozilla-central/rev/00ff5cfe0b0a