292303 - OpenGL Plugin Stack Overflow

Status: RESOLVED WORKSFORME

(Core Graveyard :: Plug-ins, defect)

Description

[Nicolas Cannasse]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; fr-FR; rv:1.7.7) Gecko/20050414 Firefox/1.0.3
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr-FR; rv:1.7.7) Gecko/20050414 Firefox/1.0.3

I'm currently writing a plugin for Firefox using OpenGL on Win32.
It works pretty well but when I resize the window Firefox close with a Stack
Overflow.

Simply setting the following Pixel Format on the plugin Window should reproduce it :

	PIXELFORMATDESCRIPTOR pfd;
	int format;
	HDC dc = GetDC( hWnd );
	ZeroMemory( &pfd, sizeof( pfd ) );
	pfd.nSize = sizeof( pfd );
	pfd.nVersion = 1;
	pfd.dwFlags = PFD_DRAW_TO_WINDOW | PFD_SUPPORT_OPENGL |PFD_DOUBLEBUFFER;
	pfd.iPixelType = PFD_TYPE_RGBA;
	pfd.cColorBits = 24;
	pfd.cDepthBits = 16;
	pfd.iLayerType = PFD_MAIN_PLANE;
	format = ChoosePixelFormat( dc, &pfd );
	SetPixelFormat( dc, format, &pfd );


Reproducible: Always

Steps to Reproduce:
1. Use a basic plugin
2. on plugin init, use SetPixelFormat to enable OpenGL on the window
3. resize the window, this will cause a stack overflow

Actual Results:  
a SetWindow NPP call is performed, than after a stack overflow occurs

Expected Results:  
resize the window

Comment 1

[William Bumgarner [:zsinj]]

Is this still a valid bug? Try to confirm it with a nightly build
http://www.mozilla.org/developer and report back whether this issue is still
pertinent.

Otherwise, this bug will be marked INVALID for for inactivity in excess of 60
days (pre-1.0.4). I am self-assigning to track this bug and will reassign to
original owner if this bug is still valid.

Comment 2

[lbaer]

This is still very much a problem in Firefox 1.5 on Windows (Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1).  

Comment 3

[Aras Pranckevicius]

After much head banging I found a couple of workarounds. This does not fix the problem of course...

1) initialize OpenGL inside NPP_SetWindow, NOT some time later. This seems to work, but is not possible in my particular case (because at NPP_SetWindow time our datafile is not loaded yet, so we don't know the required GL settings).

2) don't directly use the window passed to NPP_SetWindow. Instead, in NPP_SetWindow, create or resize your own child window to fully cover the parent one. Subclass the parent with empty erasebgnd/paint messages so it never actually paints; and register your own child window class with whatever you need (also, the class might need CS_OWNDC style for better OpenGL). This works nicely so far for me. One caveat though: NPP_SetWindow seems to be called (Firefox 1.5) before the actual resize happens; so to get the new size just take values passed in NPWindow, don't try taking them from window's client rect.

Comment 4

[Tanner M. Young [:tmyoung]]

Is this still an issue in Firefox 3.5?

If it is not, we should set the resolution of the bug to WORKSFORME.

Comment 5

[Tanner M. Young [:tmyoung]]

Resolving as WORKSFORME.  If the issue reappears, we will reopen the bug.