760323 - Runtime fails to create WebGL contexts without Direct X End-User Runtime Installed

Status: VERIFIED FIXED

(Core :: Graphics: CanvasWebGL, defect)

Description

[Myk Melez [:myk] [@mykmelez]]

We recently fixed bug 749459, which prevented webapps from creating WebGL contexts on all Windows machines that support WebGL (i.e. on which apps in the browser can create WebGL contexts).

But there are still some machines on which webapps fail to create WebGL contexts, like a Windows 7 machine in the QA Lab in Mozilla's headquarters in Mountain View.  Webapps on those machines should be able to create WebGL contexts, since their counterparts in the browser can do so.

Comment 1

[Jason Smith [:jsmith]]

Confirmed that this is still happening on my machine. Probably need to get a debug build at this point and try to debug this.

Comment 2

[Jason Smith [:jsmith]]

Nominating for k9o - We can't block WebGL on specific windows machines. I know we are having trouble diagnosing the problem here, but I think we need to figure out what's going wrong here is a k9o blocker, cause the behavior here seems strange.

Comment 3

[JP Rosevear [:jpr]]

Jeff, can you go see the machine Myk has to debug?

Comment 4

[Jason Smith [:jsmith]]

Removing qawanted (that's more appropriate to flag when a reduced test case is needed, although at this point, tag-team debugging is probably the best course of action) - this is reproducible on a machine in the test lab and reproducible on my machine. To the person who is going to investigate this - if you are in Mt View, let's meet up and debug this.

Comment 5

[Jason Smith [:jsmith]]

Jeff - Is there a time we could meet up and debug this?

Comment 6

[Jason Smith [:jsmith]]

Jeff - Did you have any luck reproducing this bug?

Comment 7

[Kelsey Gilbert [:jgilbert]]

I can't repro it currently, though I suspect it's because I disabled my intel card. I'll switch that back on in the BIOS and see what I get.

Comment 8

[Jason Smith [:jsmith]]

Jeff - I figured out the root cause on my work machine. So in the process of learning the build process for Firefox, I installed Direct X End-User Runtime here - http://www.microsoft.com/en-us/download/confirmation.aspx?id=35, re-ran ed's app here - ed.agadak.net/app, and successfully got webgl to initialize in the runtime.

Comment 9

[Benoit Jacob [:bjacob] (mostly away)]

Interesting. This is exactly the problem that we were supposed to have fixed by shipping d3dx9_43.dll and D3DCompiler.dll (the two relevant DXSDK files) with firefox. (That was bug 630628).

That's why in bug 749459 coment 19 I said:

> Here's the full list of DLLs that we ship with Firefox, that are needed for WebGL:
> libEGL.dll
> libGLESv2.dll
> D3DCompiler_43.dll
> d3dx9_43.dll

So bug 749459 fixed the loading of libEGL and libGLESv2, and now we apparently have the same issue with D3DCompiler_43.dll and d3dx9_43.dll.

IIUC we load these two latter DLLs by linking statically to corresponding .lib files: d3dx9.lib and D3DCompiler.lib, in gfx/angle/src/libGLESv2/Makefile.in :
  http://mxr.mozilla.org/mozilla-central/source/gfx/angle/src/libGLESv2/Makefile.in#145

So why is this not working for the Web App Runtime while it is working for Firefox?

Comment 10

[Andrew Williamson [:eviljeff]]

Is there anything I can do on my end to work out why I'm still experiencing the issue?  Nightly proper works without an issue (as do other versions of Firefox I have installed).

Comment 11

[Jason Smith [:jsmith]]

(In reply to Andrew Williamson [:eviljeff] from comment #10)
> Is there anything I can do on my end to work out why I'm still experiencing
> the issue?  Nightly proper works without an issue (as do other versions of
> Firefox I have installed).

If you install the Direct X Runtime here - http://www.microsoft.com/en-us/download/confirmation.aspx?id=35, it should fix the problem as a work-around in the runtime.

Comment 12

[Benjamin Smedberg]

This isn't working because the Firefox directory is not in the DLL load path. Any dependent DLLs need to either be listed in dependentlibs.list so that they are loaded automatically with xul.dll, or be loaded by hand using file->Load. In this case we can probably just add these two libraries as explicit load targets in GLLibraryEGL::EnsureInitialized

Comment 13

[Benoit Jacob [:bjacob] (mostly away)]

We don't want these libraries to be loaded automatically whenever xul.dll, as that would slow start-up down for all users regardless of whether they use WebGL, right? So yes, doing it like was done for libGLESv2 in GLLibraryEGL::EnsureInitialized seems like the right way. The tricky part is that the precise filename is determined at configure-time, we don't want to hardcode the _43 version there.

Comment 14

[Benoit Jacob [:bjacob] (mostly away)]

Question: the installed web apps have to be able to find xul.dll anyway. I don't know how they find it, but couldn't the same mechanism allow them to find the other DLLs as well? I.e. couldn't we just add the firefox bin directory to their LD_LIBRARY_PATH (or however it's called on Windows)?

Comment 15

[Benjamin Smedberg]

No, we cannot safely modify PATH (inherets, leading to frankenbuilds and possible security holes) or use SetDllDirectory (used for other purposes). We load xul.dll explicitly with LoadLibrary and dependentlibs.list to load its dependencies.

Comment 16

[Jason Smith [:jsmith]]

Flagging for tracking firefox 16, as we firmly shooting for that in our 1st release of desktop web runtime.

GFX team - Are you guys good to implement this? Is there anything you need from me (QA)? We'd like to have WebGL working correctly in the web runtime on windows, especially since there are top apps on marketplace that use WebGL.

Comment 17

[Benoit Jacob [:bjacob] (mostly away)]

I have one more question for Benjamin.

Currently, these DLLs are loaded only by statically linking to corresponding .lib's that apparently have these DLLs as dependencies (have to check that, but that's how it looks like from the Makefile.in).

So what is our best course of action here:
 - should we make our own version of the D3DX9.LIB and D3DCOMPILER.LIB with the corrected dependency?
 - or should we quit using these .libs and load the DLLs explicitly (patching ANGLE)?

Comment 18

[Benoit Jacob [:bjacob] (mostly away)]

(Hm OK, comment 12 suggests the latter, and suggests doing it in our own EGL loading code instead of in ANGLE).

Comment 19

[Benjamin Smedberg]

The .lib is an import lib for the DLL (you never link directly against a .dll, you link against its import lib). When you link against that the dynamic linker encodes the DLL dependency; in this case, libGLESv2.dll depends on d3dx9_43.dll being found and loaded. By explicitly calling LoadLibrary for d3dx9_43.dll you can shortcut the normal PATH finding mechanism for the DLL.

Comment 20

[Benoit Jacob [:bjacob] (mostly away)]

I'm writing the patch; self-assigning.

Comment 21

[Benoit Jacob [:bjacob] (mostly away)]

Attachment: load DXSDK DLLs explicitly

This should work, if I understood correctly what was said above.

Comment 22

[Benoit Jacob [:bjacob] (mostly away)]

Attachment: load DXSDK DLLs explicitly

Comment 23

[Benoit Jacob [:bjacob] (mostly away)]

Part of this tryserver push: https://tbpl.mozilla.org/?tree=Try&rev=35d614a1c367

Comment 24

[Benoit Jacob [:bjacob] (mostly away)]

Attachment: load DXSDK DLLs explicitly

There was a typo in the previous version preventing WebGL contexts from creating at all.

New try push: https://tbpl.mozilla.org/?tree=Try&rev=dda1beade23c

Comment 25

[Benjamin Smedberg]

Comment on attachment 639212 [details] [diff] [review]
load DXSDK DLLs explicitly

>Bug 760323 - Runtime fails to create WebGL contexts without Direct X End-User Runtime Installed - r=???

Please add description of what the patch does: "Load the Direct X runtime programmatically so that the EGL libraries can locate it when running webapprt and the Firefox install dir is not in the DLL load path."

>diff --git a/gfx/gl/GLLibraryEGL.cpp b/gfx/gl/GLLibraryEGL.cpp

>+#ifdef XP_WIN
>+// see the comment in GLLibraryEGL::EnsureInitialized() for the rationale here.
>+static PRLibrary* LoadLibraryForEGLOnWindows(const nsAString& filename)

style nit, please put "static PRLibrary*" on its own line, so that "LoadLibraryForEGL" is in column 0.

>+{
>+    nsresult rv;
>+    nsCOMPtr<nsIFile> file;
>+
>+    nsCOMPtr<nsIProperties> dirService =
>+        do_GetService(NS_DIRECTORY_SERVICE_CONTRACTID);
>+    if (!dirService)
>+        return nsnull;
>+
>+    rv = dirService->Get(NS_GRE_DIR, NS_GET_IID(nsIFile),
>+                            getter_AddRefs(file));

Please use nsDirectoryServiceUtils.h NS_GetSpecialDirectory to make this code cleaner.

Comment 26

[Benoit Jacob [:bjacob] (mostly away)]

Attachment: load DXSDK DLLs explicitly (updated)

Comment 27

[Benoit Jacob [:bjacob] (mostly away)]

On try, because somehow all my windows local builds fail to link at the moment.
https://tbpl.mozilla.org/?tree=Try&rev=4862f608a8d7

Comment 28

[Jason Smith [:jsmith]]

(In reply to Benoit Jacob [:bjacob] from comment #27)
> On try, because somehow all my windows local builds fail to link at the
> moment.
> https://tbpl.mozilla.org/?tree=Try&rev=4862f608a8d7

I can confirm this fixes one of my windows 7 machines that does not have direct X end-user runtime installed.

Comment 29

[Benoit Jacob [:bjacob] (mostly away)]

http://hg.mozilla.org/integration/mozilla-inbound/rev/2e6bc1480efd

Do you want this backported?

Comment 30

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/2e6bc1480efd

Comment 31

[Andrew Williamson [:eviljeff]]

(In reply to Jason Smith [:jsmith] from comment #28)
> (In reply to Benoit Jacob [:bjacob] from comment #27)
> > On try, because somehow all my windows local builds fail to link at the
> > moment.
> > https://tbpl.mozilla.org/?tree=Try&rev=4862f608a8d7
> 
> I can confirm this fixes one of my windows 7 machines that does not have
> direct X end-user runtime installed.

I can confirm the fix in Nightly 2012-07-08

Comment 32

[Jason Smith [:jsmith]]

(In reply to Andrew Williamson [:eviljeff] from comment #31)
> (In reply to Jason Smith [:jsmith] from comment #28)
> > (In reply to Benoit Jacob [:bjacob] from comment #27)
> > > On try, because somehow all my windows local builds fail to link at the
> > > moment.
> > > https://tbpl.mozilla.org/?tree=Try&rev=4862f608a8d7
> > 
> > I can confirm this fixes one of my windows 7 machines that does not have
> > direct X end-user runtime installed.
> 
> I can confirm the fix in Nightly 2012-07-08

Awesome. Thanks for testing this. Feel free to review & approve the WebGL apps in the marketplace now.

Comment 33

[Andrew Williamson [:eviljeff]]

(In reply to Jason Smith [:jsmith] from comment #32)
> Awesome. Thanks for testing this. Feel free to review & approve the WebGL
> apps in the marketplace now.

Already done!  Apart from the 2 or 3 that use Flash as well so are blocked on bug 771090 now instead.

Comment 34

[Benoit Jacob [:bjacob] (mostly away)]

This is only fixed for Firefox 16 now. Do you want this backported to 15? What is the version that matters for the marketplace?

Comment 35

[Benoit Jacob [:bjacob] (mostly away)]

(In reply to Jason Smith [:jsmith] from comment #16)
> Flagging for tracking firefox 16, as we firmly shooting for that in our 1st
> release of desktop web runtime.

Oh, alright. Good then.

Comment 36

[Jason Smith [:jsmith]]

(In reply to Benoit Jacob [:bjacob] from comment #35)
> (In reply to Jason Smith [:jsmith] from comment #16)
> > Flagging for tracking firefox 16, as we firmly shooting for that in our 1st
> > release of desktop web runtime.
> 
> Oh, alright. Good then.

Right. I don't think you need to backport this to FF 15, as we're likely going to backout of FF 15. If the story changes, I'll let you know.

Comment 37

[Julian Reschke]

This seems to break compilation for me:

c:/projects/mozilla/src/obj-i686-pc-mingw32/_virtualenv/Scripts/python.exe -O /c/projects/mozilla/src/build/cl.py cl -FoGLLibraryEGL.obj -c -D_HAS_EXCEPTIONS=0 -I../../dist/stl_wrappers  -DMOZ_D3DX9_DLL= -DMOZ_D3DCOMPILER_DLL= -DMOZILLA_INTERNAL_API -D_IMPL_NS_COM -DEXPORT_XPT_API -DEXPORT_XPTC_API -D_IMPL_NS_GFX -D_IMPL_NS_WIDGET -DIMPL_XREAPI -DIMPL_NS_NET -DIMPL_THEBES  -I/c/projects/mozilla/src/gfx/gl -I. -I../../dist/include  -Ic:/projects/mozilla/src/obj-i686-pc-mingw32/dist/include/nspr -Ic:/projects/mozilla/src/obj-i686-pc-mingw32/dist/include/nss        -TP -nologo -W3 -Gy -Fdgenerated.pdb -wd4800 -we4553 -GR-  -DDEBUG -D_DEBUG -DTRACING -Zi -O1 -Oy-  -Ic:/projects/mozilla/src/obj-i686-pc-mingw32/dist/include/cairo   -MDd           -FI ../../dist/include/mozilla-config.h -DMOZILLA_CLIENT /c/projects/mozilla/src/gfx/gl/GLLibraryEGL.cpp
GLLibraryEGL.cpp

c:/projects/mozilla/src/gfx/gl/GLLibraryEGL.cpp(105) : warning C4003: Nicht gengend bergebene Parameter fr das Makro 'NS_STRINGIFY_HELPER'

c:/projects/mozilla/src/gfx/gl/GLLibraryEGL.cpp(105) : warning C4003: Nicht gengend bergebene Parameter fr das Makro 'NS_LL'

c:/projects/mozilla/src/gfx/gl/GLLibraryEGL.cpp(105) : error C2065: 'L': nichtdeklarierter Bezeichner

c:/projects/mozilla/src/gfx/gl/GLLibraryEGL.cpp(105) : error C2065: 'L': nichtdeklarierter Bezeichner

c:/projects/mozilla/src/gfx/gl/GLLibraryEGL.cpp(105) : error C2070: ''unknown-type'': Ungltiger sizeof-Operand

c:/projects/mozilla/src/gfx/gl/GLLibraryEGL.cpp(111) : warning C4003: Nicht gengend bergebene Parameter fr das Makro 'NS_STRINGIFY_HELPER'

c:/projects/mozilla/src/gfx/gl/GLLibraryEGL.cpp(111) : warning C4003: Nicht gengend bergebene Parameter fr das Makro 'NS_LL'

c:/projects/mozilla/src/gfx/gl/GLLibraryEGL.cpp(111) : error C2065: 'L': nichtdeklarierter Bezeichner

c:/projects/mozilla/src/gfx/gl/GLLibraryEGL.cpp(111) : error C2065: 'L': nichtdeklarierter Bezeichner

c:/projects/mozilla/src/gfx/gl/GLLibraryEGL.cpp(111) : error C2070: ''unknown-type'': Ungltiger sizeof-Operand


my mozconfig:

mk_add_options MOZ_CO_PROJECT=browser
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-@CONFIG_GUESS@
# mk_add_options MOZ_MAKE_FLAGS="-j4"
ac_add_options --enable-application=browser
ac_add_options --disable-xpconnect-idispatch
ac_add_options --disable-activex
ac_add_options --disable-activex-scripting
ac_add_options --disable-accessibility
ac_add_options --disable-angle
# ac_add_options --disable-webm
ac_add_options --enable-debug
# ac_add_options --disable-optimize
mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) $(MOZ_OBJDIR)/_profile/pgo/profileserver.py'

Comment 38

[Benoit Jacob [:bjacob] (mostly away)]

(In reply to Julian Reschke from comment #37)
> This seems to break compilation for me:
> 
> c:/projects/mozilla/src/obj-i686-pc-mingw32/_virtualenv/Scripts/python.exe

Is MinGW a supported platform at all? I thought our windows build system was MSVC-only, with the aim to support Clang in the future ASAP?

> c:/projects/mozilla/src/gfx/gl/GLLibraryEGL.cpp(105) : warning C4003: Nicht
> gengend bergebene Parameter fr das Makro 'NS_STRINGIFY_HELPER'

NS_STRINGIFY_HELPER is defined here:
http://mxr.mozilla.org/mozilla-central/source/xpcom/base/nscore.h#372

I'd suggest looking into the preprocessed form of this file to figure why it's not being defined in your case. Maybe then propose a patch to this file (Core->XPCOM) but if your platform is not officially supported, the patch might not be accepted.

Update: oh, I see here
  https://developer.mozilla.org/en/Supported_build_configurations
that minGW is a tier 3 platform, "maintained by Chris Seawood".

Comment 39

[Benoit Jacob [:bjacob] (mostly away)]

Alright, sorry for the confusion. Julian is using the standard MSVC-based build-system, the mingw here just refers to the standard mozillabuild environment which is mingw based.

The problem with Julian's config is he's configured with --disable-angle, which is broken.

We should remove that option. The only choice should be to enable or disable WebGL entirely.

Comment 40

[Benoit Jacob [:bjacob] (mostly away)]

Attachment: fix the --disable-webgl build

Sorry, in fact, the --disable-webgl build itself was broken. Only reason why we didn't notice is the delay until the next m-c -> thunderbird merge.

Comment 41

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Comment on attachment 640615 [details] [diff] [review]
fix the --disable-webgl build

Review of attachment 640615 [details] [diff] [review]:
-----------------------------------------------------------------

Sure, though we should really just kill off --disable-webgl entirely.

Comment 42

[Benoit Jacob [:bjacob] (mostly away)]

Thunderbird still uses --disables-webgl; I would make bug 741742 (removing the DXSDK DLLs dependencies) a blocker for making WebGL mandatory, as as long as WebGL requires shipping these two large proprietary DLLs, there exist quite valid reasons for wanting to disable it.

Comment 43

[Julian Reschke]

I tried with the attachment and

  ac_add_options --disable-webgl

but still get the same error.

Comment 44

[Benoit Jacob [:bjacob] (mostly away)]

Oh, it appears that --disable-webgl has no effect here... at least I still have MOZ_WEBGL=1 in my autoconf.mk

Comment 45

[Benoit Jacob [:bjacob] (mostly away)]

OK, turns out that there is no --disable-webgl option... that's what we need to fix here.

Comment 46

[Benoit Jacob [:bjacob] (mostly away)]

Patch in bug 772457 adds the missing --disable-webgl option. Together with the patch here, they should fix the build, you'll just have to use --disable-webgl instead of --disable-angle (which goes away).

Comment 47

[Benoit Jacob [:bjacob] (mostly away)]

http://hg.mozilla.org/integration/mozilla-inbound/rev/ced403b699b2

and the patch in bug 772457 is landed as well. So all should be fixed now. Please retry.

Comment 48

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/ced403b699b2

Comment 49

[Honza Bambas (:mayhemer)]

(In reply to Benoit Jacob [:bjacob] from comment #47)
> http://hg.mozilla.org/integration/mozilla-inbound/rev/ced403b699b2
> 
> and the patch in bug 772457 is landed as well. So all should be fixed now.
> Please retry.

So, with current (to this moment) m-c and --disable-webgl in my mozconfig I still cannot build, xul.dll link fails with many mozilla::WebGLExtension::* unresolved externals.  Clobbered build.  What am I doing wrong?

Comment 50

[Benoit Jacob [:bjacob] (mostly away)]

You're probably not doing anything wrong, we probably are missing WebGL extension symbols in WebGLContextNotSupported.cpp. It would help to have the actual linker errors (all the unresolved externals).

Comment 51

[Honza Bambas (:mayhemer)]

Attachment: build log (unresolved externals mozilla::WebGLContext::*)

Here it is.

Comment 52

[Honza Bambas (:mayhemer)]

Comment on attachment 641148 [details]
build log (unresolved externals mozilla::WebGLContext::*)

And mozconfig, to be sure everything is OK:

# Options for client.mk.
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/_obj-browser-debug
mk_add_options MOZ_MAKE_FLAGS=-j9

# Options for 'configure' (same as command-line options).
#ac_add_options --enable-jemalloc

export MOZ_DEBUG_SYMBOLS=1

ac_add_options --enable-application=browser
ac_add_options --enable-debug
ac_add_options --disable-optimize
ac_add_options --enable-shared
ac_add_options --disable-static
ac_add_options --enable-tests
ac_add_options --enable-logging
ac_add_options --enable-visual-event-tracer
ac_add_options --disable-installer
ac_add_options --enable-chrome-format=flat
ac_add_options --disable-crashreporter
ac_add_options --disable-webm
ac_add_options --disable-accessibility
ac_add_options --disable-angle

# 772570
ac_add_options --disable-webrtc
# 772577
ac_add_options --disable-webgl

Comment 53

[Benoit Jacob [:bjacob] (mostly away)]

OK, these linking errors are because some WebGL files, such as WebGLExtensionCompressedTextureS3TC.cpp, are being built even with --disable-webgl. I'm making a patch.

Comment 54

[Benoit Jacob [:bjacob] (mostly away)]

Attachment: try to fix the webgl unresolved externals with --disable-webgl

This will work if removing the WebIDL is enough to prevent the bindings generation. The stuff remains in Bindings.conf where I didn't know how to conditionally disable it.

Comment 55

[Benoit Jacob [:bjacob] (mostly away)]

Attachment: try to fix the webgl unresolved externals with --disable-webgl

Typo: fi -> endif

Building at the moment on linux, hopefully these issues are platform-independent so i can reproduce without rebooting

Comment 56

[Honza Bambas (:mayhemer)]

Comment on attachment 641164 [details] [diff] [review]
try to fix the webgl unresolved externals with --disable-webgl

This patch fixes the last build issue for me (comment 52).

Comment 57

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 641164 [details] [diff] [review]
try to fix the webgl unresolved externals with --disable-webgl

r=me

Comment 58

[Benoit Jacob [:bjacob] (mostly away)]

http://hg.mozilla.org/integration/mozilla-inbound/rev/2ad3e97aa550

Comment 59

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/2ad3e97aa550

Comment 60

[Tom]

So: Is it true that this patch freezes the main thread of my application for ~3s when inializing webgl? This didn't happen in FF12.

Comment 61

[Benoit Jacob [:bjacob] (mostly away)]

I wasn't aware of that issue. Please open a new bug. Is it reproducible in Firefox itself or only in non-Firefox apps? To get more information, the most useful thing you could do is to use the built-in profiler to figure what it's doing during these 3 seconds. https://developer.mozilla.org/en/Performance/Profiling_with_the_Built-in_Profiler

Comment 62

[Tom]

A little update on the bugpost in comment #60:

-If i don't enable webgl.prefer-native-gl in pre-16 builds, I need to copy the mentioned DLL's to the path of the xulrunner-stub in order for GL to work at all.
-I cannot reproduce this bug in Firefox, only in XulRunner.
-The profiler does not yet work in non-FF apps (see error trace on the bottom of this post)
-when i set webgl.prefer-native-gl to true, the freeze doesn't occur in neither 14, 15b or 16a.
-When I put the this.mCanvas.getContext("experimental-webgl"); in a setTimeout, the application starts up immediatly but the thread/UI freezes for about 3/4 seconds. When i don't put it in a timeout, the initial loading of the window is delayed for about this amount of time.

Should I proceed in filing a new bugpost for this? I could very easily create a testcase (window.xul -> browser element -> canvas element -> getContext("experimental-webgl")).

Comment 63

[Andrew Williamson [:eviljeff]]

(In reply to Tom from comment #62)
> Should I proceed in filing a new bugpost for this? 

Yes.  This bug was about the webruntime.