947117 - DMD: link error in optimized builds on Windows

Status: RESOLVED FIXED

(Core :: DMD, defect)

Description

[Nicholas Nethercote [inactive]]

In bug 936784 I discovered that Windows DMD building only works for debug builds.  Non-debug builds fail to link with many errors like this:

  ProgramBinary.obj : error LNK2001: unresolved external symbol __imp___CrtDbgReportW

Comment 1

[Emanuel Hoogeveen [:ehoogeveen]]

I looked into this a while back but didn't really get anywhere as I don't understand enough about the build system. It seems DEBUG gets passed in at some point and this causes various debug logging functions to be used, calling into functions that are only defined in the debug version of the CRT. But I don't know if this is a build system error, or if DMD on Windows relies on something that forces the build into some currently broken pseudo-debug mode.

Comment 2

[Catalin Iacob]

-MDd is being passed to the compiler when compiling at least some libGLESv2 files leading to the linking error. As discussed at http://msdn.microsoft.com/en-us/library/2kzt1wy3.aspx, MDd tells the compiler to use the debug C++ runtime, MD should be used instead for --enable-release.

An example compiler invocation obtained from ./mach build -v gfx/angle/src/libGLESv2/:

0:03.18 c:\Users\Catalin\moz\mozilla-central\config\rules.mk:1031:0$ mozbuild.action.cl main  cl -FoImage.obj -c   -DLIBGLESV2_EXPORTS -DANGLE_BUILD -DNOMINMAX -D_CRT_SECURE_NO_DEPRECATE -DANGLE_DISABLE_TRACE -DCOMPILER_IMPLEMENTATION -D_SECURE_SCL=0 -DANGLE_COMPILE_OPTIMIZATION_LEVEL='D3DCOMPILE_OPTIMIZATION_LEVEL1' -DNO_NSPR_10_SUPPORT -Ic:/Users/Catalin/moz/mozilla-central/gfx/angle/src/libGLESv2 -I. -Ic:/Users/Catalin/moz/mozilla-central/gfx/angle/src/libGLESv2/.. -Ic:/Users/Catalin/moz/mozilla-central/gfx/angle/src/libGLESv2/../../include -Ic:/Users/Catalin/moz/mozilla-central/gfx/angle/src/libGLESv2/../../include/KHR -I../../../../dist/include  -Ic:/Users/Catalin/moz/mozilla-central/obj-i686-pc-mingw32/dist/include/nspr -Ic:/Users/Catalin/moz/mozilla-central/obj-i686-pc-mingw32/dist/include/nss       -MDd           -EHsc  -EHsc  -EHsc -wd4099 -TP -nologo -W3 -Gy -Fdgenerated.pdb -wd4251 -wd4244 -wd4345 -wd4351 -wd4482 -wd4800 -wd4819 -we4553 -GR-  -DNDEBUG -DTRIMMED -Zi -UDEBUG -DNDEBUG -Zi -Od -UDEBUG -DNDEBUG -Oy -I'C:/Program Files (x86)/Microsoft DirectX SDK (June 2010)/include'  c:/Users/Catalin/moz/mozilla-central/gfx/angle/src/libGLESv2/renderer/Image.cpp

Comment 3

[Catalin Iacob]

Attachment: Bug_947117___Don_t_tell_the_compiler_to_use_debug_C___runtimes_just_because_we_re_using_DMD

This patch fixes it for me, everything builds and DMD runs with --enable-release.

It crashes when dumping because some pointer is NULL but that's a separate bug to look into later.

I don't know whether this patch is ok, still need to look at MOZ_NO_DEBUG_RTL and see what that does.

Comment 4

[Catalin Iacob]

I've looked at it more and think the patch is ok.

MOZ_NO_DEBUG_RTL is only defined by netwerk/streamconv/test/moz.build. We could inhibit the wrong usage of -MDd by defining MOZ_NO_DEBUG_RTL but that would just be backward: DMD should not trigger -MDd in the first place.ick 

The code to use -MDd is there since the first version of DMD in https://hg.mozilla.org/mozilla-central/rev/b29ecfdf4255. I suppose it was simply done based on the analogy with NS_TRACE_MALLOC and was never tested since DMD didn't work on Windows back then.

Comment 5

[Nicholas Nethercote [inactive]]

Comment on attachment 8344495 [details] [diff] [review]
Bug_947117___Don_t_tell_the_compiler_to_use_debug_C___runtimes_just_because_we_re_using_DMD

Review of attachment 8344495 [details] [diff] [review]:
-----------------------------------------------------------------

> The code to use -MDd is there since the first version of DMD in
> https://hg.mozilla.org/mozilla-central/rev/b29ecfdf4255. I suppose
> it was simply done based on the analogy with NS_TRACE_MALLOC and
> was never tested since DMD didn't work on Windows back then.

That's correct.

I don't understand this stuff, and non-trivial Makefile changes need to be reviewed by a build peer.  glandium to the rescue.

Is it still crashing when dumping?  Hopefully not.  And I assume you've tested both opt and debug builds?

Thanks.

Comment 6

[Catalin Iacob]

(In reply to Nicholas Nethercote [:njn] from comment #5)
> Is it still crashing when dumping?  Hopefully not.  

It still is. I've narrowed it down a bit: from what I see in the debugger nsStackWalk is pushing a 0xffffffff pointer as Entry::mPc into DMD and DMD interprets that as Entry::kUnused which has the same value; this screws up DMD's internal state.

I'll file a new bug when I get some time and CC you on it.

Finally, there's the question what happens to Windows users that try DMD and don't have symbols because they didn't build themselves (they have no PDBs). I suppose there will only be DLL names and addresses in the stack trace which is not very useful. It's possible to fetch PDBs from a symbol server but that's for another bug.

> And I assume you've tested both opt and debug builds?

I didn't do a debug build again with the patch since it takes so long. This is tested only with --enable-dmd and --enable-release.

But if MOZ_DEBUG is set nothing changes, all changes are in an ifndef MOZ_DEBUG block; if MOZ_DEBUG == 1 the block gets activated before and after the patch anyway.

Comment 7

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/a9804d9b47c7

Comment 8

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/a9804d9b47c7