Closed Bug 1206193 Opened 9 years ago Closed 8 years ago

ICU configure failure on Windows ASan builds

Categories

(Firefox Build System :: General, defect)

Product:
Firefox Build System
Component:
General
Version:
41 Branch
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: away, Unassigned)

References

Details

1:28.64 intl\icu\target> checking whether the C compiler works... no
 1:28.64 intl\icu\target> configure: error: in `/usr/s/clangcl/obj/asan/intl/icu/target':
 1:28.64 intl\icu\target> configure: error: C compiler cannot create executables
 1:28.64 intl\icu\target> See `config.log' for more details
 
 config.log contains:
 
 configure:2962: checking whether the C compiler works
configure:2984: clang-cl  -Qunused-arguments -fsanitize=address -Z7 -fms-compatibility-version=18.00.30723 -fallback -TC -nologo -D_HAS_EXCEPTIONS=0 -W3 -Gy -FS -wd4244 -wd4267 -wd4819 -we4553 -Zi -UDEBUG -DNDEBUG -O1 -Oi -MD  -DU_USING_ICU_NAMESPACE=0 -DU_NO_DEFAULT_INCLUDE_UTF_HEADERS=1 -DUCONFIG_NO_LEGACY_CONVERSION -DUCONFIG_NO_TRANSLITERATION -DUCONFIG_NO_REGULAR_EXPRESSIONS -DUCONFIG_NO_BREAK_ITERATION -DUCONFIG_NO_IDNA -DU_CHARSET_IS_UTF8 -I/usr/s/clangcl/intl/icu/source/common -I/usr/s/clangcl/intl/icu/source/i18n -Qunused-arguments  -DEBUG -DEBUGTYPE:CV clang_rt.asan_dynamic-i386.lib clang_rt.asan_dynamic_runtime_thunk-i386.lib -DYNAMICBASE conftest.c  >&5
In file included from <built-in>:297:
<command line>(15,17) :  warning(clang): ISO C99 requires whitespace after the macro name [-Wc99-extensions]
#define EBUGTYPE:CV 1
                ^
1 warning generated.
configure:2988: $? = 0
configure:3026: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "ICU"
| #define PACKAGE_TARNAME "International Components for Unicode"
| #define PACKAGE_VERSION "55.1"
| #define PACKAGE_STRING "ICU 55.1"
| #define PACKAGE_BUGREPORT "http://icu-project.org/bugs"
| #define PACKAGE_URL "http://icu-project.org"
| /* end confdefs.h.  */
| 
| int
| main ()
| {
| 
|   ;
|   return 0;
| }
configure:3031: error: in `/usr/s/clangcl/obj/asan/intl/icu/target':
configure:3033: error: C compiler cannot create executables
See `config.log' for more details
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj/asan
ac_add_options --enable-warnings-as-errors
# ac_add_options --without-intl-api

export CC=clang-cl
export CXX=clang-cl
export CFLAGS="-fsanitize=address -Z7"
export CXXFLAGS="-fsanitize=address -Z7"

ac_add_options --enable-address-sanitizer
ac_add_options --disable-jemalloc
export LDFLAGS="clang_rt.asan_dynamic-i386.lib clang_rt.asan_dynamic_runtime_thunk-i386.lib"
export HOST_CFLAGS=" "
export HOST_CXXFLAGS=" "
export HOST_LDFLAGS=" "
The root of this problem appears to be that we're passing linker flags to the compiler:
-DEBUG -DEBUGTYPE:CV clang_rt.asan_dynamic-i386.lib clang_rt.asan_dynamic_runtime_thunk-i386.lib -DYNAMICBASE

-DEBUG gets interpreted as "#define EBUG" which is weird but benign.
-DEBUGTYPE:CV gets interpreted as "#define EBUGTYPE:CV" which causes the error above.

Glandium suggested that I remove -DEBUGTYPE:CV since it's an ancient flag that defaults to the right thing nowadays, but removing it wasn't enough, since the "clang_rt.asan_dynamic-i386.lib" parameter still confuses the compiler.
The cl driver is pretty permissive in handling arguments that it does not understand, but clang-cl doesn't accept those by default, and in fact if this was not being interpreted as -D, you'd get a compiler error about the unrecognized flag.

Removing -DEBUGTYPE:CV makes sense to me.

About the .lib issues, I have previously fixed the clang-cl driver to pass those correctly to the linker.  It's possible that I have missed something there.  Is there a minimal test case for that part of the bug?  (Please specify where the .lib files are on your disk relative to the source file and the compiler.  Also it would be helpful if you include the -### output.  Thanks!)
My memory is a little fuzzy since I encountered this a while ago and forgot to file the bug, but.

I think the lib problems are not of the sort that you have in mind. I think what happened is clang-cl thought it was being asked to produce clang_rt.asan_dynamic-i386.exe, so configure didn't find the conftest.exe that it was looking for.

If that's not helpful then I can re-run the experiment and paste actual output when I have time.
Hmm, that is not at all what I would expect from what the code tries to do, and unfortunately I have no idea what would cause such a bug without seeing a test case.  I'd appreciate if you could get some precise STRs here.  Thanks!
My STR are:
- build clang-cl and asan, have them on your path
- set your mozconfig to comment 1
- mach configure

Let me know if you need more...
I was hoping for something like:

echo 'int main() { return 0; }' > test.cpp
clang-cl <args that break the build go here> test.cpp

:-)
Well, you could look at the command line from comment 0, but reproducing that is not really "interesting". The real question is why we ran that command in the first place. This is a build system issue.
The core problem seems to be that we are passing the mozconfig's LDFLAGS to the compiler (not linker). This would fail with cl as well (actually, it fails more loudly):

$ echo 'int main() { return 0; }' > test.cpp

$ ls
test.cpp

$ cl  -Qunused-arguments -fsanitize=address -Z7 -fms-compatibility-version=18.0
0.30723 -fallback -TC -nologo -D_HAS_EXCEPTIONS=0 -W3 -Gy -FS -O1 -Oi -MD  -DEB
UG -DEBUGTYPE:CV clang_rt.asan_dynamic-i386.lib clang_rt.asan_dynamic_runtime_t
hunk-i386.lib -DYNAMICBASE test.cpp
cl : Command line warning D9002 : ignoring unknown option '-Qunused-arguments'
cl : Command line warning D9002 : ignoring unknown option '-fsanitize=address'
cl : Command line warning D9002 : ignoring unknown option '-fms-compatibility-ve
rsion=18.00.30723'
cl : Command line warning D9002 : ignoring unknown option '-fallback'
clang_rt.asan_dynamic-i386.lib
c1 : fatal error C1083: Cannot open source file: 'clang_rt.asan_dynamic-i386.lib
': No such file or directory
clang_rt.asan_dynamic_runtime_thunk-i386.lib
c1 : fatal error C1083: Cannot open source file: 'clang_rt.asan_dynamic_runtime_
thunk-i386.lib': No such file or directory
test.cpp
Generating Code...

$ ls
test.cpp  test.obj

$ rm test.obj

$ clang-cl  -Qunused-arguments -fsanitize=address -Z7 -fms-compatibility-versio
n=18.00.30723 -fallback -TC -nologo -D_HAS_EXCEPTIONS=0 -W3 -Gy -FS -O1 -Oi -MD
  -DEBUG -DEBUGTYPE:CV clang_rt.asan_dynamic-i386.lib clang_rt.asan_dynamic_run
time_thunk-i386.lib -DYNAMICBASE test.cpp
In file included from <built-in>:297:
<command line>(5,17) :  warning(clang): ISO C99 requires whitespace after the
      macro name [-Wc99-extensions]
#define EBUGTYPE:CV 1
                ^
1 warning generated.

$ ls
clang_rt.asan_dynamic-i386.exe  clang_rt.asan_dynamic-i386.pdb  test.cpp
Hmm, I think this is http://llvm.org/PR20923 which should be fixed.

Have you verified that clang_rt.asan_dynamic-i386.lib exists in a directory in $LIB (or in the current working directory)?
Flags: needinfo?(dmajor)
> Have you verified that clang_rt.asan_dynamic-i386.lib exists in a directory
> in $LIB (or in the current working directory)?

Yes, it exists in one of my $LIB directories.
Flags: needinfo?(dmajor)
How does it go if you put the .lib file names *after* test.cpp on the command line ?
(In reply to Mike Hommey [:glandium] from comment #12)
> How does it go if you put the .lib file names *after* test.cpp on the
> command line ?

That seems to work for clang-cl, but cl still complains about Cannot open source file: 'clang_rt.asan_dynamic-i386.lib'.
OK, I can reproduce this on clang-cl.  Investigating...
(In reply to David Major [:dmajor] from comment #13)
> (In reply to Mike Hommey [:glandium] from comment #12)
> > How does it go if you put the .lib file names *after* test.cpp on the
> > command line ?
> 
> That seems to work for clang-cl, but cl still complains about Cannot open
> source file: 'clang_rt.asan_dynamic-i386.lib'.

You lost me here. Why would cl's behavior matter?
> Why would cl's behavior matter?

clang-cl is intended to behave like cl. At some point in the future, clang-cl may start matching cl's behavior on this, and we'd be back to being busted.
sure, but if moving clang_rt.asan_dynamic-i386.lib last works, then you can make it -link clang_rt.asan_dynamic-i386.lib and make cl happy.
(In reply to Mike Hommey [:glandium] from comment #17)
> sure, but if moving clang_rt.asan_dynamic-i386.lib last works, then you can
> make it -link clang_rt.asan_dynamic-i386.lib and make cl happy.

If we can do that, then that is preferable.  My patch for fixing this in clang is turning into a huge mess.  It seems like clang's job construction code has hardcoded assumptions about the first argument to the linker always being used to derive the name of the output executable, so I am not in a rush to fix it in clang if we can easily work around it on our side.  :-)
Ehsan, please do not spend any more time on this while there are still much more important clang bugs outstanding :-) I kind of regret that this bug has gone on even this far.
Can anyone confirm that comment 17 works?
bug 1239083 should make this moot by not running ICU's configure anymore.
Depends on: 1239083
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.