1412983 - Unable to build unless HOST_CC=clang and HOST_CXX=clang++ are set in .mozconfig on MacOS

Status: RESOLVED WORKSFORME

(Firefox Build System :: Android Studio and Gradle Integration, defect)

Description

[Randall Barker [:rbarker]]

When building for Android on MacOS, the build will fail with:

10:21.70 In file included from /Volumes/firefox/vr/layout/style/test/ListCSSProperties.cpp:11:
10:21.70 In file included from /Volumes/firefox/vr/obj-arm-linux-androideabi/dist/include/mozilla/ArrayUtils.h:14:
10:21.70 In file included from /Volumes/firefox/vr/obj-arm-linux-androideabi/dist/include/mozilla/Assertions.h:16:
10:21.70 In file included from /Volumes/firefox/vr/obj-arm-linux-androideabi/dist/include/mozilla/Attributes.h:12:
10:21.71 /Volumes/firefox/vr/obj-arm-linux-androideabi/dist/include/mozilla/Compiler.h:49:12: fatal error: 'cstddef' file not found
10:21.71 #  include <cstddef>
10:21.71            ^~~~~~~~~
10:21.75 1 error generated.
10:21.75 make[5]: *** [host_ListCSSProperties.o] Error 1
10:21.75 make[4]: *** [layout/style/test/host] Error 2
10:21.75 make[4]: *** Waiting for unfinished jobs....

Unless HOST_CC=clang and HOST_CXX=clang++ are set in .mozconfig.

Comment 1

[Randall Barker [:rbarker]]

Attachment: config.log without HOST_CC=clang and HOST_CXX=clang++ set

Comment 2

[Randall Barker [:rbarker]]

Attachment: config.log with patch applied.

Comment 3

[Nathan Froyd [:froydnj]]

Somebody else in #build ran into this, which reminded me that I sort of have an idea how to fix this, but I'm unsure of how to pull it off.

The core problem is, from toolchain.configure:

    # Derive the host compiler from the corresponding target compiler when no
    # explicit compiler was given and we're not cross compiling. For the C++
    # compiler, though, prefer to derive from the host C compiler when it
    # doesn't match the target C compiler.
    # As a special case, since clang supports all kinds of targets in the same
    # executable, when cross compiling with clang, default to the same compiler
    # as the target compiler, resetting flags.
    if host_or_target == host:
        if other_c_compiler is not None:
            args = (c_compiler, other_c_compiler)
        else:
            args = ()

        @depends(provided_compiler, other_compiler, cross_compiling, *args)
        def provided_compiler(value, other_compiler, cross_compiling, *args):
            if value:
                return value
            c_compiler, other_c_compiler = args if args else (None, None)
            if not cross_compiling and c_compiler == other_c_compiler:
                return other_compiler
            if cross_compiling and other_compiler.type == 'clang':
                return namespace(**{
                    k: [] if k == 'flags' else v
                    for k, v in other_compiler.__dict__.iteritems()
                })

so what happens is we wind up using the NDK clang as our host compiler.  But the NDK clang doesn't understand all the special rules for finding host headers that the XCode clang has built in.  Therefore, the NDK clang isn't actually usable as a host compiler (without a bunch of other work), and we never actually check whether the host compiler can find headers or not, or we would have discovered this during configure.

It seems to me that we want to disallow this from happening on OS X (which I tried to do, but was not clever enough to insert the correct logic into `provided_compiler` above), or remove the logic for clang entirely, since the logic isn't really reliable in all cases.  glandium, WDYT?

Comment 4

[Mike Hommey [:glandium]]

How about adding a #include check in check_compiler? Like, checking we can include stddef.h, assuming that fails with the NDK clang?

Comment 5

[Nathan Froyd [:froydnj]]

(In reply to Mike Hommey [:glandium] from comment #4)
> How about adding a #include check in check_compiler? Like, checking we can
> include stddef.h, assuming that fails with the NDK clang?

We could easily do that, but then we have to propagate extra_toolchain_flags around much earlier, since the NDK compilers (for instance) can't find their headers without extra flags.  Maybe that's a good thing, though...

Comment 6

[Nathan Froyd [:froydnj]]

Oh, and we also have to propagate in the NDK flags for libc++, too...which is probably also a good thing.

Comment 7

[Nathan Froyd [:froydnj]]

Attachment: part 1 - convert stlport_cppflags to return a list

A list is a better format for these options, and we'll want a list later
when we use the options as compilation flags.

Comment 8

[Nathan Froyd [:froydnj]]

Attachment: part 2 - use extra_toolchain_flags for computing bindgen flags

We compute the same set of data in extra_toolchain_flags as we were
computing in bindgen_cflags_defaults.  We might as well reuse the former
to compute the latter.

Comment 9

[Nathan Froyd [:froydnj]]

Attachment: part 3 - move stlport_cppflags prior to extra_toolchain_flags

extra_toolchain_flags is going to depend on stlport_cppflags
momentarily, so this commit is just for moving code around.  Note that
the diff shows other things moving *after* stlport_cppflags.

Comment 10

[Nathan Froyd [:froydnj]]

Attachment: part 4 - make extra_toolchain_flags depend on stlport_cppflags

extra_toolchain_flags is used for compiling target-specific bits of code
elsewhere in configure.  Very shortly, we'll need to compile bits of
code that depend on the C++ standard library, so we'll want to point the
compiler at the C++ standard library.  Making extra_toolchain_flags
include stlport_cppflags is the best way to do that.

Comment 11

[Nathan Froyd [:froydnj]]

Attachment: part 5 - remove stlport_cppflags dependency from bindgen_cflags_defaults

Now that extra_toolchain_flags includes stlport_cppflags, there's no
reason bindgen_cflags_defaults should depend on them both.  So remove
the stlport_cppflags dependency.  (There's no harm to multiply-including
stlport_cppflags, but not repeating -I options is good practice.)

Comment 12

[Nathan Froyd [:froydnj]]

Attachment: part 6 - compile headers for cross-compilers during configure

Our toolchain detection logic checks whether we can reuse the target
C (resp. C++) compiler for the host compiler.  This is generally only
applicable in the not-cross-compiling case, but we had special logic to
check for clang in the cross-compiling case and accept it, as clang is
able to generate code for multiple architectures from a single compiler
binary.

Our recent switch to clang on Android has exposed a problem in this
logic: we would never check whether the target clang, compiling for the
host, could actually find the host's headers.  This was especially
problematic on OS X hosts, where the host clang contains special logic
to grovel inside the XCode installation to find C++ headers.  The clang
from the NDK, however, was ignorant of the XCode installation.
Therefore, the NDK clang would happily compile code for the host, even
including C headers for the host, but would be hopelessly lost when it
came to compiling C++ headers during the actual build.

In hopes of mitigating this, we now include a check for a representative
header for C and C++ when checking compilers for each of those
languages.  This check will detect such problems as the above, and will
also alert people to potentially misconfigured compilers in other
situations.

I am not totally convinced by this patch, even though I wrote it: I think it's
still possible to get yourself into situations where the target clang compiler
"works" for the host...right up until it doesn't.  But hopefully this will
solve the original reported problem?

Comment 13

[Nathan Froyd [:froydnj]]

Randall, are you up for trying this patch stack with a mozconfig that doesn't include HOST_CC/HOST_CXX?

Comment 14

[(not currently active) Ted Mielczarek]

Comment on attachment 8927480 [details] [diff] [review]
part 1 - convert stlport_cppflags to return a list

Review of attachment 8927480 [details] [diff] [review]:
-----------------------------------------------------------------

::: build/moz.configure/android-ndk.configure
@@ +253,5 @@
>      # etc.
> +    return [
> +        '-I%s' % cxx_include,
> +        '-I%s' % os.path.join(ndk, 'sources', 'android', 'support', 'include'),
> +        '-I%s' % cxxabi_include]

I've thought at times in the past that it would be nice to have richer data types for configure options similar to what we have in moz.build. Like how with `LOCAL_INCLUDES` we treat them as paths, it would be nice to have that here as well. (Just some idle thoughts.)

Comment 15

[(not currently active) Ted Mielczarek]

Comment on attachment 8927481 [details] [diff] [review]
part 2 - use extra_toolchain_flags for computing bindgen flags

Review of attachment 8927481 [details] [diff] [review]:
-----------------------------------------------------------------

::: build/moz.configure/android-ndk.configure
@@ +278,2 @@
>      return cflags_format % (' '.join(stlport_cppflags),
> +                            ' '.join(toolchain_flags),

Is there any reason this doesn't just want to be something like:
' '.join(stlport_cppflags + toolchain_flags + ['-I' + ... ])

It just seems weird to be joining a list of flags only to format it into what's effectively a list of flags.

Comment 16

[(not currently active) Ted Mielczarek]

Comment on attachment 8927483 [details] [diff] [review]
part 4 - make extra_toolchain_flags depend on stlport_cppflags

Review of attachment 8927483 [details] [diff] [review]:
-----------------------------------------------------------------

::: build/moz.configure/android-ndk.configure
@@ +248,5 @@
> +    flags = ['-isystem',
> +             os.path.join(platform_dir, 'usr', 'include'),
> +             '-gcc-toolchain',
> +             toolchain_dir]
> +    flags.extend(stlport_cppflags if stlport_cppflags else [])

Python's ternary operator still leaves a lot to be desired. :-/ This might be simpler to read with a straight conditional instead.

Comment 17

[(not currently active) Ted Mielczarek]

Comment on attachment 8927485 [details] [diff] [review]
part 6 - compile headers for cross-compilers during configure

Review of attachment 8927485 [details] [diff] [review]:
-----------------------------------------------------------------

I can definitely imagine scenarios where this still fails to do the right thing, but having *some* minimal checks that the compiler we choose will work for the intended purpose seems better than nothing.

I guess ideally for known targets we'd be able to check that the compiler can find headers from the OS SDK, or some other sort of very platform-specific include, which would tighten this up more. We can't make every weirdo configuration do the right thing without any work on the part of the person doing the build, so as long as we're covering the common cases that seems sufficient.

::: build/moz.configure/toolchain.configure
@@ +829,5 @@
>              wrapper.extend(provided_wrapper)
>              flags = provided_compiler.flags
>          else:
>              flags = []
> +        if host_or_target_str == 'target' and extra_flags:

I wonder if it'd be worthwhile to name `extra_toolchain_flags` something more verbose like `extra_target_toolchain_flags`? Not sure it's worthwhile.

Comment 18

[Nathan Froyd [:froydnj]]

Ugh, try has showed me that these patches completely fail tests, because of course the tests don't have headers wired up.

I'm not sure of the best way to handle this.  The tests delegate to mozbuild.preprocessor to handle preprocessing files.  Maybe we can wire that up to just ignore includes for the purposes of tests?

Comment 19

[Nathan Froyd [:froydnj]]

Or perhaps modify the VFS to return fake data for the includes that the preprocessor is trying to find?

Comment 20

[Nathan Froyd [:froydnj]]

Attachment: part 6a - pacify tests now that we're trying to include headers

Testing revealed that the configure tests are genuinely unhappy about
being asked to #include headers.  We address this by modifying the
preprocessor used for testing to handle the initial file and to check
for the headers being used.  Everything else is rejected.  This
modification is sufficient for the tests to pass and also should ensure
that if we start changing configure in peculiar ways, folks will need to
modify the tests as well.

If this is approved, I'll fold this into part 6.

Comment 21

[Nathan Froyd [:froydnj]]

Comment on attachment 8927836 [details] [diff] [review]
part 6a - pacify tests now that we're trying to include headers

Ugh, no, this still has problems.

Comment 22

[Nathan Froyd [:froydnj]]

And, as extra fun, this apparently breaks Android builds because it reorganizes command-line options (?!):

/builds/worker/workspace/build/src/sccache2/sccache /builds/worker/workspace/build/src/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ -isystem /builds/worker/workspace/build/src/android-ndk/platforms/android-9/arch-arm/usr/include -gcc-toolchain /builds/worker/workspace/build/src/android-ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64 -I/builds/worker/workspace/build/src/android-ndk/sources/cxx-stl/llvm-libc++/include -I/builds/worker/workspace/build/src/android-ndk/sources/android/support/include -I/builds/worker/workspace/build/src/android-ndk/sources/cxx-stl/llvm-libc++abi/include -std=gnu++11 --target=arm-linux-androideabi -o pkixbuild.o -c -I/builds/worker/workspace/build/src/obj-firefox/dist/stl_wrappers -I/builds/worker/workspace/build/src/obj-firefox/dist/system_wrappers -include /builds/worker/workspace/build/src/config/gcc_hidden.h -DNDEBUG=1 -DTRIMMED=1 -DSTATIC_EXPORTABLE_JS_API -DMOZ_HAS_MOZGLUE -DMOZILLA_INTERNAL_API -DIMPL_LIBXUL -I/builds/worker/workspace/build/src/security/pkix -I/builds/worker/workspace/build/src/obj-firefox/security/pkix -I/builds/worker/workspace/build/src/security/pkix/include -I/builds/worker/workspace/build/src/obj-firefox/dist/include -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nspr -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nss -fPIC -DMOZILLA_CLIENT -include /builds/worker/workspace/build/src/obj-firefox/mozilla-config.h -Qunused-arguments -isystem /builds/worker/workspace/build/src/android-ndk/platforms/android-9/arch-arm/usr/include -gcc-toolchain /builds/worker/workspace/build/src/android-ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64 -Qunused-arguments -Wall -Wc++11-compat -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wsign-compare -Wtype-limits -Wunreachable-code -Wunreachable-code-return -Wwrite-strings -Wno-invalid-offsetof -Wclass-varargs -Wloop-analysis -Wc++11-compat-pedantic -Wc++14-compat -Wc++14-compat-pedantic -Wc++1z-compat -Wcomma -Wimplicit-fallthrough -Werror=non-literal-null-conversion -Wstring-conversion -Wno-inline-new-delete -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wformat -Wno-gnu-zero-variadic-macro-arguments -Wformat-security -Wno-unknown-warning-option -Wno-return-type-c-linkage -fno-short-enums -fno-exceptions -I/builds/worker/workspace/build/src/android-ndk/sources/cxx-stl/llvm-libc++/include -I/builds/worker/workspace/build/src/android-ndk/sources/android/support/include -I/builds/worker/workspace/build/src/android-ndk/sources/cxx-stl/llvm-libc++abi/include -march=armv7-a -mthumb -mfpu=vfp -mfloat-abi=softfp -mno-unaligned-access -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -D_GLIBCXX_USE_CXX11_ABI=0 -pipe -g -Oz -funwind-tables -Werror -Weverything -Wno-c++98-compat -Wno-c++98-compat-pedantic -Wno-missing-prototypes -Wno-missing-variable-declarations -Wno-padded -Wno-reserved-id-macro -Wno-shadow -Wno-weak-vtables -Wno-error=shadow  -MD -MP -MF .deps/pkixbuild.o.pp   /builds/worker/workspace/build/src/security/pkix/lib/pkixbuild.cpp
In file included from /builds/worker/workspace/build/src/security/pkix/lib/pkixbuild.cpp:25:
In file included from /builds/worker/workspace/build/src/security/pkix/include/pkix/pkix.h:28:
In file included from /builds/worker/workspace/build/src/security/pkix/include/pkix/pkixtypes.h:29:
In file included from /builds/worker/workspace/build/src/security/pkix/include/pkix/Time.h:28:
In file included from /builds/worker/workspace/build/src/android-ndk/sources/cxx-stl/llvm-libc++/include/ctime:48:
/builds/worker/workspace/build/src/android-ndk/sources/android/support/include/time.h:33:2: error: #include_next is a language extension [-Werror,-Wgnu-include-next]
#include_next <time.h>
 ^
1 error generated.
/builds/worker/workspace/build/src/config/rules.mk:1028: recipe for target 'pkixbuild.o' failed

Comment 23

[Nathan Froyd [:froydnj]]

Attachment: part 6a - pacify tests now that we're trying to include headers

Testing revealed that the configure tests are genuinely unhappy about
being asked to #include headers.  We address this by modifying the
preprocessor used for testing to handle the initial file and to check
for the headers being used.  Everything else is rejected.  This
modification is sufficient for the tests to pass and also should ensure
that if we start changing configure in peculiar ways, folks will need to
modify the tests as well.

Comment 24

[Nathan Froyd [:froydnj]]

Attachment: part 6b - only use extra_toolchain_flags for compiling

We needed to pass extra_toolchain_flags into invocations of the target
compiler so target headers could be found.  Our method for doing so
produced the unexpected effect of including extra_toolchain_flags as
part of the CC/CXX value derived by configure.  That is, instead of:

  clang -std=gnu++11 --target=...

we derived:

  clang -isystem ... -I... [more -I flags] -std=gnu+11 --target=...

For reasons not understood, this second formulation causes clang to
start issuing errors about various libc++ headers.

As we already include the appropriate flags into the command line (see
build/autoconf/android.m4) in other ways, we can just use
extra_toolchain_flags solely for their effects on compilation and not as
part of CC/CXX.  It would be desirable to add extra_toolchain_flags in
just one place, configure, but that will have to wait for a separate
bug.

Comment 25

[(not currently active) Ted Mielczarek]

Comment on attachment 8927924 [details] [diff] [review]
part 6a - pacify tests now that we're trying to include headers

Review of attachment 8927924 [details] [diff] [review]:
-----------------------------------------------------------------

This seems workable, and probably leaves enough breadcrumbs for future developers to find where to add a fix if necessary.

::: python/mozbuild/mozbuild/test/configure/test_toolchain_helpers.py
@@ +66,5 @@
> +        # user doesn't have a misconfigured environment.  But for the purposes
> +        # of tests, we want to ignore headers.
> +
> +        # stddef.h and cstddef are the headers compiler configure checks use.
> +        if type(args) in (str, unicode) and '<' in args:

You probably want to write `isinstance(args, basestring)` here.

Comment 26

[Randall Barker [:rbarker]]

After applying the patches, I get:

 0:05.68 checking whether the target C++ compiler can be used...
 0:05.68 DEBUG: <truncated - see config.log for full output>
 0:05.68 DEBUG: | # 1 "<built-in>" 3
 0:05.68 DEBUG: | # 325 "<built-in>" 3
 0:05.68 DEBUG: | # 1 "<command line>" 1
 0:05.68 DEBUG: | # 1 "<built-in>" 2
 0:05.68 DEBUG: | # 1 "/var/folders/d0/fwhpqq711mb_t2dlxnvv08t00000gn/T/conftest.51a_aE.cpp" 2
 0:05.68 DEBUG: | # 10 "/var/folders/d0/fwhpqq711mb_t2dlxnvv08t00000gn/T/conftest.51a_aE.cpp"
 0:05.68 DEBUG: | %COMPILER "clang"
 0:05.68 DEBUG: | # 20 "/var/folders/d0/fwhpqq711mb_t2dlxnvv08t00000gn/T/conftest.51a_aE.cpp"
 0:05.68 DEBUG: | %cplusplus 199711L
 0:05.68 DEBUG: | # 29 "/var/folders/d0/fwhpqq711mb_t2dlxnvv08t00000gn/T/conftest.51a_aE.cpp"
 0:05.68 DEBUG: | %CPU "x86_64"
 0:05.68 DEBUG: | # 70 "/var/folders/d0/fwhpqq711mb_t2dlxnvv08t00000gn/T/conftest.51a_aE.cpp"
 0:05.68 DEBUG: | %KERNEL "Linux"
 0:05.68 DEBUG: | # 79 "/var/folders/d0/fwhpqq711mb_t2dlxnvv08t00000gn/T/conftest.51a_aE.cpp"
 0:05.68 DEBUG: | %ENDIANNESS "little"
 0:05.68 DEBUG: Its error output was:
 0:05.68 DEBUG: | /var/folders/d0/fwhpqq711mb_t2dlxnvv08t00000gn/T/conftest.51a_aE.cpp:83:10: fatal error: 'cstddef' file not found
 0:05.68 DEBUG: | #include <cstddef>
 0:05.68 DEBUG: |          ^
 0:05.68 DEBUG: | 1 error generated.
 0:05.68 ERROR: Command `/Users/rbarker/.mozbuild/android-ndk-r12b/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++ -isystem /Users/rbarker/.mozbuild/android-ndk-r12b/platforms/android-9/arch-arm/usr/include -gcc-toolchain /Users/rbarker/.mozbuild/android-ndk-r12b/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64 -E /var/folders/d0/fwhpqq711mb_t2dlxnvv08t00000gn/T/conftest.51a_aE.cpp` failed with exit status 1.
 0:05.71 *** Fix above errors and then restart with               "/Applications/Xcode.app/Contents/Developer/usr/bin/make -f client.mk build"
 0:05.71 make: *** [configure] Error 1

Comment 27

[Randall Barker [:rbarker]]

Even though I was using the wrong NDK, when I updated to the correct version, I still basically the same error:

 0:05.51 checking whether the target C++ compiler can be used...
 0:05.51 DEBUG: <truncated - see config.log for full output>
 0:05.51 DEBUG: | # 1 "<command line>" 1
 0:05.51 DEBUG: | # 1 "<built-in>" 2
 0:05.51 DEBUG: | # 1 "/var/folders/d0/fwhpqq711mb_t2dlxnvv08t00000gn/T/conftest.lS9ZJm.cpp" 2
 0:05.51 DEBUG: | # 10 "/var/folders/d0/fwhpqq711mb_t2dlxnvv08t00000gn/T/conftest.lS9ZJm.cpp"
 0:05.51 DEBUG: | %COMPILER "clang"
 0:05.51 DEBUG: | # 20 "/var/folders/d0/fwhpqq711mb_t2dlxnvv08t00000gn/T/conftest.lS9ZJm.cpp"
 0:05.51 DEBUG: | %cplusplus 199711L
 0:05.51 DEBUG: | # 29 "/var/folders/d0/fwhpqq711mb_t2dlxnvv08t00000gn/T/conftest.lS9ZJm.cpp"
 0:05.51 DEBUG: | %CPU "x86_64"
 0:05.51 DEBUG: | # 76 "/var/folders/d0/fwhpqq711mb_t2dlxnvv08t00000gn/T/conftest.lS9ZJm.cpp"
 0:05.51 DEBUG: | %KERNEL "Darwin"
 0:05.51 DEBUG: |
 0:05.51 DEBUG: |
 0:05.51 DEBUG: | %ENDIANNESS "little"
 0:05.51 DEBUG: Its error output was:
 0:05.51 DEBUG: | clang++: warning: argument unused during compilation: '--gcc-toolchain=/Users/rbarker/.mozbuild/android-ndk-r15c/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64' [-Wunused-command-line-argument]
 0:05.51 DEBUG: | /var/folders/d0/fwhpqq711mb_t2dlxnvv08t00000gn/T/conftest.lS9ZJm.cpp:83:10: fatal error: 'cstddef' file not found
 0:05.51 DEBUG: | #include <cstddef>
 0:05.51 DEBUG: |          ^~~~~~~~~
 0:05.51 DEBUG: | 1 error generated.
 0:05.51 ERROR: Command `/Users/rbarker/.mozbuild/android-ndk-r15c/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++ -isystem /Users/rbarker/.mozbuild/android-ndk-r15c/platforms/android-9/arch-arm/usr/include -gcc-toolchain /Users/rbarker/.mozbuild/android-ndk-r15c/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64 -E /var/folders/d0/fwhpqq711mb_t2dlxnvv08t00000gn/T/conftest.lS9ZJm.cpp` failed with exit status 1.
 0:05.55 *** Fix above errors and then restart with               "/Applications/Xcode.app/Contents/Developer/usr/bin/make -f client.mk build"
 0:05.55 make: *** [configure] Error 1

Comment 28

[Randall Barker [:rbarker]]

Attachment: Config log with patches applied

Comment 29

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/90e7ff50c118
part 1 - convert stlport_cppflags to return a list; r=ted.mielczarek
https://hg.mozilla.org/integration/mozilla-inbound/rev/0fb376428333
part 2 - use extra_toolchain_flags for computing bindgen flags; r=ted.mielczarek
https://hg.mozilla.org/integration/mozilla-inbound/rev/776266640eca
part 3 - move stlport_cppflags prior to extra_toolchain_flags; r=ted.mielczarek
https://hg.mozilla.org/integration/mozilla-inbound/rev/9615a90cf9b0
part 4 - make extra_toolchain_flags depend on stlport_cppflags; r=ted.mielczarek
https://hg.mozilla.org/integration/mozilla-inbound/rev/0e6f349c815e
part 5 - remove stlport_cppflags dependency from bindgen_cflags_defaults; r=ted.mielczarek
https://hg.mozilla.org/integration/mozilla-inbound/rev/b0eedb4c140b
part 6 - compile headers for cross-compilers during configure; r=ted.mielczarek

Comment 30

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/90e7ff50c118
https://hg.mozilla.org/mozilla-central/rev/0fb376428333
https://hg.mozilla.org/mozilla-central/rev/776266640eca
https://hg.mozilla.org/mozilla-central/rev/9615a90cf9b0
https://hg.mozilla.org/mozilla-central/rev/0e6f349c815e
https://hg.mozilla.org/mozilla-central/rev/b0eedb4c140b

Comment 31

[James Cheng[:JamesCheng]]

I still get this error even rebase to central with these patches.

Did I need to add something to my mozconfig?

Comment 32

[Nathan Froyd [:froydnj]]

Part 6 backed out in:

https://hg.mozilla.org/mozilla-central/rev/d49e40cb14e5de521fdade4a872d0fa804d7d400

for causing bug 1417745.  Not sure if we should just skip those includes if _MSC_VER or something more sophisticated.

(In reply to James Cheng[:JamesCheng] from comment #31)
> I still get this error even rebase to central with these patches.
> 
> Did I need to add something to my mozconfig?

You should not.  Can you attach the failing log here, please?

Comment 33

[(not currently active) Ted Mielczarek]

I would say just skip these checks for now when `info.type == 'msvc'`. You could file a followup to make that work, but it seems less likely that someone would have a misconfigured MSVC given all the moz.configure machinery we have around it.

Comment 34

[Mike Hommey [:glandium]]

Plus, the problem this fixes is inherent to clang. It's not something that will happen with msvc. You might want to exclude clang-cl too.

Comment 35

[James Cheng[:JamesCheng]]

Attach the full log, which is the same as comment 27

./mach build
 0:00.53 Clobber not needed.
 0:00.57 /usr/bin/make -f client.mk -s
 0:01.49 Adding client.mk options from /Users/jacheng/Projects/gecko-fennec/mozconfig:
 0:01.49     CONFIG_GUESS=arm-linux-androideabi
 0:01.49     AUTOCLOBBER=1
 0:01.49     MOZ_OBJDIR=/Users/jacheng/Projects/gecko-fennec/obj-arm-linux-androideabi
 0:01.49     OBJDIR=/Users/jacheng/Projects/gecko-fennec/obj-arm-linux-androideabi
 0:01.49     FOUND_MOZCONFIG=/Users/jacheng/Projects/gecko-fennec/mozconfig
 0:01.49     export FOUND_MOZCONFIG
 0:01.61 cd /Users/jacheng/Projects/gecko-fennec/obj-arm-linux-androideabi
 0:01.61 /Users/jacheng/Projects/gecko-fennec/configure
 0:01.84 Reexecuting in the virtualenv
 0:02.09 Adding configure options from /Users/jacheng/Projects/gecko-fennec/mozconfig
 0:02.09   --enable-application=mobile/android
 0:02.09   --target=arm-linux-androideabi
 0:02.09   --with-android-sdk=/Users/jacheng/.mozbuild/android-sdk-macosx
 0:02.09   --with-android-ndk=/Users/jacheng/.mozbuild/android-ndk-r15c
 0:02.09   --enable-debug
 0:02.09 checking for vcs source checkout... git
 0:02.15 checking for a shell... /bin/sh
 0:02.79 checking for host system type... x86_64-apple-darwin16.7.0
 0:02.81 checking for target system type... arm-unknown-linux-androideabi
 0:02.81 checking for a shell... /bin/sh
 0:02.94 checking for host system type... x86_64-apple-darwin16.7.0
 0:02.96 checking for target system type... arm-unknown-linux-androideabi
 0:02.96 checking for android platform directory... /Users/jacheng/.mozbuild/android-ndk-r15c/platforms/android-9/arch-arm
 0:02.97 checking for the Android toolchain directory... /Users/jacheng/.mozbuild/android-ndk-r15c/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64
 0:03.01 checking for android platform directory... /Users/jacheng/.mozbuild/android-ndk-r15c/platforms/android-9/arch-arm
 0:03.01 checking for the Android toolchain directory... /Users/jacheng/.mozbuild/android-ndk-r15c/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64
 0:03.31 checking for vcs source checkout... git
 0:03.32 checking whether cross compiling... yes
 0:03.32 checking for the target C compiler... /Users/jacheng/.mozbuild/android-ndk-r15c/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang
 0:03.46 checking whether the target C compiler can be used... yes
 0:03.48 checking for Python 3... no
 0:03.48 checking for git... /usr/local/bin/git
 0:03.48 checking for Git version... 2.13.1
 0:03.48 checking for sparse checkout... no
 0:03.49 checking for pkg_config... not found
 0:03.49 checking for yasm... /usr/local/bin/yasm
 0:03.54 checking yasm version... 1.3.0
 0:03.54 checking for android ndk version... 15.2.4203891
 0:03.54 checking for GoogleVR SDK... Not specified
 0:03.67 checking the target C compiler version... 5.0.300080
 0:03.76 checking the target C compiler works... yes
 0:03.76 checking for the target C++ compiler... /Users/jacheng/.mozbuild/android-ndk-r15c/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++
 0:03.91 checking whether the target C++ compiler can be used... yes
 0:03.91 checking the target C++ compiler version... 5.0.300080
 0:03.97 checking the target C++ compiler works... yes
 0:03.97 checking for the host C compiler... /Users/jacheng/.mozbuild/android-ndk-r15c/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang
 0:04.05 checking whether the host C compiler can be used... yes
 0:04.06 checking the host C compiler version... 5.0.300080
 0:04.10 checking the host C compiler works... yes
 0:04.10 checking for the host C++ compiler... /Users/jacheng/.mozbuild/android-ndk-r15c/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++
 0:04.15 checking whether the host C++ compiler can be used...
 0:04.15 DEBUG: <truncated - see config.log for full output>
 0:04.15 DEBUG: | # 340 "<built-in>" 3
 0:04.15 DEBUG: | # 1 "<command line>" 1
 0:04.15 DEBUG: | # 1 "<built-in>" 2
 0:04.15 DEBUG: | # 1 "/var/folders/5w/5_8f3n250q144lz5w3bw663w0000gn/T/conftest.sv6fbF.cpp" 2
 0:04.15 DEBUG: | # 10 "/var/folders/5w/5_8f3n250q144lz5w3bw663w0000gn/T/conftest.sv6fbF.cpp"
 0:04.15 DEBUG: | %COMPILER "clang"
 0:04.15 DEBUG: | # 20 "/var/folders/5w/5_8f3n250q144lz5w3bw663w0000gn/T/conftest.sv6fbF.cpp"
 0:04.15 DEBUG: | %cplusplus 199711L
 0:04.16 DEBUG: | # 29 "/var/folders/5w/5_8f3n250q144lz5w3bw663w0000gn/T/conftest.sv6fbF.cpp"
 0:04.16 DEBUG: | %CPU "x86_64"
 0:04.16 DEBUG: | # 76 "/var/folders/5w/5_8f3n250q144lz5w3bw663w0000gn/T/conftest.sv6fbF.cpp"
 0:04.16 DEBUG: | %KERNEL "Darwin"
 0:04.16 DEBUG: |
 0:04.16 DEBUG: |
 0:04.16 DEBUG: | %ENDIANNESS "little"
 0:04.16 DEBUG: Its error output was:
 0:04.16 DEBUG: | /var/folders/5w/5_8f3n250q144lz5w3bw663w0000gn/T/conftest.sv6fbF.cpp:83:10: fatal error: 'cstddef' file not found
 0:04.16 DEBUG: | #include <cstddef>
 0:04.16 DEBUG: |          ^~~~~~~~~
 0:04.16 DEBUG: | 1 error generated.
 0:04.16 ERROR: Command `/Users/jacheng/.mozbuild/android-ndk-r15c/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++ -E /var/folders/5w/5_8f3n250q144lz5w3bw663w0000gn/T/conftest.sv6fbF.cpp` failed with exit status 1.
 0:04.22 *** Fix above errors and then restart with               "/Applications/Xcode.app/Contents/Developer/usr/bin/make -f client.mk build"
 0:04.22 make[1]: *** [configure] Error 1
 0:04.22 make: *** [/Users/jacheng/Projects/gecko-fennec/obj-arm-linux-androideabi/config.status] Error 2

Comment 36

[Nick Alexander :nalexander [he/him]]

I get the same bustage after `hg graft -f -r b0eedb4c140b`, so this is still broken.

Comment 37

[Zibi Braniecki [:zbraniecki][:gandalf]]

hit this in today's m-c. Can we elevate the priority? It prevents me from debugging Android completely.

Comment 38

[Nathan Froyd [:froydnj]]

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #37)
> hit this in today's m-c. Can we elevate the priority? It prevents me from
> debugging Android completely.

You can set:

HOST_CC=clang
HOST_CXX=clang++

in your .mozconfig to work around this issue.  If that doesn't fix things for you, well, that'd be a separate bug, then. :)

Comment 39

[BugBot [:suhaib / :marco/ :calixte]]

The bug assignee didn't login in Bugzilla in the last 7 months.
:nalexander, could you have a look please?
For more information, please visit auto_nag documentation.

Comment 40

[Nick Alexander :nalexander [he/him]]

This has long since been addressed.