1509696 - Fail to build on macOS Mojave when building baldrdash with /usr/local/Cellar/llvm/7.0.0/lib/clang/7.0.0/include/inttypes.h:30:15: fatal error: 'inttypes.h' file not found

Status: RESOLVED DUPLICATE of bug 1487552

(Firefox Build System :: General, defect)

Description

[Xidorn Quan [:xidorn] UTC+11]

0:05.42 error: failed to run custom build command for `baldrdash v0.1.0 (/Users/upsuper/Projects/mozilla-central/js/src/wasm/cranelift)`
 0:05.42 process didn't exit successfully: `/Users/upsuper/Projects/mozilla-central/obj-firefox.noindex/debug/build/baldrdash-449bb6c4bda68ec6/build-script-build` (exit code: 101)
 0:05.42 --- stdout
 0:05.42 cargo:rerun-if-changed=baldrapi.h
 0:05.42 cargo:rerun-if-changed=/Users/upsuper/Projects/mozilla-central/obj-firefox.noindex/js/src/rust/extra-bindgen-flags
 0:05.42 --- stderr
 0:05.42 /usr/local/Cellar/llvm/7.0.0/lib/clang/7.0.0/include/inttypes.h:30:15: fatal error: 'inttypes.h' file not found
 0:05.42 /usr/local/Cellar/llvm/7.0.0/lib/clang/7.0.0/include/inttypes.h:30:15: fatal error: 'inttypes.h' file not found, err: true
 0:05.42 thread 'main' panicked at 'Unable to generate baldrapi.h bindings: ()', libcore/result.rs:1009:5
 0:05.42 stack backtrace:
 0:05.42    0:        0x10393f92f - std::sys::unix::backtrace::tracing::imp::unwind_backtrace::ha680c7623902b8b1
 0:05.42    1:        0x10391a0bd - std::sys_common::backtrace::print::h1c66367b411ee2b0
 0:05.43    2:        0x1039444c3 - std::panicking::default_hook::{{closure}}::h4f17ee1c25ab7ece
 0:05.43    3:        0x10394424c - std::panicking::default_hook::hc0a1fadd9830e5b5
 0:05.43    4:        0x103944bf7 - std::panicking::rust_panic_with_hook::h208eb16bfa9d451d
 0:05.43    5:        0x10394475c - std::panicking::continue_panic_fmt::h9b29adfa418c2370
 0:05.43    6:        0x103944648 - rust_begin_unwind
 0:05.43    7:        0x1039abcc1 - core::panicking::panic_fmt::hdbcabc9d57f77848
 0:05.43    8:        0x10220caa7 - core::result::unwrap_failed::h7157a77e49653f1f
 0:05.43                                at libcore/macros.rs:26
 0:05.43    9:        0x10220d804 - <core::result::Result<T, E>>::expect::h5dfb2828ee526f89
 0:05.43                                at libcore/result.rs:835
 0:05.43   10:        0x10220f265 - build_script_build::main::h5304f5c7f23eca6f
 0:05.43                                at js/src/wasm/cranelift/build.rs:73
 0:05.43   11:        0x10220f475 - std::rt::lang_start::{{closure}}::h39596f2df70fa2dc
 0:05.43                                at libstd/rt.rs:74
 0:05.43   12:        0x1039445c7 - std::panicking::try::do_call::h682cd0c950e47076
 0:05.43   13:        0x10395518e - __rust_maybe_catch_panic
 0:05.43   14:        0x1039312cc - std::rt::lang_start_internal::heabafbf362cc3e1e
 0:05.43   15:        0x10220f467 - std::rt::lang_start::h886e807433a54997
 0:05.43                                at libstd/rt.rs:74

Comment 1

[Xidorn Quan [:xidorn] UTC+11]

I think it's actually a more general issue of the build system since I'm also seeing style crate fails for roughly the same issue (although in that case it's stdio.h instead).

I suspect something going wrong with clang-sys on Mojave.

Comment 2

[Xidorn Quan [:xidorn] UTC+11]

OK I think the problem is that llvm installed from Homebrew fails to figure out the right set of include directories.

If I run `/usr/local/opt/llvm/bin/clang -E -x c++ - -v` (which is the command clang-sys runs underneath), I got:
> clang version 7.0.0 (tags/RELEASE_700/final)
> Target: x86_64-apple-darwin18.2.0
> Thread model: posix
> InstalledDir: /usr/local/opt/llvm/bin
>  "/usr/local/Cellar/llvm/7.0.0_1/bin/clang-7" -cc1 -triple x86_64-apple-macosx10.14.0 -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name - -mrelocation-model pic -pic-level 2 -mthread-model posix -mdisable-fp-elim -masm-verbose -munwind-tables -target-cpu penryn -dwarf-column-info -debugger-tuning=lldb -target-linker-version 409.12 -v -resource-dir /usr/local/Cellar/llvm/7.0.0_1/lib/clang/7.0.0 -stdlib=libc++ -fdeprecated-macro -fdebug-compilation-dir /Users/upsuper/Projects/mozilla-central -ferror-limit 19 -fmessage-length 150 -stack-protector 1 -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fobjc-runtime=macosx-10.14.0 -fcxx-exceptions -fexceptions -fmax-type-align=16 -fdiagnostics-show-option -fcolor-diagnostics -o - -x c++ -
> clang -cc1 version 7.0.0 based upon LLVM 7.0.0 default target x86_64-apple-darwin18.2.0
> ignoring nonexistent directory "/usr/include/c++/v1"
> ignoring nonexistent directory "/usr/include"
> #include "..." search starts here:
> #include <...> search starts here:
>  /usr/local/Cellar/llvm/7.0.0_1/include/c++/v1
>  /usr/local/include
>  /usr/local/Cellar/llvm/7.0.0_1/lib/clang/7.0.0/include
>  /System/Library/Frameworks (framework directory)
>  /Library/Frameworks (framework directory)
> End of search list.
which is apparently wrong! None of the directories listed for `#include <...>` includes the files we want here.

On the contrary, clang provided by Xcode gives me:
> Apple LLVM version 10.0.0 (clang-1000.11.45.5)
> Target: x86_64-apple-darwin18.2.0
> Thread model: posix
> InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
>  "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.14.0 -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name - -mrelocation-model pic -pic-level 2 -mthread-model posix -mdisable-fp-elim -fno-strict-return -masm-verbose -munwind-tables -target-cpu penryn -dwarf-column-info -debugger-tuning=lldb -target-linker-version 409.12 -v -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/10.0.0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk -I/usr/local/include -stdlib=libc++ -fdeprecated-macro -fdebug-compilation-dir /Users/upsuper/Projects/mozilla-central -ferror-limit 19 -fmessage-length 150 -stack-protector 1 -fblocks -fencode-extended-block-signature -fobjc-runtime=macosx-10.14.0 -fcxx-exceptions -fexceptions -fmax-type-align=16 -fdiagnostics-show-option -fcolor-diagnostics -o - -x c++ -
> clang -cc1 version 10.0.0 (clang-1000.11.45.5) default target x86_64-apple-darwin18.2.0
> ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/c++/v1"
> ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/local/include"
> ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/Library/Frameworks"
> #include "..." search starts here:
> #include <...> search starts here:
>  /usr/local/include
>  /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1
>  /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/10.0.0/include
>  /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
>  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include
>  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks (framework directory)
> End of search list.
in which /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include has the right files.

Comment 3

[Xidorn Quan [:xidorn] UTC+11]

The main difference seems to be
> -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
which exists in the output of system clang but not Homebrew clang.

I have no idea how can we fix this...

Comment 4

[Xidorn Quan [:xidorn] UTC+11]

After I apply this:
> diff --git a/build/moz.configure/bindgen.configure b/build/moz.configure/bindgen.configure
> --- a/build/moz.configure/bindgen.configure
> +++ b/build/moz.configure/bindgen.configure
> @@ -273,16 +273,18 @@ def basic_bindgen_cflags(host, target, i
>          'OSX': {
>              'default': [
>                  '-DOS_MACOSX=1',
>                  '-stdlib=libc++',
>                  # To disable the fixup bindgen applies which adds search
>                  # paths from clang command line in order to avoid potential
>                  # conflict with -stdlib=libc++.
>                  '--target=x86_64-apple-darwin',
> +                '-isysroot',
> +                '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk',
>              ],
>          },
>          'SunOS': {
>              'default': ['-DOS_SOLARIS=1'],
>          },
>          'WINNT': {
>              'default': [
>                  '-DOS_WIN=1',

the build seems to be proceeding now... but this apparently is not the proper solution...

Comment 5

[Xidorn Quan [:xidorn] UTC+11]

It's probably a problem of Homebrew LLVM, not something fixable from our side :/

We may need to hand MACOS_SDK_DIR to basic_bindgen_cflags when that's specified, though.

Comment 7

[Brad Werth [:bradwerth]]

I'm experiencing this, too. I'll use the approach in comment 4 until we have a proper fix. Thank you for posting.

Comment 8

[Nathan Froyd [:froydnj]]

There's also bug 1500504, which says that we can't even build with the 10.14 SDK at the moment.

Comment 9

[Xidorn Quan [:xidorn] UTC+11]

Yeah, so I think one change to do for now would be passing MACOS_SDK_DIR to basic_bindgen_cflags.

I had a look at the code, and I believe doing so would require moving the --with-macos-sdk option from old-configure into Python, which is probably more involving than I would like to try at this moment...

Comment 10

[Nathan Froyd [:froydnj]]

(In reply to Xidorn Quan [:xidorn] UTC+11 from comment #9)
> Yeah, so I think one change to do for now would be passing MACOS_SDK_DIR to
> basic_bindgen_cflags.
> 
> I had a look at the code, and I believe doing so would require moving the
> --with-macos-sdk option from old-configure into Python, which is probably
> more involving than I would like to try at this moment...

This should be pretty straightforward.  Ideally I will give this a try this weekend.

Comment 11

[Nathan Froyd [:froydnj]]

(In reply to Nathan Froyd [:froydnj] from comment #10)
> (In reply to Xidorn Quan [:xidorn] UTC+11 from comment #9)
> > Yeah, so I think one change to do for now would be passing MACOS_SDK_DIR to
> > basic_bindgen_cflags.
> > 
> > I had a look at the code, and I believe doing so would require moving the
> > --with-macos-sdk option from old-configure into Python, which is probably
> > more involving than I would like to try at this moment...
> 
> This should be pretty straightforward.  Ideally I will give this a try this
> weekend.

Actually, thinking about this, --with-macos-sdk should only be used for cross builds...so I don't think moving it to moz.configure would have any effect.

Comment 12

[Xidorn Quan [:xidorn] UTC+11]

(In reply to Nathan Froyd [:froydnj] from comment #11)
> Actually, thinking about this, --with-macos-sdk should only be used for
> cross builds...so I don't think moving it to moz.configure would have any
> effect.

Depending on the definition of "cross build", this can happen quite often. Consider the case of bug 1500504, people would want to use --with-macos-sdk to use an old SDK. Given that macOS's new SDK consitently breaks stuff, this can happen repeatedly each year.

Also combining --with-macos-sdk and having bindgen get -isysroot from that should fix this issue.