1726783 - MACOS_SDK_DIR env var + system clang breaks due to the difference between --sysroot and -isysroot

Status: RESOLVED FIXED

(Firefox Build System :: General, defect, P5)

Description

[Tooru Fujisawa [:arai]]

NOTE: I'm using 10.13 SDK as an example. The underlying issue isn't specific to SDK version.

Steps to reproduce:

1. retrieve MacOSX10.13.sdk from old Xcode
2. create mozconfig with

ac_add_options --enable-project=js
ac_add_options --enable-nspr-build
export CC=/usr/bin/clang
export CXX=/usr/bin/clang++
export MACOS_SDK_DIR=/path/to/MacOSX10.13.sdk

3. Run ./mach build

Expected result:
Successful build

Actual result:
Build failure with

 3:54.04 config/external/nspr/pr/libnspr4.dylib
 3:54.33 Undefined symbols for architecture x86_64:
 3:54.33   "___darwin_check_fd_set_overflow", referenced from:
 3:54.34       _poll in Unified_c_external_nspr_pr0.o
 3:54.34       __PR_getset in Unified_c_external_nspr_pr2.o
 3:54.34       __PR_setset in Unified_c_external_nspr_pr2.o
 3:54.34 ld: symbol(s) not found for architecture x86_64
 3:54.34 clang: error: linker command failed with exit code 1 (use -v to see invocation)

Looks like --sysroot used by bug 1724374 isn't equivalent to -isysroot, in term of include path, on system clang.

Minimal testcase:

$ clang++ --version
Apple clang version 12.0.0 (clang-1200.0.32.29)
Target: x86_64-apple-darwin20.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
$ cat a.cc
#include <sys/select.h>
int main(void) {
  fd_set set; int fd = 0;
  FD_ZERO(&set); FD_SET(fd, &set);
  return 0;
}
$ clang++ -isysroot /path/to/MacOSX10.13.sdk a.cc # works
$ clang++ --sysroot /path/to/MacOSX10.13.sdk a.cc
Undefined symbols for architecture x86_64:
  "___darwin_check_fd_set_overflow", referenced from:
      _main in a-849318.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
$ ~/.mozbuild/clang/bin/clang++ --sysroot /path/to/MacOSX10.13.sdk a.cc # works

Some observation:

clang++ --save-temps -isysroot /path/to/MacOSX10.13.sdk a.cc generates a.ii with following lines:

# 1 "/path/to/MacOSX10.13.sdk/usr/include/sys/select.h" 1 3 4
...
int main(void) {
  fd_set set; int fd = 0;
  __builtin_bzero(&set, sizeof(*(&set))); do { int __fd = (fd); ((&set)->fds_bits[(unsigned long)__fd/(sizeof(__int32_t) * 8)] |= ((__int32_t)(((unsigned long)1)<<((unsigned long)__fd % (sizeof(__int32_t) * 8))))); } while(0);
  return 0;
}

but clang++ --save-temps --sysroot /path/to/MacOSX10.13.sdk a.cc generates a.ii with following lines:

# 1 "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/sys/select.h" 1 3 4
...
int main(void) {
  fd_set set; int fd = 0;
  __builtin_bzero(&set, sizeof(*(&set))); __darwin_fd_set((fd), (&set));
  return 0;
}

that means the SDK path isn't used for include path, while the library path is used, and it results in inconsistency between the header and library.

Note:

I accidentally hit this during using autospider.py locally, with plain variant, that sets CC/CXX variables to gcc/g++, that is actually a wrapper for clang on macOS, and I just need to override the variant file to avoid this.
So this issue is very edge case, and maybe WONTFIX is fine.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1724374

Comment 2

[Marco Castelluccio [:marco]]

Set release status flags based on info from the regressing bug 1724374

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

The bug has a release status flag that shows some version of Firefox is affected, thus it will be considered confirmed.

Comment 4

[Mike Hommey [:glandium]]

Fixed by bug 1792666.