Closed Bug 1670197 Opened 4 years ago Closed 4 years ago

Errors running mach bootstrap

Categories

(Firefox Build System :: General, defect)

Product:
Firefox Build System
Component:
General
Platform:
Unspecified
macOS
Type:
defect

Tracking

(firefox84 fixed)

Status:
RESOLVED FIXED
Milestone:
84 Branch
Tracking Flags:
Tracking Status
firefox84 --- fixed

People

(Reporter: longsonr, Assigned: mhentges)

References

Details

Attachments

(1 file)

Bug 1670197: "Universal" python binaries shouldn't cause setup.py fails
47 bytes, text/x-phabricator-request
Details | Review
  1. Start without a ~/.mozbuild directory

Note that bootstrap does continue after this point and the build does run.

Please choose the version of Firefox you want to build:

  1. Firefox for Desktop Artifact Mode
  2. Firefox for Desktop
  3. GeckoView/Firefox for Android Artifact Mode
  4. GeckoView/Firefox for Android
    Your choice: 2
    Your version of Python 3 (3.8.2) is new enough.
    Your version of Python 2 (2.7.16) is new enough.

Would you like to create this directory? (Yn): y
Creating global state directory: /<elided>/.mozbuild
obsolete feature not enabled but 22 markers found!
obsolete feature not enabled but 22 markers found!
created virtual environment CPython3.8.2.final.0-64 in 4946ms
creator CPython3macOsFramework(dest=/<elided>/.mozbuild/_virtualenvs/mach, clear=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/<elided>/Library/Application Support/virtualenv)
added seed packages: pip==20.2.2, setuptools==49.6.0, wheel==0.35.1
activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
running build_ext
building 'psutil._psutil_osx' extension
xcrun -sdk macosx clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=570 -DPSUTIL_OSX=1 -I/<elided>/.mozbuild/_virtualenvs/mach/include -I/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -c psutil/_psutil_common.c -o build/temp.macosx-10.14.6-x86_64-3.8/psutil/_psutil_common.o
In file included from psutil/_psutil_common.c:9:
In file included from /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:11:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0/include/limits.h:21:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/limits.h:63:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/cdefs.h:807:2: error: Unsupported architecture
#error Unsupported architecture
^
In file included from psutil/_psutil_common.c:9:
In file included from /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:11:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0/include/limits.h:21:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/limits.h:64:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/machine/limits.h:8:2: error: architecture not supported
#error architecture not supported
^
In file included from psutil/_psutil_common.c:9:
In file included from /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:25:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/stdio.h:64:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/_stdio.h:71:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/_types.h:27:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:33:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/machine/_types.h:34:2: error: architecture not supported
#error architecture not supported
^
In file included from psutil/_psutil_common.c:9:
In file included from /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:25:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/stdio.h:64:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/_stdio.h:71:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/_types.h:27:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:55:9: error: unknown type name '__int64_t'
typedef __int64_t __darwin_blkcnt_t; /* total blocks /
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:56:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t __darwin_blksize_t; / preferred block size /
^
note: '__int128_t' declared here
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:57:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t __darwin_dev_t; / dev_t /
^
note: '__int128_t' declared here
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:60:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t __darwin_gid_t; / [???] process and group IDs /
^
note: '__uint128_t' declared here
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:61:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t __darwin_id_t; / [XSI] pid_t, uid_t, or gid_t*/
^
note: '__uint128_t' declared here
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:62:9: error: unknown type name '__uint64_t'
typedef __uint64_t __darwin_ino64_t; /* [???] Used for 64 bit inodes /
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:68:9: error: unknown type name '__darwin_natural_t'
typedef __darwin_natural_t __darwin_mach_port_name_t; / Used by mach /
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:70:9: error: unknown type name '__uint16_t'; did you mean '__uint128_t'?
typedef __uint16_t __darwin_mode_t; / [???] Some file attributes /
^
note: '__uint128_t' declared here
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:71:9: error: unknown type name '__int64_t'
typedef __int64_t __darwin_off_t; / [???] Used for file sizes /
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:72:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t __darwin_pid_t; / [???] process and group IDs /
^
note: '__int128_t' declared here
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:72:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t __darwin_pid_t; / [???] process and group IDs /
^
note: '__int128_t' declared here
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:73:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t __darwin_sigset_t; / [???] signal set /
^
note: '__uint128_t' declared here
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:74:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t __darwin_suseconds_t; / [???] microseconds /
^
note: '__int128_t' declared here
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:75:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t __darwin_uid_t; / [???] user IDs /
^
note: '__uint128_t' declared here
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:76:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t __darwin_useconds_t; / [???] microseconds */
^
note: '__uint128_t' declared here
In file included from psutil/_psutil_common.c:9:
In file included from /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:25:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/stdio.h:64:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/_stdio.h:71:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/_types.h:43:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t __darwin_wctype_t;
^
note: '__uint128_t' declared here
In file included from psutil/_psutil_common.c:9:
In file included from /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:25:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/stdio.h:64:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/_stdio.h:75:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types/_va_list.h:31:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/machine/types.h:37:2: error: architecture not supported
#error architecture not supported
^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
error: command 'xcrun' failed with exit status 1

Error processing command. Ignoring because optional. (optional:setup.py:third_party/python/psutil:build_ext:--inplace)
Error processing command. Ignoring because optional. (optional:packages.txt:comm/build/virtualenv_packages.txt)

It looks like your machine's XCode installation is fundamentally broken in some way.

Is this a newly configured machine, or did builds used to work on this machine?

Did you recently upgrade XCode?

Have you done a Google search for this error and looked at anything others have had to say? What I'm reading is that 1) Upgrading XCode and 2) Restarting the machine are a good start.

Flags: needinfo?(longsonr)

This isn't a newly configured machine (I've been using it for 3 years) and I haven't recently updated XCode, I think the last update was 26th September.

Builds still work on this machine even with the above errors. These are new errors though.

I can't upgrade XCode as this is the latest release of XCode. I can debug builds with it so it seems to be working pretty well apart from the mach bootstrap spew.

Flags: needinfo?(longsonr)

That's really odd. The relevant change that may have triggered this that happened this week is that we upgraded to a new version virtualenv, but I don't know why that would trigger this.

I suspect that if you apply this patch, you won't see this problem any more. Is that right? (This patch has been approved for a while and WILL be landed, I'm just waiting for some changes to land in CI first.)

Flags: needinfo?(longsonr)

Applying that patch gets this spew instead. Installation still continues and everything still appears to work.

ERROR: Command errored out with exit status 1:
 command: /<elided>/.mozbuild/_virtualenvs/mach/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/8m/cvvwyg6561q6km2byhk_86sm0000gn/T/pip-install-37ulfdrv/psutil/setup.py'"'"'; __file__='"'"'/private/var/folders/8m/cvvwyg6561q6km2byhk_86sm0000gn/T/pip-install-37ulfdrv/psutil/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/8m/cvvwyg6561q6km2byhk_86sm0000gn/T/pip-record-zkstd1pi/install-record.txt --single-version-externally-managed --compile --install-headers /<elided>/.mozbuild/_virtualenvs/mach/include/site/python3.8/psutil
     cwd: /private/var/folders/8m/cvvwyg6561q6km2byhk_86sm0000gn/T/pip-install-37ulfdrv/psutil/
Complete output (140 lines):
running install
running build
running build_py
creating build
creating build/lib.macosx-10.14.6-x86_64-3.8
creating build/lib.macosx-10.14.6-x86_64-3.8/psutil
copying psutil/_pswindows.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil
copying psutil/_common.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil
copying psutil/__init__.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil
copying psutil/_psosx.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil
copying psutil/_psbsd.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil
copying psutil/_psaix.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil
copying psutil/_pslinux.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil
copying psutil/_compat.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil
copying psutil/_psposix.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil
copying psutil/_pssunos.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil
creating build/lib.macosx-10.14.6-x86_64-3.8/psutil/tests
copying psutil/tests/test_contracts.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil/tests
copying psutil/tests/test_connections.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil/tests
copying psutil/tests/runner.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil/tests
copying psutil/tests/test_unicode.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil/tests
copying psutil/tests/test_misc.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil/tests
copying psutil/tests/test_posix.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil/tests
copying psutil/tests/test_linux.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil/tests
copying psutil/tests/test_sunos.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil/tests
copying psutil/tests/__init__.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil/tests
copying psutil/tests/test_aix.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil/tests
copying psutil/tests/test_process.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil/tests
copying psutil/tests/test_bsd.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil/tests
copying psutil/tests/test_system.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil/tests
copying psutil/tests/test_osx.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil/tests
copying psutil/tests/test_memory_leaks.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil/tests
copying psutil/tests/test_windows.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil/tests
copying psutil/tests/__main__.py -> build/lib.macosx-10.14.6-x86_64-3.8/psutil/tests
running build_ext
building 'psutil._psutil_osx' extension
creating build/temp.macosx-10.14.6-x86_64-3.8
creating build/temp.macosx-10.14.6-x86_64-3.8/psutil
creating build/temp.macosx-10.14.6-x86_64-3.8/psutil/arch
creating build/temp.macosx-10.14.6-x86_64-3.8/psutil/arch/osx
xcrun -sdk macosx clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=570 -DPSUTIL_OSX=1 -I/<elided>/.mozbuild/_virtualenvs/mach/include -I/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -c psutil/_psutil_common.c -o build/temp.macosx-10.14.6-x86_64-3.8/psutil/_psutil_common.o
In file included from psutil/_psutil_common.c:9:
In file included from /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:11:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0/include/limits.h:21:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/limits.h:63:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/cdefs.h:807:2: error: Unsupported architecture
#error Unsupported architecture
 ^
In file included from psutil/_psutil_common.c:9:
In file included from /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:11:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0/include/limits.h:21:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/limits.h:64:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/machine/limits.h:8:2: error: architecture not supported
#error architecture not supported
 ^
In file included from psutil/_psutil_common.c:9:
In file included from /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:25:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/stdio.h:64:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/_stdio.h:71:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/_types.h:27:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:33:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/machine/_types.h:34:2: error: architecture not supported
#error architecture not supported
 ^
In file included from psutil/_psutil_common.c:9:
In file included from /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:25:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/stdio.h:64:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/_stdio.h:71:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/_types.h:27:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:55:9: error: unknown type name '__int64_t'
typedef __int64_t       __darwin_blkcnt_t;      /* total blocks */
        ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:56:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t       __darwin_blksize_t;     /* preferred block size */
        ^
note: '__int128_t' declared here
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:57:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t       __darwin_dev_t;         /* dev_t */
        ^
note: '__int128_t' declared here
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:60:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_gid_t;         /* [???] process and group IDs */
        ^
note: '__uint128_t' declared here
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:61:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_id_t;          /* [XSI] pid_t, uid_t, or gid_t*/
        ^
note: '__uint128_t' declared here
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:62:9: error: unknown type name '__uint64_t'
typedef __uint64_t      __darwin_ino64_t;       /* [???] Used for 64 bit inodes */
        ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:68:9: error: unknown type name '__darwin_natural_t'
typedef __darwin_natural_t __darwin_mach_port_name_t; /* Used by mach */
        ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:70:9: error: unknown type name '__uint16_t'; did you mean '__uint128_t'?
typedef __uint16_t      __darwin_mode_t;        /* [???] Some file attributes */
        ^
note: '__uint128_t' declared here
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:71:9: error: unknown type name '__int64_t'
typedef __int64_t       __darwin_off_t;         /* [???] Used for file sizes */
        ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:72:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t       __darwin_pid_t;         /* [???] process and group IDs */
        ^
note: '__int128_t' declared here
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:73:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_sigset_t;      /* [???] signal set */
        ^
note: '__uint128_t' declared here
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:74:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t       __darwin_suseconds_t;   /* [???] microseconds */
        ^
note: '__int128_t' declared here
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:75:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_uid_t;         /* [???] user IDs */
        ^
note: '__uint128_t' declared here
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:76:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_useconds_t;    /* [???] microseconds */
        ^
note: '__uint128_t' declared here
In file included from psutil/_psutil_common.c:9:
In file included from /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:25:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/stdio.h:64:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/_stdio.h:71:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/_types.h:43:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_wctype_t;
        ^
note: '__uint128_t' declared here
In file included from psutil/_psutil_common.c:9:
In file included from /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:25:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/stdio.h:64:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/_stdio.h:75:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_types/_va_list.h:31:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/machine/types.h:37:2: error: architecture not supported
#error architecture not supported
 ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
error: command 'xcrun' failed with exit status 1
----------------------------------------

ERROR: Command errored out with exit status 1: /<elided>/.mozbuild/_virtualenvs/mach/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/8m/cvvwyg6561q6km2byhk_86sm0000gn/T/pip-install-37ulfdrv/psutil/setup.py'"'"'; file='"'"'/private/var/folders/8m/cvvwyg6561q6km2byhk_86sm0000gn/T/pip-install-37ulfdrv/psutil/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /private/var/folders/8m/cvvwyg6561q6km2byhk_86sm0000gn/T/pip-record-zkstd1pi/install-record.txt --single-version-externally-managed --compile --install-headers /<elided>/.mozbuild/_virtualenvs/mach/include/site/python3.8/psutil Check the logs for full command output.

Flags: needinfo?(longsonr) → needinfo?(rstewart)

To unblock yourself, you can comment out the line from python/mozbuild/mozbuild/mach_commands.py that installs psutil==5.7.0.

Can you reproduce the issue if you just pip3 install --no-binary psutil==5.7.0? That would again suggest some sort of problem with your machine.

This doesn't seem to have anything to do with the build system or with bootstrap, and I can't reproduce on my own machine, so I'm limited in the amount of support I can provide. The most I can do is Google around. Two things you can try:

  1. Are you using python3 from Homebrew? If so, uninstalling/brew doctor/re-installing may help.

  2. Try export ARCHFLAGS="-arch x86_64".

Status: NEW → RESOLVED
Closed: 4 years ago
Flags: needinfo?(rstewart)
Resolution: --- → WORKSFORME

Ah!!! I can repro this if I uninstall the Homebrew Python 3.

I wonder if that means this is a duplicate of bug 1669934.

Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
See Also: → 1669934

Okay. This is apparently a problem with the system-level pip3 that macOS installs. I ran sudo /usr/bin/pip3 install --upgrade pip, and that fixed the psutil install. So anyone who's encountering this should just upgrade their pip, or use Homebrew pip which also works fine.

I'm going to re-close this in the meantime, but if that doesn't fix it for you, let me know.

Status: REOPENED → RESOLVED
Closed: 4 years ago → 4 years ago
Resolution: --- → FIXED

(Oh, I also made sure my XCode command-line tools were up-to-date and everything. Not sure how much of a factor that is. But it definitely did not work until I upgraded pip.)

Hmph. Talking to Mitch, it seems like it won't be that easy, something is still going wrong. :( Re-opening in the meantime. Sorry for all the churn on this, lol.

Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Hey, yeah, I'm running into the same issue when I install psutil.
I'm running:

  • Xcode 12.0.1 (12A7300)
  • Python 3.8.2 (from Xcode)
  • Pip 20.2.3

When I pip3 install psutil==5.7.0, I run into the same errors reported in the first post (architecture not supported, unknown type name, etc).
However, it's worth noting that setting the following environment variable "fixed" it:

export ARCHFLAGS="-arch x86_64"

I don't think that setting this in .bash_profile and forgetting it is the right solution, so I'll dig in a bit and see why it solved the problem for me, but was unnecessary for Ricky

Are you running Big Sur? It happens when python is universal x86-64/arm64 binaries, in which case pip wants to build both. But your xcode/clt doesn't know how to build arm64. Next version of Xcode (12.1, coming soon) is supposed to fix that.

No, I'm still running MacOS Catalina.
Good point about about the universal python, my instance is both x86_64 and arm64

$ file /Applications/Xcode.app/Contents/Developer/usr/bin/python3
/Applications/Xcode.app/Contents/Developer/usr/bin/python3: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64:Mach-O 64-bit executable arm64]
/Applications/Xcode.app/Contents/Developer/usr/bin/python3 (for architecture x86_64):	Mach-O 64-bit executable x86_64
/Applications/Xcode.app/Contents/Developer/usr/bin/python3 (for architecture arm64):	Mach-O 64-bit executable arm64

Weirdly, Ricky is no longer having psutil failures (according to the comment above), but I haven't been able to track down how our systems are different:

  • Our Xcode versions (and build numbers) match
  • Our pip version match
  • Our Xcode python versions match
  • When he does file /Applications/Xcode.app/.../python3, he also sees a universal binary with both x86_64 and arm64 support.

Weird. Perhaps he somehow has an arm64-capable Xcode/clt?

"distutils" builds packages to fit the architectures of the active
python.
However, the system Mac SDK doesn't always support all the same
architectures as the local Python binary.
To resolve this, we explicitly influence the build to only compile for
the specific architecture that the system runs on.

Assumes that "platform.machine()" will always line up with the "-arch"
flag of compilers.

Assignee: nobody → mhentges
Pushed by mhentges@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/af3b04e5e877
"Universal" python binaries shouldn't cause setup.py fails r=firefox-build-system-reviewers,rstewart

https://hg.mozilla.org/mozilla-central/rev/af3b04e5e877

Status: REOPENED → RESOLVED
Closed: 4 years ago → 4 years ago
status-firefox84: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 84 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: