1789794 - Build issues with Thunderbird 102.2.1 and 102.2.2 on Archlinux due to outdated pip and typing_extensions packages

Status: RESOLVED WONTFIX

(Thunderbird :: Build Config, defect)

Description

[Pascal Ernster]

Attachment: fix.patch

Steps to reproduce:

Since Thunderbird 102.2.1, I'm unable to compile Thunderbird on Archlinux due to outdated pip and typing_extensions packages.

I've used the PKGBUILD from here:
https://github.com/archlinux/svntogit-packages/tree/ecdf6e653f9a3ee3a1ea18e6de3c7fd4da738bb2/trunk

and simply updated the pkgver to 102.2.1 (or later 102.2.2) and the sha512sums in the PKGBUILD.

Actual results:

When trying the build the package (using either "makepkg" or "testing-x86_64-build", after a few minutes, I get the following error messages:

2022-09-07_03:11:54 voluptuous 0.12.1 /build/thunderbird/src/thunderbird-102.2.1/third_party/python/voluptuous
2022-09-07_03:11:54 wheel 0.37.0 /tmp/tmppbolsq_q/lib/python3.10/site-packages pip
2022-09-07_03:11:54 yamllint 1.23.0 /build/thunderbird/src/thunderbird-102.2.1/third_party/python/yamllint
2022-09-07_03:11:54 yarl 1.6.3 /build/thunderbird/src/thunderbird-102.2.1/third_party/python/yarl
2022-09-07_03:11:54 zipp 3.4.1 /build/thunderbird/src/thunderbird-102.2.1/third_party/python/zipp
2022-09-07_03:11:55 WARNING: You are using pip version 21.2.3; however, version 22.2.2 is available.
2022-09-07_03:11:55 You should consider upgrading via the '/tmp/tmppbolsq_q/bin/python -m pip install --upgrade pip' command.
2022-09-07_03:11:55 pydantic 1.10.2 has requirement typing-extensions>=4.1.0, but you have typing-extensions 3.10.0.0.
2022-09-07_03:11:55
2022-09-07_03:11:55 Creating global state directory from environment variable: /build/thunderbird/src/mozbuild
2022-09-07_03:11:55 Running "pip check" to verify compatibility between the system Python and the "mach" site.
2022-09-07_03:11:55 Traceback (most recent call last):
2022-09-07_03:11:55 File "/build/thunderbird/src/thunderbird-102.2.1/./mach", line 96, in <module>
2022-09-07_03:11:55 main(sys.argv[1:])
2022-09-07_03:11:55 File "/build/thunderbird/src/thunderbird-102.2.1/./mach", line 88, in main
2022-09-07_03:11:55 mach = check_and_get_mach(os.path.dirname(os.path.realpath(file)))
2022-09-07_03:11:55 File "/build/thunderbird/src/thunderbird-102.2.1/./mach", line 35, in check_and_get_mach
2022-09-07_03:11:55 return load_mach(dir_path, mach_path)
2022-09-07_03:11:55 File "/build/thunderbird/src/thunderbird-102.2.1/./mach", line 21, in load_mach
2022-09-07_03:11:55 return mach_initialize.initialize(dir_path)
2022-09-07_03:11:55 File "/build/thunderbird/src/thunderbird-102.2.1/comm/build/mach_initialize.py", line 50, in initialize
2022-09-07_03:11:55 driver = mach_init.initialize(topsrcdir)
2022-09-07_03:11:55 File "/build/thunderbird/src/thunderbird-102.2.1/build/mach_initialize.py", line 153, in initialize
2022-09-07_03:11:55 _activate_python_environment(
2022-09-07_03:11:55 File "/build/thunderbird/src/thunderbird-102.2.1/build/mach_initialize.py", line 99, in _activate_python_environment
2022-09-07_03:11:55 mach_environment.activate()
2022-09-07_03:11:55 File "/build/thunderbird/src/thunderbird-102.2.1/python/mach/mach/site.py", line 359, in activate
2022-09-07_03:11:55 self.ensure()
2022-09-07_03:11:55 File "/build/thunderbird/src/thunderbird-102.2.1/python/mach/mach/site.py", line 337, in ensure
2022-09-07_03:11:55 result = self._up_to_date()
2022-09-07_03:11:55 File "/build/thunderbird/src/thunderbird-102.2.1/python/mach/mach/site.py", line 322, in _up_to_date
2022-09-07_03:11:55 _assert_pip_check(
2022-09-07_03:11:55 File "/build/thunderbird/src/thunderbird-102.2.1/python/mach/mach/site.py", line 1121, in _assert_pip_check
2022-09-07_03:11:55 raise Exception(
2022-09-07_03:11:55 Exception: According to "pip check", the current Python environment has package-compatibility issues.
2022-09-07_03:11:55 ==> ERROR: A failure occurred in build().
2022-09-07_03:11:55 Aborting...

Expected results:

The package should build instead of throwing errors.

I've managed to pix the issue by replacing the outdated pip and typing_extensions packages that ship with the Thunderbird source tarball:

rm -rf third_party/python/pip/*
bsdtar --extract --file "${srcdir}/pip-22.2.2-py3-none-any.whl" --directory "${srcdir}/${pkgname}-${pkgver}/third_party/python/pip"
rm -rf third_party/python/typing_extensions/*
bsdtar --extract --file "${srcdir}/typing_extensions-4.3.0-py3-none-any.whl" --directory "${srcdir}/${pkgname}-${pkgver}/third_party/python/typing_extensions"

I've also attached a patch that updates poetry.lock, requirements.in and requirements.txt in third_party/python, though it appears like only applying that patch without also replacing/updating the oudated packages contained within the source tarball isn't enough to fix the build process.

Comment 1

[Rob Lemley [:rjl]]

You're better off changing the PKGBUILD file to set MACH_BUILD_PYTHON_NATIVE_PACKAGE_SOURCE to either pip or none. system is really meant for use in Mozilla CI builds where compatibility is ensured. Per FS#75848, the Firefox package is using pip to deal with this same issue.

See https://firefox-source-docs.mozilla.org/python/index.html#mach-build-native-3rd-party-dependencies for what the different values for MACH_BUILD_PYTHON_NATIVE_PACKAGE_SOURCE are. I just built the Arch package with this diff applied and setting MACH_BUILD_PYTHON_NATIVE_PACKAGE_SOURCE to none. That's probably the way to go so you're not relying on PyPi services to build.

FWIW, you should be able to remove yasm, imake, xorg-server-xvfb, and autoconf2.13 from makedepends and add json-c as a dependency along with ac_add_options --with-system-jsonc in mozconfig.cfg. The --disable-elf-hack line should be safe to remove from mozconfig.cfg as well.

I'm going to mark this WONTFIX, as updating PKGBUILD is the preferred fix.

Comment 2

[Pascal Ernster]

(In reply to Rob Lemley [:rjl] from comment #1)

You're better off changing the PKGBUILD file to set MACH_BUILD_PYTHON_NATIVE_PACKAGE_SOURCE to either pip or none. system is really meant for use in Mozilla CI builds where compatibility is ensured. Per FS#75848, the Firefox package is using pip to deal with this same issue.

See https://firefox-source-docs.mozilla.org/python/index.html#mach-build-native-3rd-party-dependencies for what the different values for MACH_BUILD_PYTHON_NATIVE_PACKAGE_SOURCE are. I just built the Arch package with this diff applied and setting MACH_BUILD_PYTHON_NATIVE_PACKAGE_SOURCE to none. That's probably the way to go so you're not relying on PyPi services to build.

FWIW, you should be able to remove yasm, imake, xorg-server-xvfb, and autoconf2.13 from makedepends and add json-c as a dependency along with ac_add_options --with-system-jsonc in mozconfig.cfg. The --disable-elf-hack line should be safe to remove from mozconfig.cfg as well.

I'm going to mark this WONTFIX, as updating PKGBUILD is the preferred fix.

Thanks a lot for your suggestions, they were applied today to the Thunderbird package in Archlinux :)

https://github.com/archlinux/svntogit-packages/commit/b62f23f9522ca46080b1fd1369415396e841ab13