1912605 - zstandard fails to build on aarch64 windows python

Status: RESOLVED FIXED

(Firefox Build System :: General, task, P3)

Description

[Kagami Rosylight [:saschanaz] (they/them)]

Collecting zstandard<=0.22.0,>=0.11.1
  Using cached zstandard-0.22.0.tar.gz (660 kB)
  Installing build dependencies ... error
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [56 lines of output]
      Collecting cffi==1.16.0
        Using cached cffi-1.16.0.tar.gz (512 kB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'error'
        error: subprocess-exited-with-error

        Getting requirements to build wheel did not run successfully.
        exit code: 1

        [33 lines of output]
        Traceback (most recent call last):
          File "D:\gecko\third_party\python\pip\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
            main()
          File "D:\gecko\third_party\python\pip\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
            json_out['return_val'] = hook(**hook_input['kwargs'])
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          File "D:\gecko\third_party\python\pip\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel
            return hook(config_settings)
                   ^^^^^^^^^^^^^^^^^^^^^
          File "C:\Users\sasch\AppData\Local\Temp\pip-build-env-f6yfun9m\overlay\Lib\site-packages\setuptools\build_meta.py", line 327, in get_requires_for_build_wheel
            return self._get_build_requires(config_settings, requirements=[])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          File "C:\Users\sasch\AppData\Local\Temp\pip-build-env-f6yfun9m\overlay\Lib\site-packages\setuptools\build_meta.py", line 297, in _get_build_requires
            self.run_setup()
          File "C:\Users\sasch\AppData\Local\Temp\pip-build-env-f6yfun9m\overlay\Lib\site-packages\setuptools\build_meta.py", line 313, in run_setup
            exec(code, locals())
          File "<string>", line 126, in <module>
          File "<string>", line 105, in uses_msvc
          File "C:\Users\sasch\AppData\Local\Temp\pip-build-env-f6yfun9m\overlay\Lib\site-packages\setuptools\_distutils\command\config.py", line 220, in try_compile
            self._compile(body, headers, include_dirs, lang)
          File "C:\Users\sasch\AppData\Local\Temp\pip-build-env-f6yfun9m\overlay\Lib\site-packages\setuptools\_distutils\command\config.py", line 132, in _compile
            self.compiler.compile([src], include_dirs=include_dirs)
          File "C:\Users\sasch\AppData\Local\Temp\pip-build-env-f6yfun9m\overlay\Lib\site-packages\setuptools\_distutils\_msvccompiler.py", line 343, in compile
            self.initialize()
          File "C:\Users\sasch\AppData\Local\Temp\pip-build-env-f6yfun9m\overlay\Lib\site-packages\setuptools\_distutils\_msvccompiler.py", line 253, in initialize
            vc_env = _get_vc_env(plat_spec)
                     ^^^^^^^^^^^^^^^^^^^^^^
          File "C:\Users\sasch\AppData\Local\Temp\pip-build-env-f6yfun9m\overlay\Lib\site-packages\setuptools\msvc.py", line 233, in msvc14_get_vc_env
            return _msvc14_get_vc_env(plat_spec)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          File "C:\Users\sasch\AppData\Local\Temp\pip-build-env-f6yfun9m\overlay\Lib\site-packages\setuptools\msvc.py", line 190, in _msvc14_get_vc_env
            raise distutils.errors.DistutilsPlatformError("Unable to find vcvarsall.bat")
        distutils.errors.DistutilsPlatformError: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
        [end of output]

        note: This error originates from a subprocess, and is likely not a problem with pip.
      error: subprocess-exited-with-error

      Getting requirements to build wheel did not run successfully.
      exit code: 1

      See above for output.

      note: This error originates from a subprocess, and is likely not a problem with pip.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
Could not install zstandard, so zstd archives will not be possible to extract. Continuing.

zstandard has no wheel for aarch64 windows and also fails to detect MSVC. For now this can be worked around by installing x86/x64 MSVC along with ARM64 one.

Comment 1

[Kagami Rosylight [:saschanaz] (they/them)]

This is setuptools bug.

Comment 2

[Mike Hommey [:glandium]]

Attachment: Bug 1912605 - Upgrade zstandard to 0.24.0.

This version has arm64 windows wheels \o/

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/35214d4c3b1a
https://hg.mozilla.org/integration/autoland/rev/4af47920626d
Upgrade zstandard to 0.24.0. r=firefox-build-system-reviewers,mach-reviewers,ahochheiden

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/651839f8f098
https://hg.mozilla.org/integration/autoland/rev/65e7ecfe968a
Revert "Bug 1912605 - Upgrade zstandard to 0.24.0. r=firefox-build-system-reviewers,mach-reviewers,ahochheiden" for causing gcc timeouts.

Comment 5

[agoloman]

Backed out for causing linux snap failures.

• Backout link

• Push with failures snap failures

• Failure Log

• Push with failures mochitest failures

• Failure Log

And can you take a look at these failures? These jobs didn't run since August 7th.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/b83880796690
https://hg.mozilla.org/integration/autoland/rev/41c07717914e
Upgrade zstandard to 0.24.0. r=firefox-build-system-reviewers,mach-reviewers,ahochheiden

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/9ffad7de5b98
https://hg.mozilla.org/integration/autoland/rev/617dd1405189
Revert "Bug 1912605 - Upgrade zstandard to 0.24.0. r=firefox-build-system-reviewers,mach-reviewers,ahochheiden" for causing multiple failures

Comment 8

[Cristina Horotan [:chorotan]]

Backed out for causing multiple failures

Backout link

Push with failures

Failure log gl
Failure log gl
Failure log amd

Comment 9

[Mike Hommey [:glandium]]

(In reply to Cristina Horotan [:chorotan] from comment #8)

Failure log gl

I don't see how this can be related. Well, actually, I kind of see how... but all it likely means is we have a time bomb and I just happened to trigger it.

Failure log amd

This is bug 1984560.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/fea14916d791
https://hg.mozilla.org/integration/autoland/rev/ce745b8966c2
Upgrade zstandard to 0.24.0. r=firefox-build-system-reviewers,mach-reviewers,ahochheiden

Comment 11

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/ce745b8966c2