Closed Bug 1763695 Opened 4 months ago Closed 4 months ago

Perma mach.site.InstallPipRequirementsException: As part of validation after installing "/builds/worker/checkouts/gecko/tools/lint/python/black_requirements.txt" into the "python-test" site, the site appears to contain installed packages th

Categories

(Firefox Build System :: Lint and Formatting, defect, P5)

defect

Tracking

(firefox-esr91 unaffected, firefox99 unaffected, firefox100 unaffected, firefox101 fixed)

RESOLVED FIXED
101 Branch
Tracking Status
firefox-esr91 --- unaffected
firefox99 --- unaffected
firefox100 --- unaffected
firefox101 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: mhentges)

References

(Regression)

Details

(Keywords: intermittent-failure, regression, Whiteboard: [retriggered])

Attachments

(1 file)

Filed by: abutkovits [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=373780818&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/IdXec23ATQK_dkH-Jv2Ulg/runs/0/artifacts/public/logs/live_backing.log


[task 2022-04-07T18:59:45.117Z] Successfully installed black-21.11b1 click-8.0.3 dataclasses-0.6 mypy-extensions-0.4.3 platformdirs-2.4.0 regex-2021.11.10 tomli-1.2.2 typed-ast-1.5.2 typing-extensions-3.10.0.2
[task 2022-04-07T18:59:45.117Z] 
[task 2022-04-07T18:59:45.117Z] Error running mach:
[task 2022-04-07T18:59:45.117Z] 
[task 2022-04-07T18:59:45.117Z]     ['python-test', '--subsuite', 'mozlint', '--run-slow']
[task 2022-04-07T18:59:45.118Z] 
[task 2022-04-07T18:59:45.118Z] The error occurred in code that was called by the mach command. This is either
[task 2022-04-07T18:59:45.118Z] a bug in the called code itself or in the way that mach is calling it.
[task 2022-04-07T18:59:45.118Z] You can invoke |./mach busted| to check if this issue is already on file. If it
[task 2022-04-07T18:59:45.118Z] isn't, please use |./mach busted file python-test| to report it. If |./mach busted| is
[task 2022-04-07T18:59:45.118Z] misbehaving, you can also inspect the dependencies of bug 1543241.
[task 2022-04-07T18:59:45.119Z] 
[task 2022-04-07T18:59:45.119Z] If filing a bug, please include the full output of mach, including this error
[task 2022-04-07T18:59:45.119Z] message.
[task 2022-04-07T18:59:45.119Z] 
[task 2022-04-07T18:59:45.119Z] The details of the failure are as follows:
[task 2022-04-07T18:59:45.119Z] 
[task 2022-04-07T18:59:45.119Z] mach.site.InstallPipRequirementsException: As part of validation after installing "/builds/worker/checkouts/gecko/tools/lint/python/black_requirements.txt" into the "python-test" site, the site appears to contain installed packages that are incompatible with each other.
[task 2022-04-07T18:59:45.119Z] 
[task 2022-04-07T18:59:45.119Z]   File "/builds/worker/checkouts/gecko/python/mach_commands.py", line 146, in python_test
[task 2022-04-07T18:59:45.119Z]     return run_python_tests(command_context, *args, **kwargs)
[task 2022-04-07T18:59:45.119Z]   File "/builds/worker/checkouts/gecko/python/mach_commands.py", line 210, in run_python_tests
[task 2022-04-07T18:59:45.120Z]     test["requirements"], quiet=True
[task 2022-04-07T18:59:45.120Z]   File "/builds/worker/checkouts/gecko/python/mach/mach/site.py", line 667, in install_pip_requirements
[task 2022-04-07T18:59:45.120Z]     f'As part of validation after installing "{path}" into the '
[taskcluster 2022-04-07 18:59:45.570Z] === Task Finished ===
[taskcluster 2022-04-07 18:59:45.571Z] Unsuccessful task run with exit code: 1 completed in 599.097 seconds
Regressed by: 1617283
Summary: Intermittent [tier2] mach.site.InstallPipRequirementsException: As part of validation after installing "/builds/worker/checkouts/gecko/tools/lint/python/black_requirements.txt" into the "python-test" site, the site appears to contain installed packages th → Perma [tier2] mach.site.InstallPipRequirementsException: As part of validation after installing "/builds/worker/checkouts/gecko/tools/lint/python/black_requirements.txt" into the "python-test" site, the site appears to contain installed packages th

:ssummar, since you are the author of the regressor, bug 1617283, could you take a look?
For more information, please visit auto_nag documentation.

Flags: needinfo?(shazib)

Set release status flags based on info from the regressing bug 1617283

Has Regression Range: --- → yes

Ofc. I'm looking into it. Thanks for the ping.

Flags: needinfo?(shazib)

Bug 1617283 is Windows audio C++ code and this failure is about python lint package dependencies. Turns out this error is actually from bug 1659593, which landed a few patches earlier. Compare pushes immediately before and after its patches on central.

It's just some versioning issue. The relevant log lines are

123456789101112[task 2022-04-07T19:11:23.347Z] Collecting importlib-metadata>=3.6
[task 2022-04-07T19:11:23.362Z]   Downloading importlib_metadata-4.11.3-py3-none-any.whl (18 kB)
[...]
[task 2022-04-07T19:11:24.067Z] Installing collected packages: msgpack, webencodings, pywin32-ctypes, pylev, ptyprocess, lockfile, importlib-metadata, filelock, distlib, crashtest, cachecontrol, virtualenv, tomlkit, shellingham, requests-toolbelt, poetry-core, pkginfo, pexpect, keyring, html5lib, entrypoints, cleo, cachy, poetry
[task 2022-04-07T19:11:27.190Z]   Attempting uninstall: importlib-metadata
[task 2022-04-07T19:11:27.204Z]     Found existing installation: importlib-metadata 1.7.0
[task 2022-04-07T19:11:27.204Z]     Not uninstalling importlib-metadata at z:\task_164935603523361\build\src\third_party\python\importlib_metadata, outside environment Z:\task_164935603523361\build\src\obj-x86_64-pc-mingw32\_virtualenvs\python-test
[task 2022-04-07T19:11:27.204Z]     Can't uninstall 'importlib-metadata'. No files were found to uninstall.
[task 2022-04-07T19:11:30.202Z] Successfully installed cachecontrol-0.12.10 cachy-0.3.0 cleo-1.0.0a4 crashtest-0.3.1 distlib-0.3.4 entrypoints-0.3 filelock-3.6.0 html5lib-1.1 importlib-metadata-4.11.3 keyring-23.5.0 lockfile-0.12.2 msgpack-1.0.3 pexpect-4.8.0 pkginfo-1.8.2 poetry-1.2.0a2 poetry-core-1.1.0a7 ptyprocess-0.7.0 pylev-1.4.0 pywin32-ctypes-0.2.0 requests-toolbelt-0.9.1 shellingham-1.4.0 tomlkit-0.10.1 virtualenv-20.4.4 webencodings-0.5.1
[...]
[task 2022-04-07T19:11:50.777Z] keyring 23.5.0 has requirement importlib-metadata>=3.6, but you have importlib-metadata 1.7.0.

NI to mhentges as bug 1659593 author

Flags: needinfo?(mhentges)
Regressed by: 1659593
No longer regressed by: 1617283
Whiteboard: [retriggered]

Running the mozlint tests involves installing lint package lockfiles
into the python-test site. This conflicts with existing packages,
which require dataclasses==0.8.

The easy workaround here is to bump the black's lockfile to also use
dataclasses==0.8.

Note that dataclasses and typing-extensions were removed from the
black_requirements.in file, because the only top-level requirement we
use is black - I don't believe the others belonged in there.

Flags: needinfo?(mhentges)
Attachment #9271727 - Attachment description: Bug 1763695: Bump black's `dataclasses` dependency → Bug 1763695: Move python-test poetry to ini "requirements"
Assignee: nobody → mhentges
Status: NEW → ASSIGNED
Summary: Perma [tier2] mach.site.InstallPipRequirementsException: As part of validation after installing "/builds/worker/checkouts/gecko/tools/lint/python/black_requirements.txt" into the "python-test" site, the site appears to contain installed packages th → Perma mach.site.InstallPipRequirementsException: As part of validation after installing "/builds/worker/checkouts/gecko/tools/lint/python/black_requirements.txt" into the "python-test" site, the site appears to contain installed packages th
Duplicate of this bug: 1764460
Pushed by mhentges@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/9a0d0a399266
Move python-test poetry to ini "requirements" r=ahal
Status: ASSIGNED → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED
Target Milestone: --- → 101 Branch
You need to log in before you can comment on or make changes to this bug.