Closed Bug 1328940 Opened 5 years ago Closed 5 years ago

mach install fails with: cannot import name _psutil_linux

Categories

(Firefox Build System :: Mach Core, enhancement)

enhancement
Not set
normal

Tracking

(firefox53 fixed)

RESOLVED FIXED
mozilla53
Tracking Status
firefox53 --- fixed

People

(Reporter: rillian, Unassigned)

Details

Attachments

(1 file)

./mach install fails on my machine:

$ ./mach install
Error running mach:

    ['install']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

ImportError: cannot import name _psutil_linux

  File "/home/giles/firefox/python/mozbuild/mozbuild/mach_commands.py", line 1088, in install
    from mozrunner.devices.android_device import verify_android_device
  File "/home/giles/firefox/build/mach_bootstrap.py", line 373, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/home/giles/firefox/testing/mozbase/mozrunner/mozrunner/devices/android_device.py", line 9, in <module>
    import psutil
  File "/home/giles/firefox/build/mach_bootstrap.py", line 373, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/home/giles/firefox/python/psutil/psutil/__init__.py", line 63, in <module>
    from . import _pslinux as _psplatform
  File "/home/giles/firefox/build/mach_bootstrap.py", line 373, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/home/giles/firefox/python/psutil/psutil/_pslinux.py", line 24, in <module>
    from . import _psutil_linux as cext

I've run ./mach bootstrap and am using the android toolchain it installed.
This is Fedora 25 on x86_64 linux. The python2-psutil package is installed.

Typing `make` in `python/psutil` results in the following error:

> gcc: error: /usr/lib/rpm/redhat/redhat-hardened-cc1: No such file or directory
> error: command 'gcc' failed with exit status 1
The issue is this switch passed to gcc: `-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1`

Installing `redhat-rpm-config` provides the missing file. I don't see this in the makefile itself, so it might be a fedora python packaging bug. We should implement the workaround in mach bootstrap in the meantime.
Comment on attachment 8824169 [details]
Bug 1328940 - mozboot: Install hardened cc for python.

https://reviewboard.mozilla.org/r/102706/#review103042

Do you have insight into the question below?

::: python/mozboot/mozboot/centosfedora.py:78
(Diff revision 1)
>                  'python-dbus',
>              ]
>  
>              self.mobile_android_packages += [
>                  'java-1.8.0-openjdk-devel',
> +                'redhat-rpm-config',

It is curious that this isn't required for desktop browser development, since I'd expect the same bits of python to be executed for both?
Attachment #8824169 - Flags: review?(nfroyd) → review+
Comment on attachment 8824169 [details]
Bug 1328940 - mozboot: Install hardened cc for python.

https://reviewboard.mozilla.org/r/102706/#review103086

::: python/mozboot/mozboot/centosfedora.py:78
(Diff revision 1)
>                  'python-dbus',
>              ]
>  
>              self.mobile_android_packages += [
>                  'java-1.8.0-openjdk-devel',
> +                'redhat-rpm-config',

I didn't investigate at the time, since I'd only hit the issue with deploying fennec builds, but it looks like it's used generally for resource measurement. However, that code is optional, in case we haven't bootstrapped yet when mach is invoked.

`mach doctor` also tries to build it, but seems to work when that fails.

`mach mochitest` *does* require it, so we should install it for all uses, not just fennec.
Comment on attachment 8824169 [details]
Bug 1328940 - mozboot: Install hardened cc for python.

https://reviewboard.mozilla.org/r/102706/#review103088
Pushed by rgiles@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/34282e47da35
mozboot: Install hardened cc for python. r=froydnj
https://hg.mozilla.org/mozilla-central/rev/34282e47da35
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla53
We've historically surrounded all "import psutil" statements inside a `try...except Exception:` because psutil has a history of failing to compile or work, especially on Windows. I'd love to make it a required dependency because it's such a nice library for inspecting process and system state.

It looks like testing/mozbase/mozrunner/mozrunner/devices/android_device.py isn't using psutil as an optional dependency (it uses psutil for inspecting running process names). So I guess psutil is no longer an optional dependency for some Fennec tasks. TIL.
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.