1650410 - Running android tests with macOS host: ImportError: cannot import name _psutil_osx

Status: RESOLVED FIXED

(Testing :: General, defect, P2)

Description

[Andrew Creskey [:acreskey]]

When I try to run the following android test:
./mach reftest --this-chunk 1 --total-chunks 100
on a ./mach bootstrapped macOS Catalina, 10.15.5 host, I see the following error:
ImportError: cannot import name _psutil_osx

Full stack trace:

ImportError: cannot import name _psutil_osx

  File "/Users/acreskey/dev/src/mozilla-central/layout/tools/reftest/mach_commands.py", line 208, in run_reftest
    return self._run_reftest(**kwargs)
  File "/Users/acreskey/dev/src/mozilla-central/layout/tools/reftest/mach_commands.py", line 243, in _run_reftest
    return reftest.run_android_test(**kwargs)
  File "/Users/acreskey/dev/src/mozilla-central/layout/tools/reftest/mach_commands.py", line 101, in run_android_test
    import remotereftest
  File "/Users/acreskey/dev/src/mozilla-central/build/mach_bootstrap.py", line 476, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/Users/acreskey/dev/src/build/obj-release-android/_tests/reftest/remotereftest.py", line 9, in <module>
    import psutil
  File "/Users/acreskey/dev/src/mozilla-central/build/mach_bootstrap.py", line 476, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/Users/acreskey/dev/src/mozilla-central/third_party/python/psutil/psutil/__init__.py", line 161, in <module>
    from . import _psosx as _psplatform
  File "/Users/acreskey/dev/src/mozilla-central/build/mach_bootstrap.py", line 476, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/Users/acreskey/dev/src/mozilla-central/third_party/python/psutil/psutil/_psosx.py", line 15, in <module>
    from . import _psutil_osx as cext

I've tried to manually install psutil at the terminal, but it appears to already be installed:
Requirement already satisfied: psutil in /Users/acreskey/.pyenv/versions/2.7.11/lib/python2.7/site-packages (5.6.3)

Comment 1

[Geoff Brown [:gbrown]]

Do you get the same error from python -c "import psutil"?

Comment 2

[Andrew Creskey [:acreskey]]

I get no output when I run python -c "import psutil" from the terminal.

But when I run the tests I do see the same ImportError: cannot import name _psutil_osx

Comment 3

[Geoff Brown [:gbrown]]

Interesting. That means you can import psutil from the terminal.

I wonder:

• would an update of third_party/python/psutil help?
• can remotereftest.py eliminate its use of psutil?
• can remotereftest.py warn and continue when psutil is unavailable?

Comment 4

[Geoff Brown [:gbrown]]

(In reply to Geoff Brown [:gbrown] from comment #3)

• would an update of third_party/python/psutil help?

No, it is up to date, bug 1620513.

Comment 5

[Geoff Brown [:gbrown]]

(In reply to Geoff Brown [:gbrown] from comment #3)

• can remotereftest.py eliminate its use of psutil?

I can't think of a way. remotereftest.py uses psutil to find ssltunnel and xpcshell processes that might interfere with this run; I don't know of a good cross-platform alternative.

Comment 6

[Geoff Brown [:gbrown]]

xpcshell-test and mochitest already check for psutil ImportError.

Comment 7

[Geoff Brown [:gbrown]]

Attachment: Bug 1650410 - Tolerate psutil import failure in android reftests; r=

Comment 8

[Geoff Brown [:gbrown]]

:acreskey - Try running reftests with this patch, please.

Comment 9

[Andrew Creskey [:acreskey]]

:gbrown, with your patch I can successfully run the reftests -- thank you!

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/bb12d0328ce9
Tolerate psutil import failure in android reftests; r=bc

Comment 11

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/bb12d0328ce9