1520282 - "Host utilities not found: environment variable MOZ_HOST_BIN is not set to a directory containing host xpcshell" when trying to run WPT in local x86 emulator build (mach not waiting for input)

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: Testing, defect, P2)

Description

[Botond Ballo [:botond]]

I'm trying to run a web platform test in a local x86 emulator build, and I'm getting the following error:

$ ./mach wpt testing/web-platform/tests/css/cssom-view/scroll-behavior-smooth.html
 0:00.00 INFO Creating config file /home/botond/dev/projects/mozilla/out-of-tree-objdirs/central-android-x86/_tests/web-platform/wptrunner.local.ini
 0:00.01 INFO Creating directory /home/botond/dev/projects/mozilla/out-of-tree-objdirs/central-android-x86/_tests/web-platform/meta
 0:00.01 INFO Creating directory /home/botond/dev/projects/mozilla/out-of-tree-objdirs/central-android-x86/_tests/web-platform/mozilla/meta
 0:02.27 INFO Downloading manifest from https://index.taskcluster.net/v1/task/gecko.v2.mozilla-central.revision.7be73b4e5299792d47667c8587d56a5b2e36c71e.source.manifest-upload/artifacts/public/manifests.tar.gz
 0:23.15 adb INFO Using adb 1.0.40
 0:23.38 adb INFO adbd running as root
 0:23.60 adb INFO su 0 supported
 0:23.82 adb INFO /system/bin/ls -1A supported
 0:23.93 adb INFO Native cp support: True
 0:24.05 adb INFO Native chmod -R support: True
 0:24.16 adb INFO Native chown -R support: True
 0:24.27 adb INFO Setting SELinux Permissive
 0:24.60 adb INFO Native pidof support: True
Host utilities not found: environment variable MOZ_HOST_BIN is not set to a directory containing host xpcshell
Download and setup your host utilities? (Y/n) Error running mach:

    ['wpt', 'testing/web-platform/tests/css/cssom-view/scroll-behavior-smooth.html']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

EOFError

  File "/home/botond/dev/projects/mozilla/central/testing/web-platform/mach_commands.py", line 380, in run_wpt
    return self.run_web_platform_tests(**params)
  File "/home/botond/dev/projects/mozilla/central/testing/web-platform/mach_commands.py", line 372, in run_web_platform_tests
    return wpt_runner.run(logger, **params)
  File "/home/botond/dev/projects/mozilla/central/testing/web-platform/mach_commands_base.py", line 35, in run
    kwargs = wptcommandline.check_args(self.setup.kwargs_common(kwargs))
  File "/home/botond/dev/projects/mozilla/central/testing/web-platform/mach_commands.py", line 48, in kwargs_common
    verify_android_device(self, install=True, verbose=False, xre=True, app=package_name)
  File "/home/botond/dev/projects/mozilla/central/testing/mozbase/mozrunner/mozrunner/devices/android_device.py", line 304, in verify_android_device
    "Download and setup your host utilities? (Y/n) ").strip()

Note that at the step where it prints:

Host utilities not found: environment variable MOZ_HOST_BIN is not set to a directory containing host xpcshell
Download and setup your host utilities? (Y/n)

it doesn't pause and give me a chance to answer "Y", it just prints the rest of the error and quits.

Comment 1

[Geoff Brown [:gbrown]]

I have seen something like that before also, but not consistently: Usually that prompt works for me, waiting for input. What kind of console/terminal are you using?

Workaround: remove the raw_input call here and just call _install_host_utils unconditionally.

Comment 2

[Geoff Brown [:gbrown]]

https://searchfox.org/mozilla-central/rev/c21d6620d384dfb13ede6054015da05a6353b899/testing/mozbase/mozrunner/mozrunner/devices/android_device.py#303-306

Comment 3

[Botond Ballo [:botond]]

(In reply to Geoff Brown [:gbrown] from comment #1)

I have seen something like that before also, but not consistently: Usually that prompt works for me, waiting for input. What kind of console/terminal are you using?

Huh. I tried it again in a new terminal window (KDE Konsole, on Linux), and it worked. Perhaps my original terminal window was in some sort of bad state from previous commands.

Comment 4

[Geoff Brown [:gbrown]]

I found a way to reproduce this:

• ensure no android devices connected and no emulators running
• delete ~/.mozbuild/android-device/host-utils*
• run a mach test command, like 'mach mochitest testing/mochitest/tests/Harness_sanity'

The mach test command will offer to launch an emulator - accept. It will then offer to update host-utils, but will throw EOFError rather than waiting for input.

Comment 5

[Geoff Brown [:gbrown]]

Attachment: use new pipe for emulator stdin

We have been launching the emulator with stdin=None, which indicates the child inherits the stdin file handle from the parent. Subsequently, raw_input() fails with EOFError. It seems that's easily avoided with stdin=PIPE.

Comment 6

[Bob Clary [:bc] (inactive)]

Comment on attachment 9038416 [details] [diff] [review]
use new pipe for emulator stdin

Review of attachment 9038416 [details] [diff] [review]:
-----------------------------------------------------------------

r+. Good catch.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/cac13d592b66
When launching android emulator from mach, use a new pipe to stdin; r=bc

Comment 8

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/cac13d592b66