1294447 - Android emulator doesn't launch with ./mach marionette-test

Status: RESOLVED WONTFIX

(Remote Protocol :: Marionette, defect, P3)

Description

[No-Jun Park [:njpark]]

In https://developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette/Running_Tests#With_a_local_build_2 , the command to launch the simulator is defined as 
./mach marionette-test --emulator --app='fennec' --avd-home /path/to/.mozbuild/android-device/avd --emulator-binary /path/to/.mozbuild/android-sdk-macosx/tools/emulator

After verifying that .mozbuild folder is in ~, I have sent the following command: 
./mach marionette-test --emulator --app='fennec' --avd-home ~/.mozbuild/android-device/avd --emulator-binary ~/.mozbuild/android-sdk-macosx/tools/emulator

but got following error.  It's also worth nothing that executing ./mach run command starts up 4.2 Android simulator with no issues, while above command starts up 4.3.  (in the mozbuild folder, the avd image it has is 4.3)
 
0:00.00 LOG: MainThread INFO Using workspace for temporary data: "/Users/mozilla/hg/mozilla-central"
['/Users/mozilla/.mozbuild/android-sdk-macosx/tools/emulator', '-avd', 'mozemulator-4.3', '-port', '5554', '-show-kernel', '-debug', 'init,console,gles,memcheck,adbserver,adbclient,adb,avd_config,socket']
Error running mach:

    ['marionette-test', '--emulator', '--app=fennec', '--avd-home', '/Users/mozilla/.mozbuild/android-device/avd', '--emulator-binary', '/Users/mozilla/.mozbuild/android-sdk-macosx/tools/emulator']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

OSError: Error possibly due to runner or device args:

  File "/Users/mozilla/hg/mozilla-central/testing/marionette/mach_commands.py", line 158, in run_marionette_test
    return run_marionette(tests, topsrcdir=self.topsrcdir, **kwargs)
  File "/Users/mozilla/hg/mozilla-central/testing/marionette/mach_commands.py", line 58, in run_marionette
    failed = MarionetteHarness(MarionetteTestRunner, args=vars(args)).run()
  File "/Users/mozilla/hg/mozilla-central/testing/marionette/harness/marionette/runtests.py", line 68, in run
    runner.run_tests(tests)
  File "/Users/mozilla/hg/mozilla-central/testing/marionette/harness/marionette/runner/base.py", line 857, in run_tests
    need_external_ip = self._start_marionette()
  File "/Users/mozilla/hg/mozilla-central/testing/marionette/harness/marionette/runner/base.py", line 808, in _start_marionette
    self.marionette = self.driverclass(**self._build_kwargs())
  File "/Users/mozilla/hg/mozilla-central/testing/marionette/client/marionette_driver/marionette.py", line 580, in __init__
    self.instance.start()
  File "/Users/mozilla/hg/mozilla-central/testing/marionette/client/marionette_driver/geckoinstance.py", line 222, in start
    self.runner.start()
  File "/Users/mozilla/hg/mozilla-central/testing/mozbase/mozrunner/mozrunner/base/device.py", line 69, in start
    self.device.start()
  File "/Users/mozilla/hg/mozilla-central/testing/mozbase/mozrunner/mozrunner/devices/emulator.py", line 283, in start
    super(EmulatorAVD, self).start()
  File "/Users/mozilla/hg/mozilla-central/testing/mozbase/mozrunner/mozrunner/devices/emulator.py", line 122, in start
    self.proc.run()
  File "/Users/mozilla/hg/mozilla-central/testing/mozbase/mozprocess/mozprocess/processhandler.py", line 722, in run
    self.proc = self.Process([self.cmd] + self.args, **args)
  File "/Users/mozilla/hg/mozilla-central/testing/mozbase/mozprocess/mozprocess/processhandler.py", line 107, in __init__
    universal_newlines, startupinfo, creationflags)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
Exception RuntimeError: RuntimeError('Calling kill() on a non started process is not allowed.',) in <bound method Marionette.__del__ of <marionette_driver.marionette.Marionette object at 0x10cc0a1d0>> ignored

HG changeset ID: 6cf0089510fad8deb866136f5b92bbced9498447+

Comment 1

[Geoff Brown [:gbrown] (pto Jul 1-19)]

(In reply to No-Jun Park [:njpark] from comment #0)
> but got following error.  It's also worth nothing that executing ./mach run
> command starts up 4.2 Android simulator with no issues, while above command
> starts up 4.3.  (in the mozbuild folder, the avd image it has is 4.3)

'mach run' looks at your build configuration to determine what kind of Android build you have; if you have built for x86, 'mach run' will start the Android x86 emulator running Android 4.2; if you have built for arm, 'mach run' will start the Android arm emulator running Android 4.3.

I don't know how 'mach marionette-test' determines which emulator/image to run, but perhaps there is some confusion there. It is not possible to install or run an x86 build of Firefox for Android on the arm emulator, or vice versa.

Comment 2

[No-Jun Park [:njpark]]

Ah, then ./mach marionette-test might be thinking that I have the ARM build (or it is defaulted to run ARM emulator image), even though I built for the x86.  This also happens when I run ./mach android-emulator command as well (it always runs 4.3 regardless of the type of the build)

Just for fun, I have made an ARM build, but ./mach run gives me the following error after prompting for the start of emulator:
DMError: Non-zero return code for command: ['pm', 'list', 'packages', 'org.mozilla.'] (output: '', retval: 'None')

  File "/Users/mozilla/hg/mozilla-central/python/mozbuild/mozbuild/mach_commands.py", line 1104, in run
    verify_android_device(self, install=True)
  File "/Users/mozilla/hg/mozilla-central/testing/mozbase/mozrunner/mozrunner/devices/android_device.py", line 117, in verify_android_device
    'packages', 'org.mozilla.'])
  File "/Users/mozilla/hg/mozilla-central/testing/mozbase/mozdevice/mozdevice/devicemanager.py", line 405, in shellCheckOutput
    raise DMError("Non-zero return code for command: %s (output: '%s', retval: '%s')" % (cmd, output, retval))

And when connected to the actual Android (ARM) device, following error was given (this is after enabling USB debugging on fennec, and running the following command:
adb forward tcp:2828 tcp:2828
./mach marionette-test --address=localhost:2828)

File "/Users/mozilla/hg/mozilla-central/testing/marionette/client/marionette_driver/transport.py", line 177, in receive
    raise socket.error("No data received over socket")

Comment 3

[Geoff Brown [:gbrown] (pto Jul 1-19)]

(In reply to No-Jun Park [:njpark] from comment #2)
>  This also happens when I run ./mach android-emulator command as well
> (it always runs 4.3 regardless of the type of the build)

That is the intended behavior of 'mach android-emulator'. You can start the x86/4.2 emulator with 'mach android-emulator --version x86', or start the arm/4.3 with 'mach android-emulator' regardless of your build. 'mach android-emulator' has that flexibility because it is not related to the build. (On the other hand, to install, run, or test Firefox, you need compatibility between the emulator and the build, so those mach commands enforce that.)

> Just for fun, I have made an ARM build, but ./mach run gives me the
> following error after prompting for the start of emulator:
> DMError: Non-zero return code for command: ['pm', 'list', 'packages',
> 'org.mozilla.'] (output: '', retval: 'None')

I cannot reproduce that. mach is trying to run 'adb shell pm list packages org.mozilla.' to determine if fennec is installed. That works for me:

gbrown@mozpad:~/src$ adb shell pm list packages org.mozilla.
package:org.mozilla.fennec_gbrown
gbrown@mozpad:~/src$ adb uninstall org.mozilla.fennec_gbrown
Success
gbrown@mozpad:~/src$ adb shell pm list packages org.mozilla.
gbrown@mozpad:~/src$ 

But we shouldn't hijack this bug to debug 'mach run' -- I'll leave it to someone else to figure out the 'mach marionette-test' failure.

Comment 4

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

./mach marionette-test doesn't try to determine anything related to the emulator automatically; it just uses arm/4.3 by default and relies on command-line options otherwise. It also assumes that you've installed Fennec on the emulator.

You can override the default. This isn't highlighted in the MDN docs, but the info is available via ./mach marionette-test -h:
  --avd AVD             Name of an AVD available in your environment.See
                        mozrunner.FennecEmulatorRunner

So, please try adding --avd=mozemulator-x86 to your marionette-test command and let me know how it goes.

[1] https://dxr.mozilla.org/mozilla-central/source/testing/mozbase/mozrunner/mozrunner/runners.py#107-108

Comment 5

[No-Jun Park [:njpark]]

I added --avd-home parameter as well since it seemed to complain about it, but afterwards I get the failure as the following:
https://pastebin.mozilla.org/8891584

it looks as if it couldn't find the avd.  (but mozemulator-x86.avd is in the path that I specified)

Comment 6

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

I can't reproduce your error - the x86 emulator is launched successfully when I try this locally. Maybe there's something platform-specific going wrong? I'm trying this on OS X.

My command:
> ./mach marionette-test --emulator --app='fennec' --avd-home /Users/mozilla/.mozbuild/android-device/avd --emulator-binary /Users/mozilla/.mozbuild/android-sdk-macosx/tools/emulator --avd=mozemulator-x86 testing/marionette/harness/marionette/tests/unit/test_click.py

In case it helps, here's my mozconfig:
>mk_add_options AUTOCLOBBER=1
>ENABLE_MARIONETTE=1
># Build Firefox for Android Artifact Mode:
>ac_add_options --enable-application=mobile/android
>ac_add_options --target=i386-linux-android
>ac_add_options --enable-artifact-builds
># With the following Android SDK:
>ac_add_options --with-android-sdk="/Users/mozilla/.mozbuild/android-sdk-macosx"
># Write build artifacts to:
>mk_add_options MOZ_OBJDIR=./objdir-frontend

And my command history:
> ./mach clobber && ./mach build && ./mach package
> ./mach android-emulator # downloaded the x86 avd for me and launched emulator, which I closed
> ./mach install
> ./mach marionette-test ... # as shown above

You can also try launching the emulator yourself (./mach android-emulator) and then have Marionette Runner connect to that emulator and start Fennec in it: add `--address=localhost:2828` to the marionette-test options to try that.

Comment 7

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

Oh, I forgot to mention that my results in Comment 6 are using a fix from Bug 1294879 to make the --avd flag work properly.

However, the error in your pastebin looks like a different issue altogether, since it indicates that no emulator gets launched at all.

Comment 8

[No-Jun Park [:njpark]]

When I use your mozconfig, I get this error during the build: possibly because I am using a different sdk maybe? (just in case, my java version is "1.8.0_101"
https://pastebin.mozilla.org/8901301

So for me workaround was to not enable artifact build, and build the backend as well, as specified in the MDN. 

My mozconfig is:

mk_add_options AUTOCLOBBER=1
# Build Firefox for Android Artifact Mode:
ac_add_options --enable-application=mobile/android
ac_add_options --target=i386-linux-android
ac_add_options --enable-artifact-builds
# With the following Android SDK:
ac_add_options --with-android-sdk="/Users/mozilla/Library/Android/sdk"
# Write build artifacts to:
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-android_x86

ENABLE_MARIONETTE=1

and the mozconfig that I used to build the backend is:
# Build Firefox for Android:
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-android_x86
mk_add_options MOZ_MAKE_FLAGS="-s -j8"
ac_add_options --enable-debug
ac_add_options --enable-optimize
ac_add_options --with-ccache
ac_add_options --disable-crashreporter

# With the following Android SDK:
ac_add_options --with-android-sdk="/Users/mozilla/Library/Android/sdk"
ac_add_options --with-android-ndk="/Users/mozilla/android-ndk-r12b" 

## Native UI:
ac_add_options --enable-application=mobile/android
ac_add_options --target=i386-linux-android

Comment 9

[No-Jun Park [:njpark]]

Ah, turns out I didn't properly install the prereqs for the artifact build - your mozconfig works for me, and I can build with artifacts now, but while following your steps, ./mach android-emulator starts 4.3 instead of 4.2, and running your ./mach marionette-test command gives me https://pastebin.mozilla.org/8901643

Comment 10

[No-Jun Park [:njpark]]

Sorry, I forgot to say I'm on Mac OS X as well.

Comment 11

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

As far as I know, the intended behaviour of ./mach android-emulator is to either launch 4.3 or x86 -- that's what is stated in ./mach android-emulator -h. 

./mach marionette-test will also only launch 4.3 or x86, but it should be able to connect to any emulator that you have started yourself and launch the installed Fennec for you.

If ./mach marionette-test isn't able to even start a 4.3 emulator for you, maybe there's something unusual about your adnroid sdk installation... it might help to delete your .mozbuild and run ./mach bootstrap for Fennec again if you haven't done so in a while.

Also, does adb logcat yield anything interesting? errors from the emulator?

Comment 12

[No-Jun Park [:njpark]]

I wiped the .mozbuild directory and did config from the scratch including re-cloning the repo, and including ./mach bootstrap. now I get a slightly different error when I run your command:

https://pastebin.mozilla.org/8901944

And the process it cannot terminate is (emulator) which I don't understand, since it also says that it fails to locate the emulator.
71827 ttys000    0:00.99 python2.7 ./mach marionette-test --emulator --app=fenn
71842 ttys000    0:00.00 (emulator)

No errors I see on adb logcat though.  
But since there is a workaround (I can always start the emulator via ./mach run beforehand) , I suppose this is a low-priority issue.

Comment 13

[Henrik Skupin [:whimboo][⌚️UTC+1]]

No-Jun, is this still a problem for you?

Comment 14

[No-Jun Park [:njpark]]

I haven't checked whether this is still the case recently, but this issue no longer affects me, and I'm good with this issue being closed.

Comment 15

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Ok, so I just tried myself and it looks like that it is still not working:

> % mach marionette-test --emulator --app fennec --avd-home ~/.mozbuild/android-device/avd
>  0:00.00 INFO Using workspace for temporary data: "/Volumes/data/code/gecko"
>  0:00.05 mozdevice INFO Detected adb 1.0.39
> ^C 0:49.17 mozdevice ERROR Non-zero return code (1) from ['/Volumes/data/Users/henrik/.mozbuild/android-sdk-macosx/platform-tools/adb', 'shell', 'echo']
>  0:49.17 mozdevice ERROR Output: ['error: no devices/emulators found']
> mach interrupted by signal or user action. Stopping.

It hangs after mozdevice prints it's INFO log line.

Comment 16

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Fennec is close to be desupported, and we won't update the harness to support GeckoView embedding applications. As such this isn't going to happen. Just start the emulator on your own.