Closed Bug 1486908 Opened 3 years ago Closed 3 years ago

./mach raptor-test hangs if geckoview_example isn't installed


(Testing :: Raptor, defect)

Not set


(firefox63 fixed)

Tracking Status
firefox63 --- fixed


(Reporter: bc, Assigned: bc)




(2 files)

When I try to run raptor via mach locally without installing geckoview_example first, I get

$ ./mach raptor-test --test raptor-speedometer --app geckoview --binary org.mozilla.geckoview_example 
12:37:56     INFO -  adb Ignoring attempt to chmod external storage
12:37:56     INFO -  raptor-main starting geckoview
12:37:56     INFO -  adb launch_application: am start -W -n org.mozilla.geckoview_example/org.mozilla.geckoview_example.GeckoViewActivity -a android.intent.action.Main --ez use_multiprocess False --es args '-profile /sdcard/raptor-profile --es env0 LOG_VERBOSE=1 --es env1 R_LOG_LEVEL=6' -d about:blank
12:37:57     INFO -  adb shell_output: adb -s HT84D1A04087 wait-for-device shell am start -W -n org.mozilla.geckoview_example/org.mozilla.geckoview_example.GeckoViewActivity -a android.intent.action.Main --ez use_multiprocess False --es args '-profile /sdcard/raptor-profile --es env0 LOG_VERBOSE=1 --es env1 R_LOG_LEVEL=6' -d about:blank; echo rc=$?, timeout: None, root: False, timedout: None, exitcode: 0, output: Starting: Intent { act=android.intent.action.Main dat=about:blank cmp=org.mozilla.geckoview_example/.GeckoViewActivity (has extras) }
12:37:57     INFO -  Error type 3
12:37:57     INFO -  Error: Activity class {org.mozilla.geckoview_example/org.mozilla.geckoview_example.GeckoViewActivity} does not exist.

which just hangs there. If you wait 1800 seconds, it will time out and everything will be cleaned up.

If I ctrl-c I get

^CProcess Process-1:
12:40:27     INFO - Running post-action listener: _package_coverage_data
Traceback (most recent call last):
  File "/usr/lib64/python2.7/multiprocessing/", line 267, in _bootstrap
  File "/usr/lib64/python2.7/multiprocessing/", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/bclary/mozilla/builds/inbound-taskcluster/mozilla/fennec-opt-arm/testing/raptor-venv/lib/python2.7/site-packages/mozsystemmonitor/", line 90, in _collect
    while not pipe.poll(sleep_interval):
12:40:27     INFO - Running post-action listener: _resource_record_post_action
12:40:27     INFO - Running post-action listener: process_java_coverage_data
12:40:27     INFO - [mozharness: 2018-08-28 19:40:27.553235Z] Finished run-tests step (failed)
12:40:27     INFO - Running post-run listener: _resource_record_post_run
12:40:27  WARNING - Exception when reporting resource usage: Traceback (most recent call last):
12:40:27  WARNING -   File "/home/bclary/mozilla/builds/inbound-taskcluster/mozilla/testing/mozharness/mozharness/base/", line 544, in _resource_record_post_run
12:40:27  WARNING -     self._resource_monitor.stop()
12:40:27  WARNING -   File "/home/bclary/mozilla/builds/inbound-taskcluster/mozilla/fennec-opt-arm/testing/raptor-venv/lib/python2.7/site-packages/mozsystemmonitor/", line 285, in stop
12:40:27  WARNING -     self._pipe.send(('terminate',))
12:40:27  WARNING - IOError: [Errno 32] Broken pipe
12:40:27     INFO - Running post-run listener: copy_logs_to_upload_dir
12:40:27     INFO - Copying logs to upload dir...
mach interrupted by signal or user action. Stopping.

which leaves an orphaned process

bclary@ruby ~/mozilla/builds/inbound-taskcluster/mozilla (sisyphus-dev)
$ ps aux | grep
bclary   26898  0.1  0.1 474292 25068 pts/2    Sl   12:37   0:00 /home/bclary/mozilla/builds/inbound-taskcluster/mozilla/fennec-opt-arm/testing/raptor-venv/bin/python /home/bclary/mozilla/builds/inbound-taskcluster/mozilla/testing/raptor/raptor/ --run-local --test raptor-speedometer --app geckoview --binary org.mozilla.geckoview_example --app geckoview --obj-path /home/bclary/mozilla/builds/inbound-taskcluster/mozilla/fennec-opt-arm --log-tbpl-level=debug
bclary   27135  0.0  0.0 213800  1076 pts/2    S+   12:40   0:00 grep --color=auto

To get this far you must have first applied the patch in bug 1486892.

I've tried to figure this out but the multiprocess, multithreaded stuff related to resource monitoring and logging has me stumped. The easiest thing I can think of is to quickly fail.
Fail if app isn't installed or the launch fails.
Attachment #9004661 - Flags: review?(jmaher)
Attachment #9004661 - Flags: review?(jmaher) → review+
Duplicate of this bug: 1484230
Blocks: 1473078
In addition to the attached patch (which lgtm), consider adding more user-friendliness at the mach command level. Android mochitest/reftest/xpcshell/junit mach commands use code like:

before starting the test harness.
gbrown: Thanks. I'll look into that.

Another note about the hang. In bug 1485638 comment 2, I mentioned the problem with the installed version of mozsystemmonitor. I reversed this patch locally, uninstalled geckoview_example and using hacks to force using the in-tree version of mozsystemmonitor I still reproduced the hang but it is the same hang valentin saw in bug 1485638 at which sort of confirms the conclusion we also need a fix for that poll().
Depends on: 1485638
Follow up patch to prompt for installation using verify_android_device when running under mach.
Attachment #9005021 - Flags: review?(gbrown)
Comment on attachment 9005021 [details] [diff] [review]

Review of attachment 9005021 [details] [diff] [review]:

Awesome. Glad that worked out.
Attachment #9005021 - Flags: review?(gbrown) → review+
Good thing I did a try push first:

busted can not import psutil_linux

$ find . -name 'requirements*.txt' | xargs grep psutil

Including android_device broke mach. Adding psutil to raptor's requirements didn't help.
Flags: needinfo?(gbrown)
Pushed by
Raptor should terminate with an exception if the app isn't installed during run_test, r=jmaher.
/mach raptor-test should prompt to install geckoview_example if needed, r=gbrown.
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla63
Depends on: 1488544
Blocks: 1483695
You need to log in before you can comment on or make changes to this bug.