1412766 - Execute robocop test with Expected results: 0

Status: RESOLVED INCOMPLETE

(Firefox for Android Graveyard :: Testing, defect, P5)

Description

[Nevin Chen(Not active on Bugzilla)]

After reading https://wiki.mozilla.org/Mobile/Fennec/Android/Testing#robocop_UI_tests and https://wiki.mozilla.org/Mobile/Fennec/Android/UITest
I run ./mach cobocop testHomeListsProvider
on API 16 and 25 emulator

But it looks like the test is run with no expected result and it seems wrong.
Please help!



 0:00.29 LOG: MainThread INFO Checking for ssltunnel processes...
 0:00.30 LOG: MainThread INFO Checking for xpcshell processes...
Error deleting /data/anr/traces.txt
 0:00.93 LOG: MainThread INFO Android sdk version '16'; will use this to filter manifests
pk12util: PKCS12 IMPORT SUCCESSFUL
 0:02.83 LOG: MainThread INFO MochitestServer : launching [u'/Users/nechen/.mozbuild/android-device/host-utils-50.0a2.en-US.mac/xpcshell', '-g', '/Users/nechen/.mozbuild/android-device/host-utils-50.0a2.en-US.mac', '-v', '170', '-f', '/Users/nechen/.mozbuild/android-device/host-utils-50.0a2.en-US.mac/components/httpd.js', '-e', "const _PROFILE_PATH = '/var/folders/ht/cjhx2qmx2x31mwys6kh_6bb00000gn/T/tmpCgKKP7.mozrunner'; const _SERVER_PORT = '8888'; const _SERVER_ADDR = '10.247.29.190'; const _TEST_PREFIX = undefined; const _DISPLAY_RESULTS = false;", '-f', '/Users/nechen/Desktop/mozilla-central/objdir-frontend/_tests/testing/mochitest/server.js']
 0:02.83 LOG: MainThread INFO runtests.py | Server pid: 55908
 0:02.84 LOG: MainThread INFO runtests.py | Websocket server pid: 55909
 0:02.85 LOG: MainThread INFO runtests.py | SSL tunnel pid: 55910
 0:02.96 SUITE_START: MainThread 1
 0:03.19 LOG: MainThread INFO Unable to copy remote profile; falling back to push.
INFO | automation.py | Application pid: 0
-1:59.92 LOG: None INFO SimpleTest START
-1:59.92 TEST_START: None testHomeListsProvider
INFO | automation.py | Application ran for: 0:00:22.447079
INFO | zombiecheck | Reading PID log: /var/folders/ht/cjhx2qmx2x31mwys6kh_6bb00000gn/T/tmplNNYnlpidlog
/data/anr/traces.txt not found
MOZ_UPLOAD_DIR not defined; tombstone check skipped
 0:26.41 LOG: MainThread INFO PROCESS-CRASH | Automation Error: Missing end of test marker (process crashed?)
INFO | runtests.py | Test summary: start.
0 INFO TEST-START | Shutdown
1 INFO Passed: 0
2 INFO Failed: 0
3 INFO Todo: 0
4 INFO SimpleTest FINISHED
INFO | runtests.py | Test summary: end.
 0:26.41 LOG: MainThread INFO Stopping web server
 0:26.42 LOG: MainThread INFO Stopping web socket server
 0:26.44 LOG: MainThread INFO Stopping ssltunnel
 0:27.31 LOG: MainThread INFO Buffered messages finished
 0:27.31 SUITE_END: MainThread 
Summary
=======

Ran 1 tests
Expected results: 0
Unexpected results: 0

OK

Comment 1

[Nevin Chen(Not active on Bugzilla)]

Max ponited out that if I use ./mach install to install the main apk instead of using Android Studio, it will work.
Maybe we should find out why and add this in https://wiki.mozilla.org/Auto-tools/Projects/Robocop#Frequently_found_errors

Comment 2

[Geoff Brown [:gbrown]]

Sorry this was confusing, but there are already some safeguards in place.

https://wiki.mozilla.org/Mobile/Fennec/Android/Testing#robocop_UI_tests says:

  To run robocop tests, first build and install Firefox for Android,

    mach build
    mach package
    mach install


Also, when I run robocop without Firefox installed, I see this:

./mach robocop testHomeListsProvider
It looks like Firefox is not installed on this device.
Install Firefox? (Y/n) 


Was your experience different?