Closed Bug 719443 Opened 11 years ago Closed 11 years ago
create a robocop test type for 'android'
in bug 715215, we install the robocop.apk. Now we need to create a test type robocop to replace browser-chrome. This would require a command similar to browser-chrome, but without the "--browser-chrome --test-path=mobile" and instead a "--robocop=robocop.ini".
Hal, I know you're working on one part of this, so I'm opportunistically assigning this to you. Throw it back if that's wrong.
Assignee: nobody → hwine
(In reply to Joel Maher (:jmaher) from comment #0) > Now we need to create a test > type robocop to replace browser-chrome. > > This would require a command similar to browser-chrome, but without the > "--browser-chrome --test-path=mobile" and instead a "--robocop=robocop.ini". Joel - once bug 719697 get's resolved, please post the final exact command line that works, and I'll pick this up.
how close are we to landing this. I am testing a final patch for bug 719697, so I wanted to make sure we were ready to land this as soon as my patch lands.
This command gets the tests running: python runtestsremote.py --app=org.mozilla.fennec_jmaher --xre-path=/home/jmaher/mozilla/inbound/obj-i686-pc-linux-gnu/dist/bin/ --deviceIP=192.168.1.100 --robocop=robocop.ini
Here is some output: jmaher@jmaher-MacBookPro:~/Downloads/robo/mochitest$ python runtestsremote.py --app=org.mozilla.fennec --xre-path=/home/jmaher/mozilla/inbound/obj-i686-pc-linux-gnu/dist/bin/ --deviceIP=192.168.1.100 --robocop=robocop.ini unable to execute 'cp' on device; consider installing busybox from Android Market NOT running as root adbd cannot run as root in production builds reconnecting socket removing file: /sdcard/fennec_ids.txt removing file: /sdcard/robotium.config INFO | runtests.py | Installing extension at /home/jmaher/Downloads/robo/mochitest/extensions/specialpowers to /tmp/tmpyZsYvd. INFO | runtests.py | Installing extension at /home/jmaher/Downloads/robo/mochitest/extensions/workerbootstrap to /tmp/tmpyZsYvd. INFO | runtests.py | Installing extension at /home/jmaher/Downloads/robo/mochitest/extensions/worker to /tmp/tmpyZsYvd. INFO | runtests.py | Installing extension at /home/jmaher/Downloadsfirstname.lastname@example.org to /tmp/tmpyZsYvd. pushing directory: /tmp/tmpyZsYvd to /mnt/sdcard/tests/profile args: ['/home/jmaher/mozilla/inbound/obj-i686-pc-linux-gnu/dist/bin/xpcshell', '-g', '/home/jmaher/mozilla/inbound/obj-i686-pc-linux-gnu/dist/bin', '-v', '170', '-f', './httpd.js', '-e', "const _PROFILE_PATH = '/tmp/tmpL83YT4';const _SERVER_PORT = '8888'; const _SERVER_ADDR ='192.168.1.109';", '-f', './server.js'] INFO | runtests.py | Server pid: 25167 pushing directory: /tmp/tmpyZsYvd to /mnt/sdcard/tests/profile INFO | runtests.py | Running tests: start. FIRE PROC: '"MOZ_CRASHREPORTER=1,XPCOM_DEBUG_BREAK=stack,MOZ_CRASHREPORTER_NO_REPORT=1,NO_EM_RESTART=1,MOZ_PROCESS_LOG=/tmp/tmpToaiFopidlog,XPCOM_MEM_BLOAT_LOG=/tmp/tmpyZsYvd/runtests_leaks.log" am instrument -w -e class org.mozilla.fennec.tests.testAwesomebar org.mozilla.roboexample.test/android.test.InstrumentationTestRunner' INFO | automation.py | Application pid: 0 0 INFO SimpleTest START 1 INFO TEST-START | testAwesomebar 2 INFO TEST-PASS | testAwesomebar | Awesomebar URL typed properly - http://mochi.test:8888/tests/robocop/robocop_blank_01.html should equal http://mochi.test:8888/tests/robocop/robocop_blank_01.html 3 INFO TEST-PASS | testAwesomebar | Awesomebar URL stayed the same - http://mochi.test:8888/tests/robocop/robocop_blank_01.html should equal http://mochi.test:8888/tests/robocop/robocop_blank_01.html 4 INFO TEST-END | testAwesomebar | finished in 13923ms 5 INFO TEST-START | Shutdown 6 INFO Passed: 2 7 INFO Failed: 0 8 INFO Todo: 0 9 INFO SimpleTest FINISHED INFO | automation.py | Application ran for: 0:01:05.333367 INFO | automation.py | Reading PID log: /tmp/tmpToaiFopidlog getting files in '/mnt/sdcard/tests/profile/minidumps/' WARNING | automationutils.processLeakLog() | refcount logging is off, so leaks can't be detected! INFO | runtests.py | Running tests: end.
I have seen output from this test type in android staging. Are we done with this, Are we waiting on the signing issue in bug 719697 to be verified and closed out?
(In reply to Joel Maher (:jmaher) from comment #6) > I have seen output from this test type in android staging. Are we done with > this, Are we waiting on the signing issue in bug 719697 to be verified and > closed out? correct
does clean config and runs on tegra okay. Naming is set to trigger the correct download behavior in the steps, per bug 715215
Comment on attachment 594365 [details] [diff] [review] add robocop aware test tested in dev with tegra for full set of android tests. Works fine there. r+ from aki & catlee in irc supporting code committed with bug 715215
This is live now as of 1300 PST.
I see the tests running: https://tbpl.mozilla.org/?tree=Mozilla-Inbound&jobname=robocop Unfortunately I believe the exit codes are not working properly. I see : > 8 INFO TEST-UNEXPECTED-FAIL | test_bug720538 | Checking bottom-left corner of viewport - Color rgba(255,255,255,255) not close enough to expected rgb(0,128,0) but the exit code is success. Is this expected?  https://tbpl.mozilla.org/php/getParsedLog.php?id=9158753&tree=Mozilla-Inbound&full=1#error0
we can fix this in robocop and return the proper error code. It isn't expected for the test to fail, but the tool worked as it should.
Also, if there are branches that do not have robocop builds, but do execute this (modified) android suite. Those branches will fail red/orange, and we should just hide that column in tbpl until the robocop changes land on the branch. (We don't expect this to happen, I just want to document the workaround if needed.)
Is this bug then FIXED? Should now the focus go to bug 723667? (In reply to Joel Maher (:jmaher) from comment #13) > we can fix this in robocop and return the proper error code. It isn't > expected for the test to fail, but the tool worked as it should. Do you have a Talos bug for this I can follow?
Based on Armen's comment that this test is running, and Joel's comment that this is a robocop issue (not config), marking this fixed.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
the bug to track the log stuff is bug 717018
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.