Closed Bug 774322 Opened 7 years ago Closed 7 years ago
Long-running robocop tests run over sut may fail: wait
For Finish does not wait
remoteautomation.waitForFinish waits, up to a specified timeout, for a process to finish. It does this via a call to proc.wait, where proc is a remoteautomation.RProcess. RProcess.wait uses devicemanager to obtain a list of process names from the device and then searches the list for the RProcess' procname -- if the name is not in the list of running processes, the wait finishes. Unfortunately, the procname associated with the RProcess is "am" -- from the "am instrument -w ..." command line used to launch robocop tests. "am" does not appear in the list of processes -- remoteautomation should actually wait for something like "org.mozilla.fennec" to complete. This only seems to be a problem over SUT. When using devicemanagerADB, dmADB.launchProcess waits indefinitely - or for a very long time - for the process to finish, virtually ensuring that the process has completed before remoteautomation.waitForFinish is called. When waitForFinish does not wait for completion of the test, test output may be lost (not reported), resulting in test failure. A try run here shows that long-running robocop tests often fail: https://tbpl.mozilla.org/?tree=Try&rev=f8bcccdc5e5a I don't fully understand how they can sometimes succeed.
An obvious hack, but it gets the job done! This examines an RProcess command line like 'am instrument -w -e class org.mozilla.fennec.tests.testAwesomeBar org.mozilla.roboexample.test/android.test.InstrumentationTestRunner' and sorts out that the process name to expect from the process list is 'org.mozilla.fennec'.
Attachment #642931 - Flags: review?(jmaher)
Normal try run for patch: https://tbpl.mozilla.org/?tree=Try&rev=3d5a71ac18a1 Try run for patch + long-running robocop tests: https://tbpl.mozilla.org/?tree=Try&rev=b91e116b9d8f (robocop failures here are caused by unfortunate placement of the sleep() calls -- even so, the tests run to completion).
Comment on attachment 642931 [details] [diff] [review] use correct process name for fennec in RProcess Review of attachment 642931 [details] [diff] [review]: ----------------------------------------------------------------- agree on the hack here, but this gets us running. iirc, fireproc doesn't return until robocop is finished. This is different than launching fennec normally.
Attachment #642931 - Flags: review?(jmaher) → review+
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla17
You need to log in before you can comment on or make changes to this bug.