Closed Bug 774322 Opened 7 years ago Closed 7 years ago

Long-running robocop tests run over sut may fail: waitForFinish does not wait

Categories

(Testing :: General, defect)

x86
Android
defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED
mozilla17

People

(Reporter: gbrown, Assigned: gbrown)

References

Details

Attachments

(1 file)

remoteautomation.waitForFinish waits, up to a specified timeout, for a process to finish. It does this via a call to proc.wait, where proc is a remoteautomation.RProcess. RProcess.wait uses devicemanager to obtain a list of process names from the device and then searches the list for the RProcess' procname -- if the name is not in the list of running processes, the wait finishes.

Unfortunately, the procname associated with the RProcess is "am" -- from the "am instrument -w ..." command line used to launch robocop tests. "am" does not appear in the list of processes -- remoteautomation should actually wait for something like "org.mozilla.fennec" to complete.


This only seems to be a problem over SUT. When using devicemanagerADB, dmADB.launchProcess waits indefinitely - or for a very long time - for the process to finish, virtually ensuring that the process has completed before remoteautomation.waitForFinish is called.


When waitForFinish does not wait for completion of the test, test output may be lost (not reported), resulting in test failure. A try run here shows that long-running robocop tests often fail: https://tbpl.mozilla.org/?tree=Try&rev=f8bcccdc5e5a

I don't fully understand how they can sometimes succeed.
Blocks: 771862
An obvious hack, but it gets the job done!

This examines an RProcess command line like 'am instrument -w -e class org.mozilla.fennec.tests.testAwesomeBar org.mozilla.roboexample.test/android.test.InstrumentationTestRunner' and sorts out that the process name to expect from the process list is 'org.mozilla.fennec'.
Attachment #642931 - Flags: review?(jmaher)
Normal try run for patch: https://tbpl.mozilla.org/?tree=Try&rev=3d5a71ac18a1

Try run for patch + long-running robocop tests: https://tbpl.mozilla.org/?tree=Try&rev=b91e116b9d8f (robocop failures here are caused by unfortunate placement of the sleep() calls -- even so, the tests run to completion).
Comment on attachment 642931 [details] [diff] [review]
use correct process name for fennec in RProcess

Review of attachment 642931 [details] [diff] [review]:
-----------------------------------------------------------------

agree on the hack here, but this gets us running.  iirc, fireproc doesn't return until robocop is finished.  This is different than launching fennec normally.
Attachment #642931 - Flags: review?(jmaher) → review+
https://hg.mozilla.org/mozilla-central/rev/64ca6f0f5c9f
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla17
You need to log in before you can comment on or make changes to this bug.