Closed Bug 1529960 Opened 10 months ago Closed 10 months ago

[mozdevice] pidof can be flaky

Categories

(Testing :: Mozbase, defect)

Version 3
defect
Not set

Tracking

(firefox67 fixed)

RESOLVED FIXED
mozilla67
Tracking Status
firefox67 --- fixed

People

(Reporter: gbrown, Assigned: gbrown)

References

(Blocks 1 open bug)

Details

Attachments

(1 file, 1 obsolete file)

Marionette tests make frequent calls to process_exist() to verify that the test app (fennec) is still running. On the Android 7.0 x86_64 emulator, pidof is available and used by process_exist, but sometimes returns incorrect results: Fennec is still running, but pidof reports no pids.

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=229945549&repo=try&lineNumber=1973

[task 2019-02-22T17:52:16.656Z] 17:52:16 INFO - ZZZ _pidof pidof found no pids matching org.mozilla.fennec_aurora!!
[task 2019-02-22T17:52:16.656Z] 17:52:16 INFO - ZZZ pid_output=
[task 2019-02-22T17:52:16.656Z] 17:52:16 INFO - ZZZ mozrunner returncode finds org.mozilla.fennec_aurora does not exist!

but the fennec process has not died:

https://taskcluster-artifacts.net/AkE14ww-Tt-U2q-Ru8HeuA/0/public/test_info//logcat-emulator-5554.log

02-22 17:51:42.460 1335 1349 I ActivityManager: START u0 {act=android.intent.action.VIEW flg=0x10000000 cmp=org.mozilla.fennec_aurora/org.mozilla.gecko.BrowserApp (has extras)} from uid 0 on display 0

02-22 17:51:42.540 2423 2423 I GeckoApplication: zerdatime 69380 - application start
...
02-22 17:58:34.480 2423 2423 D GeckoBrowserApp: BrowserApp.onTabChanged: 54: STOP

Think it would be worth while to rig up a test to run on android-{em,hw} to see how well pidof works on our emulators and hardware for Android 7 and 8 to be sure we don't have other cases where it is flaky? Then we can blacklist the offending versions?

Here's a process_exist stress test masquerading as mochitest:

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=c7691c7267c0cb34bde716a5624faa6faf5b7871

The stress test turns mochitest orange consistently -- check the logs for actual failures.

I don't see any actual failures on Android 4.3 - it doesn't use pidof anyway.

I don't see any actual failures on android-hw-p2 either, which does use pidof.

There are failures on Android 7.0 x86_64, but notice that they are only on one or two iterations

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=229987275&repo=try&lineNumber=1353

[task 2019-02-22T22:21:56.234Z] 22:21:56 INFO - hammering away at process_exist...
[task 2019-02-22T22:23:54.495Z] 22:23:54 INFO - 0 ERROR TEST-UNEXPECTED-FAIL process_exist(org.mozilla.geckoview.test) failed on iteration 1140
[task 2019-02-22T22:30:55.460Z] 22:30:55 INFO - 1 ERROR TEST-UNEXPECTED-FAIL process_exist(org.mozilla.geckoview.test) failed on iteration 5151
[task 2019-02-22T22:32:22.236Z] 22:32:22 INFO - done!

So I think there's hope for a simple retry.

Assignee: nobody → gbrown

On the Android 7.0 x86_64 emulator, pidof occasionally returns
no results for a running process. To guard against this case, mozdevice
retries exactly once.

On the Android 7.0 x86_64 emulator, pidof occasionally returns
no results for a running process. To guard against this case, mozdevice
retries exactly once.

Attachment #9046089 - Attachment is obsolete: true
Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/c3c0216fbb61
[mozdevice] Retry pidof if no output; r=bc
Duplicate of this bug: 1527963
Status: NEW → RESOLVED
Closed: 10 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla67
See Also: → 1535932
See Also: → 1540648
You need to log in before you can comment on or make changes to this bug.