Closed Bug 1435433 Opened 7 years ago Closed 7 years ago

With runByManifest, Android mochitests fail: "No tests run or summary not found"

Categories

(Testing :: Mochitest, defect)

Product:
Testing
Component:
Mochitest
Version:
Version 3
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox60 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla60
Tracking Flags:
Tracking Status
firefox60 --- fixed

People

(Reporter: gbrown, Assigned: gbrown)

References

Details

Attachments

(1 file)

pass pass/fail/todo counts back to mochitest from remoteautomation
3.44 KB, patch
jmaher
: review+
Details | Diff | Splinter Review
Tests run, by manifest, and an appropriate summary is displayed after each manifest. Then there is an additional summary with 0/0/0 passed/failed/todo, and I think that must trigger "No tests run" in the mozharness output parser. https://treeherder.mozilla.org/logviewer.html#?job_id=160111029&repo=try&lineNumber=2676 [task 2018-02-02T17:57:24.854Z] 17:57:24 INFO - 449 INFO TEST-START | Shutdown [task 2018-02-02T17:57:24.855Z] 17:57:24 INFO - 450 INFO Passed: 474 [task 2018-02-02T17:57:24.856Z] 17:57:24 INFO - 451 INFO Failed: 0 [task 2018-02-02T17:57:24.856Z] 17:57:24 INFO - 452 INFO Todo: 2 [task 2018-02-02T17:57:24.856Z] 17:57:24 INFO - 453 INFO Mode: non-e10s [task 2018-02-02T17:57:24.857Z] 17:57:24 INFO - 454 INFO Slowest: 113723ms - /tests/dom/base/test/test_anchor_area_referrer.html [task 2018-02-02T17:57:24.858Z] 17:57:24 INFO - 455 INFO SimpleTest FINISHED [task 2018-02-02T17:57:45.705Z] 17:57:45 INFO - INFO | automation.py | Application ran for: 0:08:53.509954 [task 2018-02-02T17:57:45.706Z] 17:57:45 INFO - INFO | zombiecheck | Reading PID log: /tmp/tmpZd73Nkpidlog [task 2018-02-02T17:57:46.712Z] 17:57:46 INFO - Stopping web server [task 2018-02-02T17:57:46.718Z] 17:57:46 INFO - Stopping web socket server [task 2018-02-02T17:57:46.738Z] 17:57:46 INFO - Stopping ssltunnel [task 2018-02-02T17:57:46.759Z] 17:57:46 INFO - leakcheck | refcount logging is off, so leaks can't be detected! [task 2018-02-02T17:57:46.759Z] 17:57:46 INFO - runtests.py | Running tests: end. [task 2018-02-02T17:57:47.739Z] 17:57:47 INFO - Buffered messages finished [task 2018-02-02T17:57:47.739Z] 17:57:47 INFO - 0 INFO TEST-START | Shutdown [task 2018-02-02T17:57:47.740Z] 17:57:47 INFO - 1 INFO Passed: 0 [task 2018-02-02T17:57:47.740Z] 17:57:47 INFO - 2 INFO Failed: 0 [task 2018-02-02T17:57:47.740Z] 17:57:47 INFO - 3 INFO Todo: 0 [task 2018-02-02T17:57:47.741Z] 17:57:47 INFO - 4 INFO Mode: non-e10s [task 2018-02-02T17:57:47.741Z] 17:57:47 INFO - 5 INFO SimpleTest FINISHED ... [task 2018-02-02T17:57:48.546Z] 17:57:48 ERROR - Return code: 1 [task 2018-02-02T17:57:48.547Z] 17:57:48 ERROR - No tests run or test summary not found
Linux mochitests have the same final/master summary, but the counts are non-zero: task 2018-02-02T22:48:04.673Z] 22:48:04 INFO - TEST-START | Shutdown [task 2018-02-02T22:48:04.673Z] 22:48:04 INFO - Passed: 2 [task 2018-02-02T22:48:04.673Z] 22:48:04 INFO - Failed: 0 [task 2018-02-02T22:48:04.674Z] 22:48:04 INFO - Todo: 0 [task 2018-02-02T22:48:04.674Z] 22:48:04 INFO - Mode: e10s [task 2018-02-02T22:48:04.674Z] 22:48:04 INFO - Slowest: 1445ms - /tests/dom/credentialmanagement/tests/mochitest/test_credman_iframes.html [task 2018-02-02T22:48:04.675Z] 22:48:04 INFO - SimpleTest FINISHED [task 2018-02-02T22:48:04.685Z] 22:48:04 INFO - TEST-INFO | Ran 1 Loops [task 2018-02-02T22:48:04.686Z] 22:48:04 INFO - SimpleTest FINISHED [task 2018-02-02T22:48:04.875Z] 22:48:04 INFO - GECKO(5802) | *** UTM:SVC TimerManager:registerTimer called after profile-before-change notification. Ignoring timer registration for id: telemetry_modules_ping [task 2018-02-02T22:48:04.934Z] 22:48:04 INFO - GECKO(5802) | 1517611684922 Marionette DEBUG Received observer notification xpcom-shutdown [task 2018-02-02T22:48:04.936Z] 22:48:04 INFO - GECKO(5802) | 1517611684922 Marionette DEBUG New connections will no longer be accepted [task 2018-02-02T22:48:05.260Z] 22:48:05 INFO - TEST-INFO | Main app process: exit 0 [task 2018-02-02T22:48:05.261Z] 22:48:05 INFO - runtests.py | Application ran for: 0:00:05.879697 [task 2018-02-02T22:48:05.261Z] 22:48:05 INFO - zombiecheck | Reading PID log: /tmp/tmpbLJE4qpidlog [task 2018-02-02T22:48:05.261Z] 22:48:05 INFO - ==> process 5802 launched child process 5824 [task 2018-02-02T22:48:05.261Z] 22:48:05 INFO - ==> process 5802 launched child process 5854 [task 2018-02-02T22:48:05.261Z] 22:48:05 INFO - ==> process 5802 launched child process 5910 [task 2018-02-02T22:48:05.261Z] 22:48:05 INFO - zombiecheck | Checking for orphan process with PID: 5824 [task 2018-02-02T22:48:05.261Z] 22:48:05 INFO - zombiecheck | Checking for orphan process with PID: 5910 [task 2018-02-02T22:48:05.261Z] 22:48:05 INFO - zombiecheck | Checking for orphan process with PID: 5854 [task 2018-02-02T22:48:05.261Z] 22:48:05 INFO - Stopping web server [task 2018-02-02T22:48:05.274Z] 22:48:05 INFO - Stopping web socket server [task 2018-02-02T22:48:05.311Z] 22:48:05 INFO - Stopping ssltunnel [task 2018-02-02T22:48:05.327Z] 22:48:05 WARNING - leakcheck | refcount logging is off, so leaks can't be detected! [task 2018-02-02T22:48:05.328Z] 22:48:05 INFO - runtests.py | Running tests: end. [task 2018-02-02T22:48:05.344Z] 22:48:05 INFO - Buffered messages finished [task 2018-02-02T22:48:05.345Z] 22:48:05 INFO - 0 INFO TEST-START | Shutdown [task 2018-02-02T22:48:05.345Z] 22:48:05 INFO - 1 INFO Passed: 67212 [task 2018-02-02T22:48:05.346Z] 22:48:05 INFO - 2 INFO Failed: 0 [task 2018-02-02T22:48:05.347Z] 22:48:05 INFO - 3 INFO Todo: 299 [task 2018-02-02T22:48:05.347Z] 22:48:05 INFO - 4 INFO Mode: e10s [task 2018-02-02T22:48:05.348Z] 22:48:05 INFO - 5 INFO SimpleTest FINISHED [task 2018-02-02T22:48:05.349Z] 22:48:05 INFO - Buffered messages finished [task 2018-02-02T22:48:05.349Z] 22:48:05 INFO - SUITE-END | took 886s [task 2018-02-02T22:48:05.419Z] 22:48:05 INFO - Return code: 0 [task 2018-02-02T22:48:05.420Z] 22:48:05 INFO - TinderboxPrint: mochitest-plain-chunked<br/>1040/0/0
Recall that the per-manifest summaries are printed from the mochitest js harness, at https://dxr.mozilla.org/mozilla-central/rev/bda9adefe73902685d6689a205e7114ae9df7f83/testing/mochitest/tests/SimpleTest/TestRunner.js#463-468 while the final summary is printed from the python harness, at https://dxr.mozilla.org/mozilla-central/rev/bda9adefe73902685d6689a205e7114ae9df7f83/testing/mochitest/runtests.py#2604-2609 Those final summary counts are updated by the OutputHandler https://dxr.mozilla.org/mozilla-central/rev/bda9adefe73902685d6689a205e7114ae9df7f83/testing/mochitest/runtests.py#2985-2990 and the OutputHandler is not used on Android (it is created in runtests.py runApp(), but runApp() is overridden in runtestsremote.py, which delegates output handling to remoteautomation.py).
We jump through some hoops here to get the Passed/Failed/Todo counts from the js harness back to the python mochitest harness, so that the final counts are correct (only matters when runByManifest is enabled). No harm done, without runByManifest: https://treeherder.mozilla.org/#/jobs?repo=try&revision=34605353ff640d7f08411305d2bbc77aac6cb1e2
Attachment #8948881 - Flags: review?(jmaher)
Attachment #8948881 - Flags: review?(jmaher) → review+
Pushed by gbrown@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/0af93cb1ff94 Print master test summary after android mochitests when runByManifest; r=jmaher
https://hg.mozilla.org/mozilla-central/rev/0af93cb1ff94
Status: NEW → RESOLVED
Closed: 7 years ago
status-firefox60: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla60
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: