1425010 - Intermittent Android Test-verify No tests run or test summary not found - test/unit/test_storage_connection.js

Status: RESOLVED FIXED

(Testing :: General, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: csabou [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=151452736&repo=autoland

https://queue.taskcluster.net/v1/task/OsLuTu1cQO-KLlQDviDWJQ/runs/0/artifacts/public/logs/live_backing.log

[task 2017-12-13T12:51:49.902Z] 12:51:49     INFO -  :::
[task 2017-12-13T12:51:49.903Z] 12:51:49     INFO -  ::: Test verification summary for: /builds/worker/workspace/build/tests/xpcshell/tests/storage/test/unit/test_storage_connection.js
[task 2017-12-13T12:51:49.904Z] 12:51:49     INFO -  :::
[task 2017-12-13T12:51:49.905Z] 12:51:49     INFO -  ::: 1. Run each test 10 times, sequentially. : Pass
[task 2017-12-13T12:51:49.906Z] 12:51:49     INFO -  ::: 2. Run each test 10 times, sequentially, in chaos mode. : Pass
[task 2017-12-13T12:51:49.907Z] 12:51:49     INFO -  :::
[task 2017-12-13T12:51:49.908Z] 12:51:49     INFO -  ::: Test verification PASSED
[task 2017-12-13T12:51:49.909Z] 12:51:49     INFO -  :::
[task 2017-12-13T12:51:49.922Z] 12:51:49     INFO - Return code: 0
[task 2017-12-13T12:51:49.923Z] 12:51:49    ERROR - No tests run or test summary not found
[task 2017-12-13T12:51:49.923Z] 12:51:49     INFO - TinderboxPrint: xpcshell<br/><em class="testfail">T-FAIL</em>
[task 2017-12-13T12:51:49.924Z] 12:51:49     INFO - ##### xpcshell log ends
[task 2017-12-13T12:51:49.924Z] 12:51:49  WARNING - # TBPL WARNING #
[task 2017-12-13T12:51:49.925Z] 12:51:49  WARNING - setting return code to 1
[task 2017-12-13T12:51:49.925Z] 12:51:49  WARNING - TinderboxPrint: Verification of .../unit/test_storage_connection.js<br/>: WARNING

Comment 1

[Geoff Brown [:gbrown]]

Bug 1423820 modified this test, triggering verification. Verification passed on all platforms except android.

This test is annotated:

fail-if = os == "android"

Comment 2

[Intermittent Failures Robot]

3 failures in 423 pushes (0.007 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 3

Platform breakdown:
* android-4-3-armv7-api16: 2
* android-api-16-gradle: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1425010&startday=2017-12-11&endday=2017-12-17&tree=all

Comment 3

[Geoff Brown [:gbrown]]

This happens at

https://dxr.mozilla.org/mozilla-central/rev/1624b88874765bf57e9feba176d30149c748d9d2/testing/mozharness/mozharness/mozilla/testing/unittest.py#189-191

I reproduced with additional logging on try.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=cac14a883c953fd9e69fa8d4c2b153219ee18020

As expected, pass_count and fail_count are 0, but known_fail_count is None -- interesting.

Comment 4

[Geoff Brown [:gbrown]]

Attachment: update mozharness error regex for xpcshell

All is well if we just make mozharness aware of the Todo: summary logging.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=e73d17b7688cd113a240ca52b6df635872f127e3

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8938447 [details] [diff] [review]
update mozharness error regex for xpcshell

Review of attachment 8938447 [details] [diff] [review]:
-----------------------------------------------------------------

this gets better every day

Comment 6

[Geoff Brown [:gbrown]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=d70bedb0355e70aeb7cc1f0db2e0d2448f13f9d0

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/07d3581a0230
Update mozharness error regex for xpcshell known failures; r=jmaher

Comment 8

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/07d3581a0230