1599587 - Intermittent Android 7.0 reftest No tests run or test summary not found

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: Testing, defect, P1)

Description

[Geoff Brown [:gbrown]]

+++ This bug was initially created as a clone of Bug #1575266 +++

The android reftest harness still infrequently fails to complete its logging, even after the changes in bug 1575266.

Comment 1

[Geoff Brown [:gbrown]]

Also, bug 1575266 introduced this (which isn't causing any trouble afaik -- it is present in failure and success cases):

11-26 10:35:41.222  2786  2801 E GeckoConsole: [JavaScript Error: "[Exception... "Component returned failure code: 0x80470002 (NS_BASE_STREAM_CLOSED) [nsIConverterOutputStream.writeString]"  nsresult: "0x80470002 (NS_BASE_STREAM_CLOSED)"  location: "JS frame :: resource://reftest/reftest.jsm :: gDumpRawLog :: line 60"  data: no]"]
11-26 10:35:41.222  2786  2801 E GeckoConsole: gDumpRawLog@resource://reftest/reftest.jsm:60:15
11-26 10:35:41.222  2786  2801 E GeckoConsole: _logData@resource://reftest/StructuredLog.jsm:218:10
11-26 10:35:41.222  2786  2801 E GeckoConsole: log@resource://reftest/StructuredLog.jsm:171:10
11-26 10:35:41.222  2786  2801 E GeckoConsole: warning@resource://reftest/StructuredLog.jsm:183:10
11-26 10:35:41.222  2786  2801 E GeckoConsole: getTestPlugin@resource://reftest/reftest.jsm:150:10
11-26 10:35:41.222  2786  2801 E GeckoConsole: OnRefTestUnload@resource://reftest/reftest.jsm:554:30
11-26 10:35:41.222  2786  2801 E GeckoConsole: onShutdown@file:///storage/emulated/0/tests/reftest/profile/extensions/[e-mail]/api.js:116:7
11-26 10:35:41.222  2786  2801 E GeckoConsole: ExtensionAPI/<@resource://gre/modules/ExtensionCommon.jsm:356:14
11-26 10:35:41.222  2786  2801 E GeckoConsole: wrapper@resource://gre/modules/ExtensionCommon.jsm:300:14
11-26 10:35:41.222  2786  2801 E GeckoConsole: emit@resource://gre/modules/ExtensionCommon.jsm:327:32
11-26 10:35:41.222  2786  2801 E GeckoConsole: emit@resource://gre/modules/Extension.jsm:1790:25
11-26 10:35:41.222  2786  2801 E GeckoConsole: shutdown@resource://gre/modules/Extension.jsm:2437:10
11-26 10:35:41.222  2786  2801 E GeckoConsole: shutdown@resource://gre/modules/Extension.j

Comment 2

[Geoff Brown [:gbrown]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=6732bc81b0495a61e1d535e1606bf153baf87771 suggests that the issue is, again, flaky adb responses. If the is_file() test is flaky at:

https://searchfox.org/mozilla-central/rev/cce8b90aece0f42e5025e45282de16066eeaa662/build/mobile/remoteautomation.py#257

then there is a greater chance of the wait loop checking for process completion and exiting before reading all output.

Additional evidence for this vulnerability is that test log timestamps sometimes have "jumps" of 20 seconds (rather than 10) with no apparent reason.

Rather than retry the is_file (which appears effective from that try run), let's move it: Once the process has started and the log file has been found, there's no good reason to keep checking for its existence.

Comment 3

[Geoff Brown [:gbrown]]

Attachment: Bug 1599587 - Adjust android mochitest/reftest logging; r=

Guard against flaky results from the remote is_file and reduce adb traffic
by waiting once for the log file to be created rather than checking for it
throughout the test run.
(Minor ride-along change to reftest.jsm resolves javascript error introduced
by bug 1575266.)

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/36598eb92aa6
Adjust android mochitest/reftest logging; r=jmaher

Comment 5

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/36598eb92aa6