[jittest] Improve ADBProcessError messages in run_test_remote

RESOLVED FIXED in Firefox 67

Status

()

enhancement
RESOLVED FIXED
5 months ago
4 months ago

People

(Reporter: bc, Assigned: bc)

Tracking

65 Branch
mozilla67
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox67 fixed)

Details

(Whiteboard: [cia:logs:2019])

Attachments

(1 attachment)

Assignee

Description

5 months ago

Bug 1518628 and Bug 1518719 are examples of Android jittest failures which are hard for Sheriffs to classify due to the limited and somewhat misleading error message which is produced in run_test_remote.

In one instance we have the device serial number in the "not found" message and test failure line which causes Treeherder to fail to provide suggested bugs in the Failure Summary.

In the other, we have an "error: closed" message which is not helpful.

This can be improved.

Assignee

Comment 1

4 months ago

Try run with 100 iterations:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=4ad2e7b6fcb47812e302e0763e118f81b3b72b20

Things are complicated by the fact that the js-style shell tests depend on non-zero exit codes.

After several iterations where I attempted to prevent the log parser from inadvertently marking the ADBProcessError text from being marked as an error, I realized that check_output had the logic to determine if a non-zero return code from the shell process was an error or not. One thing led to another and I believe that we shouldn't output the exception output at all in the except: block but should instead defer to the remainder of the output logic to determine if the exception really was an error.

Attachment #9041073 - Flags: review?(gbrown)
Comment on attachment 9041073 [details] [diff] [review]
bug-1524352-adbprocess-error-handling.patch

Review of attachment 9041073 [details] [diff] [review]:
-----------------------------------------------------------------

That seems right to me.
Attachment #9041073 - Flags: review?(gbrown) → review+

Comment 3

4 months ago
Pushed by bclary@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/8a8810509cdb
[jittest] Improve ADBProcessError messages in run_test_remote, r=gbrown.

Comment 4

4 months ago
bugherder
Status: ASSIGNED → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla67
Assignee

Updated

4 months ago
See Also: → 1525288
Assignee

Updated

4 months ago
Depends on: 1517381
Assignee

Updated

4 months ago
See Also: → 1518650
You need to log in before you can comment on or make changes to this bug.