ScriptTimeoutExceptions should be reported with the failure on the same line as the test name

RESOLVED FIXED in Firefox 32, Firefox OS v1.3

Status

P1
normal
RESOLVED FIXED
5 years ago
4 years ago

People

(Reporter: jgriffin, Assigned: automatedtester)

Tracking

(Blocks: 1 bug, {pi-marionette-runner, sheriffing-P1})

unspecified
mozilla32
pi-marionette-runner, sheriffing-P1
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox30 wontfix, firefox31 wontfix, firefox32 fixed, b2g-v1.3 fixed, b2g-v1.3T fixed, b2g-v1.4 fixed, b2g-v2.0 fixed)

Details

(Whiteboard: [runner])

Attachments

(1 attachment)

(Reporter)

Description

5 years ago
Marionette tests are currently reporting ScriptTimeoutExceptions in a way that makes them difficult to sheriff correctly.  They're reported like:

07:45:13    ERROR -      raise ScriptTimeoutException(message=message, status=status, stacktrace=stacktrace)
07:45:13    ERROR -  ScriptTimeoutException: ScriptTimeoutException: timed out
07:45:13     INFO -  TEST-UNEXPECTED-FAIL | test_outgoing_emergency_in_airplane_mode.js |
07:45:13     INFO -  ----------------------------------------------------------------------

(see https://tbpl.mozilla.org/php/getParsedLog.php?id=34012307&tree=Fx-Team#error2)

...but we'd like them to be reported like:

07:45:13    ERROR -      raise ScriptTimeoutException(message=message, status=status, stacktrace=stacktrace)
07:45:13    ERROR -  TEST-UNEXPECTED-FAIL | test_outgoing_emergency_in_airplane_mode.js |
ScriptTimeoutException: ScriptTimeoutException: timed out
07:45:13     INFO -  ----------------------------------------------------------------------

I.e., "ScriptTimeoutException" should be given as the reason for the failure in the TEST-UNEXPECTED-FAIL line.

We used to handle this better, see e.g., bug 837827.
This would make life much easier for TBPL starring.
Keywords: sheriffing-P1
I'm distracted by another task right now, but for the record I think it would be useful to introduce a test case that ensured tbpl-friendly exception formatting so that this doesn't get repeated.
(Assignee)

Updated

5 years ago
Whiteboard: [runner]

Updated

5 years ago
Blocks: 910614
(Assignee)

Updated

5 years ago
Keywords: ateam-marionette-runner
(Assignee)

Updated

5 years ago
Priority: -- → P1
(Assignee)

Updated

5 years ago
Assignee: nobody → dburns

Updated

5 years ago
Blocks: 948895
(Assignee)

Comment 3

5 years ago
pushed to try https://tbpl.mozilla.org/?tree=Try&rev=3be9ddc32a9d

Lets see if the results are better with intermittents
(Assignee)

Updated

5 years ago
Flags: needinfo?(dburns)
(Assignee)

Comment 4

5 years ago
Created attachment 8416554 [details] [diff] [review]
Put the error reason in the message that is reported on TBPL for easier starring
Comment on attachment 8416554 [details] [diff] [review]
Put the error reason in the message that is reported on TBPL for easier starring

\o/
Attachment #8416554 - Flags: review+
https://hg.mozilla.org/mozilla-central/rev/6f0fbf786c5e
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla32

Updated

5 years ago
Depends on: 1006511
Flags: needinfo?(ryanvm)

Updated

5 years ago
Blocks: 1017559

Updated

4 years ago
Blocks: 1020296
https://hg.mozilla.org/releases/mozilla-b2g30_v1_4/rev/610edb2bc69f
https://hg.mozilla.org/releases/mozilla-b2g28_v1_3/rev/6d79000bf580
status-b2g-v1.3: --- → fixed
status-b2g-v1.3T: --- → affected
status-b2g-v1.4: --- → fixed
status-b2g-v2.0: --- → fixed
status-firefox30: --- → wontfix
status-firefox31: --- → wontfix
status-firefox32: --- → fixed
Flags: needinfo?(ryanvm)
status-b2g-v1.3T: affected → fixed
You need to log in before you can comment on or make changes to this bug.