967318 - ScriptTimeoutExceptions should be reported with the failure on the same line as the test name

Status: RESOLVED FIXED

(Remote Protocol :: Marionette, defect, P1)

Description

[Jonathan Griffin (:jgriffin)]

Marionette tests are currently reporting ScriptTimeoutExceptions in a way that makes them difficult to sheriff correctly.  They're reported like:

07:45:13    ERROR -      raise ScriptTimeoutException(message=message, status=status, stacktrace=stacktrace)
07:45:13    ERROR -  ScriptTimeoutException: ScriptTimeoutException: timed out
07:45:13     INFO -  TEST-UNEXPECTED-FAIL | test_outgoing_emergency_in_airplane_mode.js |
07:45:13     INFO -  ----------------------------------------------------------------------

(see https://tbpl.mozilla.org/php/getParsedLog.php?id=34012307&tree=Fx-Team#error2)

...but we'd like them to be reported like:

07:45:13    ERROR -      raise ScriptTimeoutException(message=message, status=status, stacktrace=stacktrace)
07:45:13    ERROR -  TEST-UNEXPECTED-FAIL | test_outgoing_emergency_in_airplane_mode.js |
ScriptTimeoutException: ScriptTimeoutException: timed out
07:45:13     INFO -  ----------------------------------------------------------------------

I.e., "ScriptTimeoutException" should be given as the reason for the failure in the TEST-UNEXPECTED-FAIL line.

We used to handle this better, see e.g., bug 837827.

Comment 1

[Ryan VanderMeulen [:RyanVM]]

This would make life much easier for TBPL starring.

Comment 2

[Andreas Tolfsen ❲:ato❳]

I'm distracted by another task right now, but for the record I think it would be useful to introduce a test case that ensured tbpl-friendly exception formatting so that this doesn't get repeated.

Comment 3

[David Burns :automatedtester]

pushed to try https://tbpl.mozilla.org/?tree=Try&rev=3be9ddc32a9d

Lets see if the results are better with intermittents

Comment 4

[David Burns :automatedtester]

Attachment: Put the error reason in the message that is reported on TBPL for easier starring

Comment 5

[David Burns :automatedtester]

protip, remove pdb https://tbpl.mozilla.org/?tree=Try&rev=d1055e8d9ba5

Comment 6

[Phil Ringnalda (:philor)]

Comment on attachment 8416554 [details] [diff] [review]
Put the error reason in the message that is reported on TBPL for easier starring

\o/

Comment 7

[David Burns :automatedtester]

landed in http://hg.mozilla.org/integration/mozilla-inbound/rev/6f0fbf786c5e

Comment 8

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/6f0fbf786c5e

Comment 9

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-b2g30_v1_4/rev/610edb2bc69f
https://hg.mozilla.org/releases/mozilla-b2g28_v1_3/rev/6d79000bf580