970673 - gaia-ui-test failure messages need the failure back at the end of the TEST-UNEXPECTED-FAILURE line

Status: RESOLVED DUPLICATE of bug 958024

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[Phil Ringnalda (:philor)]

Once upon a time, test failures like https://tbpl.mozilla.org/php/getParsedLog.php?id=34447794&tree=Mozilla-Inbound#error1 were logged as

TEST-UNEXPECTED-FAIL | test_clock_set_alarm.py test_clock_set_alarm.TestClockSetAlarm.test_clock_set_alarm | TimeoutException: Timed out after 5.06385302544 seconds

(with a little fuzz around what the actual timed out message was, but with it there at the end after the second pipe).

Now, they are

15:03:21    ERROR -  TimeoutException: TimeoutException: Timed out after 5.06385302544 seconds
15:03:21     INFO -  TEST-UNEXPECTED-FAIL | test_clock_set_alarm.py test_clock_set_alarm.TestClockSetAlarm.test_clock_set_alarm |

which presents two problems for tbpl - because there's nothing after the second pipe, tbpl does a whole-line search for "TEST-UNEXPECTED-FAIL | test_clock_set_alarm.py test_clock_set_alarm.TestClockSetAlarm.test_clock_set_alarm |" rather than searching for the "filename" test_clock_set_alarm.py test_clock_set_alarm.TestClockSetAlarm.test_clock_set_alarm, and because the timeout is separated from the failure line, we are vastly more likely to misstar some future other failure that does something other than timeout as a bug about the timeout.

Also: TimeoutException: TimeoutException: Timed out? Really? ;)

Comment 1

[Zac C (:zac)]

Yes up until a week ago we used to raise our own Exceptions but now they come from a Marionette support class and it doesn't really know what it's doing aside from just waiting for something to be true.

We can pass in a custom message into each wait, it would just say something like "Element not found after timeout", but even that is vague. You always need the whole stack for context.

I'll ni? ato to see if he can help out here.