790613 - Make sut_tools/verify.py output failures in the format "Automation Error: " so TBPL's parser recognises them

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Ed Morley [:emorley]]

(Bah, I should have done this as part of bug 787019)

Failures of the type bug 781419 require the log to be opened, since TBPL doesn't recognise the error string and so doesn't show it in the annotated summary box.

We can either change the string to match one of the ones at:
http://hg.mozilla.org/users/mstange_themasta.com/tinderboxpushlog/file/b3f9daa143ce/php/inc/GeneralErrorFilter.php#l24

Or add our own, after changing from 'WARNING: ' since that's too generic & will match against other parts of the log.

I suggest changing:
WARNING: Unable to ping tegra after 5 attempts
to (the already recognised):
Automation Error: Unable to ping tegra after 5 attempts

(Whilst we're at it, we might as well change other error strings in verify.py at the same time).

Comment 1

[Ed Morley [:emorley]]

installApp.py uses a lot of:
"Remote Device Error: foo"
...perhaps we should add that to the parser in addition to "Automation Error: " ?

Comment 2

[Ed Morley [:emorley]]

Attachment: Patch v1

Prefixes a number of failures with "Automation Error:" or "Remote Device Error:" so TBPL's parser can recognise them.

The "Remote Device Error:" case will require a TBPL change, I'll file that now.

Had to make best guess as to which errors to put under which category, tweaks welcome :-)

Comment 3

[Justin Wood (:Callek)]

Comment on attachment 660415 [details] [diff] [review]
Patch v1

From a releng perspective just changing these strings are fine.

Add'l r? to jmaher since he has code that this will bitrot, and for the determination of which strings are correct.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 660415 [details] [diff] [review]
Patch v1

Review of attachment 660415 [details] [diff] [review]:
-----------------------------------------------------------------

r- because I am concerned about the error for all cases when we are not installing robocop.apk.

This is from a jsreftest log on m-c:
return code [0]
$>
WARNING (robocop): zip file not as expected: There is no item named 'bin/robocop.apk' in the archive (("There is no item named 'bin/robocop.apk' in the archive",))
WARNING (robocop): robocop.apk will not be installed
/builds/sut_tools/installApp.py:84: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
  print "WARNING (robocop): zip file not as expected: %s (%s)" % (e.message,
program finished with exit code 0
elapsedTime=122.863866
========= Finished Install App on Device (results: 0, elapsed: 2 mins, 2 secs) (at 2012-09-12 14:32:14.329316) =========

::: sut_tools/installApp.py
@@ +85,2 @@
>                                                                          str(e.args))
> +                print "Remote Device Error: (robocop) robocop.apk will not be installed"

these are not really remote device errors, they would be 'Automation Error'

@@ +89,2 @@
>                                                                        str(matches))
> +            print "Remote Device Error: (robocop) robocop.apk will not be installed"

Won't this occur when we are testing without robocop.apk?  This really seems problematic.

Comment 5

[Ed Morley [:emorley]]

Attachment: Patch v2

As before except without the changes to the robocop.apk warnings (I had forgotten we indiscriminately display that warning in non-robocop runs).

Those strings weren't the ones I was most fussed about anyway; the bug 781419 case is the primary goal - I was just trying to do some others whilst I was there :-)

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 660890 [details] [diff] [review]
Patch v2

Review of attachment 660890 [details] [diff] [review]:
-----------------------------------------------------------------

this looks pretty good.

Comment 7

[Ed Morley [:emorley]]

Comment on attachment 660890 [details] [diff] [review]
Patch v2

Thank you :-)

http://hg.mozilla.org/build/tools/rev/709ef2c85ac7

Comment 8

[Justin Wood (:Callek)]

(In reply to Ed Morley (Away 18th-20th) [:edmorley UTC+1] from comment #7)
> Comment on attachment 660890 [details] [diff] [review]
> Patch v2
> 
> Thank you :-)
> 
> http://hg.mozilla.org/build/tools/rev/709ef2c85ac7

Deployed to all foopies just now.

Comment 9

[Ed Morley [:emorley]]

(In reply to Justin Wood (:Callek) from comment #8)
> Deployed to all foopies just now.

Thank you, working great on TBPL now :-)