Closed Bug 816532 Opened 9 years ago Closed 9 years ago

Print logcat for failed robocop tests

Categories

(Firefox for Android Graveyard :: General, defect)

x86
Android
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED
Firefox 20

People

(Reporter: gbrown, Assigned: emorley)

References

Details

Attachments

(1 file)

Robocop tests collect and display logcat data at the end of a test run. When a test fails or crashes, the error is reported, then the next test is run...and only when all tests have completed is the logcat collected and printed in the log. Usually, the logcat only shows activity for the most recent (last) test. 

An example:

https://tbpl.mozilla.org/php/getParsedLog.php?id=17440427&full=1&branch=mozilla-inbound

It would be more useful if the logcat was dumped whenever an error is detected, or if the logcat for the entire run was collected.
Blocks: 817518
Attached patch Patch v1Splinter Review
Something like this?

This will always output the logcat/device info at the end of the entire run - and for robocop, will additionally output it after each individual runTests, if the exit code was non-zero. (I think dumping the logcat after every robocop runTests will be too verbose).

The only problem with this is if the non-zero return code happened on the last runTests, we'll end up dumping near-identical logcat twice. Does this matter?
Assignee: nobody → edmorley.bz
Status: NEW → ASSIGNED
Attachment #687790 - Flags: review?(gbrown)
Comment on attachment 687790 [details] [diff] [review]
Patch v1

Review of attachment 687790 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks much Ed -- this is just what I had in mind. I would not worry about the double logcat on last test failure issue -- I don't think it will bother or confuse anyone.
Attachment #687790 - Flags: review?(gbrown) → review+
https://hg.mozilla.org/mozilla-central/rev/198a2cd21e1b
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 20
This was a good fix, but I think we did not go far enough. We dump logcat when runTests returns an error, but robocop failures often do not return an error: we still see many robocop failures without logcats.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
I don't suppose you have some links?

The root cause is the harness not outputting the correct error code surely?

I guess we could always dump the logcat, but that's going to get bloated pretty quickly I think.
Examples:
https://tbpl.mozilla.org/php/getParsedLog.php?id=17851959&tree=Firefox&full=1#error0
https://tbpl.mozilla.org/php/getParsedLog.php?id=17881486&tree=Firefox&full=1#error0

I would prefer to dump the logcat only on error...I think the thing to do is tighten up our error detection.
Depends on: 822823
Since this is an orthogonal problem to the one fixed in this bug, re-closing in favour of the bug I've filed for this (bug 822823).
Status: REOPENED → RESOLVED
Closed: 9 years ago9 years ago
Resolution: --- → FIXED
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in before you can comment on or make changes to this bug.