Browser crashes not reported as test failures

RESOLVED DUPLICATE of bug 808410

Status

Testing
Mochitest
RESOLVED DUPLICATE of bug 808410
5 years ago
5 years ago

People

(Reporter: kats, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

STR:
1. Go to https://tbpl.mozilla.org/?rev=90cea19e27e2
2. Click on the second "rc" under "Android 2.2 NoIon opt" (note that it is green, indicating all tests passed). View the full log (this should take you to https://tbpl.mozilla.org/php/getParsedLog.php?id=16893601&tree=Firefox&full=1)

Expected results:
- No failures in log

Actual results:
testHistoryTab has a big ol' crash dump, and the tbpl log parser even identifies this as a failure.

Why does the tbpl log parser find a failure in the log but the tbpl test summary not report this as a failure? Which code is *actually* responsible for reporting test results?
to turn a job orange we need a return code from the harness.  Our handling of return codes is not going to get us orange:
http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/runtestsremote.py#467

Lets assume the exception is handled and we still fail.  We need to fix the return value in printLog to ensure we have the proper summary:
http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/runtestsremote.py#520

We are calling CheckForCrashes, we should make that routine return a value so we can check if there were any crashes.

Comment 2

5 years ago
I'm happy to fix this as part of bug 808410 if that helps? :-)
it sounds like that might be the right place to fix this.

Updated

5 years ago
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 808410
You need to log in before you can comment on or make changes to this bug.