reftest "This test left crash dumps behind" failure doesn't actually count as a failure

RESOLVED FIXED in mozilla17

Status

defect
RESOLVED FIXED
8 years ago
7 years ago

People

(Reporter: dbaron, Assigned: heycam)

Tracking

(Blocks 1 bug)

Trunk
mozilla17
x86_64
macOS
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

The reftest failure:

REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/talos-slave/test/build/reftest/tests/dom/plugins/test/reftest/pluginproblemui-direction-1.html | This test left crash dumps behind, but we weren't expecting it to!

(as is currently occuring for Mac OS X 10.7 (Lion) reftests; see bug 705047) is printing to the log, but it does not do anything to:
 1. cause the number of failures reported to increase
 2. cause the suite to be considered failing if it's the only failure
Things that report TEST-UNEXPECTED-FAIL should do both of these things.

This bug was introduced in http://hg.mozilla.org/mozilla-central/rev/38b18b59e267
Assignee

Updated

8 years ago
Assignee: nobody → cam
Status: NEW → ASSIGNED
Comment on attachment 578970 [details] [diff] [review]
Count unexpected reftest crash dump files as test failures.

r=dbaron

But, in order to land this, you either need to re-hide the Mac OS X 10.7 reftests or fix bug 705047 first.
Attachment #578970 - Flags: review?(dbaron) → review+
Assignee

Updated

8 years ago
Blocks: 708165
No longer blocks: 705047
So what needs to be fixed in order to land this?  And is there any progress on it?
Assignee

Comment 4

7 years ago
It has been a while, so it might be worth pushing the patch to try and seeing what test failures due to un-cleaned up crash dumps remain.
Assignee

Comment 6

7 years ago
The test run is fine, but it turns out the only reftests to deliberately cause plugin crashes (dom/plugins/test/reftest/pluginproblemui*) have been permanently disabled.  I think we can just land this, and if we want to re-add some tests to handle deliberate plugin crashes we can finish off bug 708165 at that point.
https://hg.mozilla.org/mozilla-central/rev/fdc76cb3b523
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla17
You need to log in before you can comment on or make changes to this bug.