Process crashes in wpt logs are not marked as failures
Categories
(Testing :: web-platform-tests, enhancement)
Tracking
(Not tracked)
People
(Reporter: bryce, Unassigned)
Details
While looking at Bug 1522547 I noticed that we're not marking some crashes as failures on treeherder. Compare this log, which is prior to the fix landing, to this log, form after the fix landed. In both cases treeherder is showing a pass, but in the first log there are clearly crashes.
I'm uncertain as if there's something more going on here, but this seemed unexpected to me. I'm also not sure if there's a better way to save these logs on the bug, since I imagine that they'll expire from treeherder at some point.
|
||
Comment 1•6 years ago
|
||
I think I saw a change that deliberately started ignoring crashes. Maybe now that bug 1522547 has landed this can be reverted? Or is it still needed for new imports that cause crashes?
|
||
Comment 2•6 years ago
|
||
Note that we can mark things as expected: CRASH in the metadata, in which case the crash won't fail the job. Without that we couldn't import new tests when there's a crash. So in general crashes can't always set the status to fail.
However what seems to be going on here is that we get a test result back from the browser and then later when we check for a crash file we find one. So ideally we'd change the test status to crash there, and for a while we did that, but it was reverted in https://bugzilla.mozilla.org/show_bug.cgi?id=1519862 because we were seeing tests flip between CRASH and ERROR. There might be some way to fix that, but doing so is the prerequisite for making this kind of crash-after-the-test-ends able to fail the job.
|
||
Updated•3 years ago
|
Description
•