1611165 - Crash under ASAN fails job even if the crash is "expected"

Status: NEW

(Testing :: web-platform-tests, defect, P3)

Description

[James Graham [:jgraham]]

See https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=286141140&repo=autoland&lineNumber=17160

Because these are logged at ERROR level, they cause mozharness to regard the job as a fail, even though the crash is marked as "expected" in metadata.

I'm not sure what the best solution is here. mccr8: thoughts? In the short term fixing the crash is obviously the best option, but we may just need to disable these tests under ASAN if the crash fix isn't straightforward. I'm not sure what to do in the longer term though. We could try to suppress the error in the case that a crash is expected, or just accept that this gives us a chance to see the crash and with the builds being T2 we have some time to investigate before disabling the tests.

Comment 1

[Andrew McCreight [:mccr8]]

I don't really know anything about the mechanism that marks these as failing. Maybe it could be changed to work more like the LSan failures so we can choose to ignore things?

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The priority flag is not set for this bug.
:jgraham, could you have a look please?

For more information, please visit auto_nag documentation.