1399993 - Change log level for MOZ_CRASH lines from INFO to ERROR

Status: NEW

(Testing :: General, enhancement)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to Ed Morley [:emorley] from bug 1261598 comment #185)
> (In reply to Henrik Skupin (:whimboo) from bug 1261598 comment #181)
> > Shouldn't we also add MOZ_CRASH to the error list, so that those cases are
> > getting reported correctly on Treeherder? Otherwise we will get falsely
> > starred test failures like in the above case. Ed, can you give feedback?
> 
> I agree this sounds useful to have marked as an error (presuming it would
> result in no false positives). However we're trying to move away from
> hardcoding error strings in Treeherder, so I would prefer this to be
> addressed by making the mozharness log level prefix be ERROR rather than the
> current INFO. eg:
> 
> [task 2017-09-02T11:53:03.283917Z] 11:53:03     ERROR - GECKO(2785) | Hit
> MOZ_CRASH(...

Ted, I assume this wouldn't lead to false positives, or?

Comment 1

[(not currently active) Ted Mielczarek]

We originally made check_for_crashes print PROCESS-CRASH instead of TEST-UNEXPECTED-FAIL in bug 539290 so that crashes from child processes (in tests where we expected to crash the child process) would not cause test failures:
https://hg.mozilla.org/mozilla-central/rev/e828ac500665#l2.12

These days test harnesses have methods to mark crashes as expected, and the harness cleans up ahead of time, so that shouldn't be an issue.

Comment 2

[Chris AtLee [:catlee]]

So we should be able to make this change in mozharness easily then?

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I'm actually not the right person to ask if that is easy or not. Aki, mind giving your feedback?

Comment 4

[Aki Sasaki (not active)]

I don't know which script(s) we're talking about here, but adding it to any `run_command` call should be fairly straightforward.

For instance, this is the line that marks PROCESS-CRASH as an error: https://hg.mozilla.org/mozilla-central/file/tip/testing/mozharness/mozharness/mozilla/testing/errors.py#l117

then the `run_command` call that runs the command that prints MOZ_CRASH would need to use the appropriate error list.