1831106 - 'mh' builder doesn't expose testing/mozharness/test/ failures to treeherder short summaries

Status: NEW

(Release Engineering :: Applications: MozharnessCore, defect)

Description

[Mark Banner (:standard8)]

I have a patch that is causing errors in test_malformed_json in testing/mozharness/test/test_base_config.py.

When looking through the mh failure output on treeherder, there is nothing which indicates that that specific test has failed:

unable to open nonexistent_file!!!: No such file or directory
<bound method TestRetry._alwaysFail of <test_base_script.TestRetry testMethod=testRetryFailEnsureRaisesLastException>> failed after 5 tries!
Caught exception: Fail
Running post_fatal callback...
Exiting -1
Testing ERROR
Testing CRITICAL
Testing FATAL
Running post_fatal callback...
Exiting -1
Exception during post-action for clobber: Traceback (most recent call last):
File "/builds/worker/checkouts/gecko/testing/mozharness/mozharness/base/script.py", line 2339, in run_action
method(action, success=success and self.return_code == 0)
File "/builds/worker/checkouts/gecko/testing/mozharness/test/test_base_script.py", line 834, in post_action_1
raise Exception(self.raise_during_post_action_1)
Exception: Error during post 1
Aborting due to failure in post-action listener.
Running post_fatal callback...
Exiting -1 

Within the log, I see this buried away:

[task 2023-05-03T10:24:54.924Z] test_malformed_json (test_base_config.TestParseConfigFile) ... FAILED